Purpose: Increasing evidence indicated that thyroid-associated ophthalmopathy (TAO) might be a neural related disease more than an ocular disease. In this study, we aimed to investigate the alterations of structural brain connectome in patients with TAO. Methods: Twenty-seven patients with TAO and 27 well-matched healthy controls underwent diffusion tensor imaging. Graph theoretical analyses, including global (shortest path length, clustering coefficient, small-worldness, global efficiency, and local efficiency) and nodal (nodal betweenness, nodal degree, and nodal efficiency) topological properties and network-based statistics were performed to evaluate TAO-related changes in brain network pattern. Correlations were assessed between the network properties and clinical variables, including disease duration, visual acuity, neuropsychiatric measurements, and serum thyroid function indexes. Results: Compared with healthy controls, patients with TAO exhibited preserved global network parameters but altered nodal properties. We found decreased nodal betweenness and nodal degree in right anterior cingulate and paracingulate gyri, decreased nodal degree and nodal efficiency in the right orbital part of middle frontal gyrus (ORBmid), whereas increased nodal degree and nodal efficiency in the left cuneus. Decrease of structural connectivity strength was found involving the right ORBmid, right putamen, left caudate nucleus, and left medial superior frontal gyrus. Significant correlations were also found between nodal properties and neuropsychological performances as well as visual acuity. Conclusions: Patients with TAO developed disruption of structural brain network connectome. Disrupted topological organization of the brain structural network may be associated with the clinical-psychiatric dysfunction of patients with TAO.