(1) Background: environmental gradient strongly affects microbial biodiversity, but which factors drive the diversity of arbuscular mycorrhizal fungi (AMF) associated with roots at relatively large spatial scales requires further research; (2) Methods: an experiment on large spatial scales of Mt. Taibai was conducted to explore the biodiversity and drivers of AMF-associated with roots using high-throughput sequencing; (3) Results: a total of 287 operational taxonomic units (OTUs) belong to 62 species representing 4 identified and 1 unclassified order were identified along different altitudinal gradients. With increasing altitude, AMF colonization could be simulated by a quadratic function trend, and altitude has a significant impact on colonization. AMF alpha diversity, including the Sobs and Shannon indexes, tended to be quadratic function trends with increasing altitude. The highest diversity indices occurred at mid-altitudes, and altitude had a significant effect on them. AMF communities have different affinities with soil and root nutrient, and Glomus is most affected by soil and root nutrient factors through the analysis of the heatmap. Glomus are the most dominant, with an occurrence frequency of 91.67% and a relative abundance of 61.29% and 53.58% at the level of species and OTU, respectively. Furthermore, AMF diversity were mostly associated with soil and root nutrients; (4) Conclusions: in general, AMF molecular diversity is abundant in Mt. Taibai, and altitude and nutrient properties of soil and root are the main influencing factors on AMF diversity and distribution.