Untangling the multiple drivers that affect biodiversity along elevation gradients is crucial for predicting the consequences of climate change on mountain ecosystems. However, the distribution patterns of microorganisms along elevation gradients have not yet been clarified, in particular when associated with strong changes in dominant species. Five typical vegetation types (i.e., coniferous forests, meadow grasslands, alpine shrubs, alpine meadows, and sparse vegetation of limestone flats) from contrasting vegetation belts were selected to explore the influence of elevation gradients on soil microbial communities. The results showed that Actinobacteriota and Proteobacteria were the dominant bacterial phyla. Ascomycota and Basidiomycota were the prevalent fungal phyla. Soil bacterial alpha diversity increased with increasing elevation, while soil fungal alpha diversity showed an obvious mid-elevation pattern. The beta diversity of the bacterial and fungal communities reflected a clear spatial niche-differentiation, and indicated that herbaceous plants affected soil bacterial communities while shrubs preferred soil fungal communities. A correlation analysis showed that environmental factors had different contributions to the composition and diversity of soil microbial communities. Soil bacteria were primarily affected by soil properties, whereas fungi were affected by vegetation. The research results can improve the prediction of soil microbial ecological processes and patterns related to elevation, and provide a theoretical basis for maintaining the sustainable development of soil microbial biodiversity under the background of global change.