The rhizome of Atractylodes lancea (Thunb.) DC. (AL), called Maocangzhu in Chinese, is a geoherbalism medical herb in Jiangsu Province that is often used in the prescription of traditional Chinese medicine (TCM), such as for the treatment of COVID-19. The landform and climatic environment of each province varies greatly from south to north, which has an important influence on the chemical constituents in AL. However, there is a lack of research on the significance of its geoherbalism, especially in water-soluble parts other than volatile oil. In this study, eight known compounds were isolated and obtained as reference substances from AL. In addition, liquid chromatography coupled with triple-quadrupole time-of-flight tandem mass spectrometry (LC-triple TOF-MS/MS) and gas chromatography–mass spectrometry (GC-MS) were used to analyze and characterize chemical constituents from different habitats. Moreover, orthogonal partial least-squares discriminant analysis (OPLS-DA) was applied to reveal the differential metabolomics in AL from different habitats based on the qualitative information of the chemical constituents. Results showed that a total of 33 constituents from GC-MS and 106 constituents from LC-triple TOF-MS/MS were identified or inferred, including terpenoids, polyacetylenes, and others; meanwhile, the fragmentation pathways of different types of compounds were preliminarily deduced from the fragmentation behavior of the major constituents. According to the variable importance in projection (VIP) and p-values, only one volatile differential metabolite was identified by GC-MS screening: β-eudesmol. Overall, five differential metabolites were identified by LC-triple TOF-MS/MS screening: sucrose, 4(15),11-eudesmadiene; atractylenolide I, 3,5,11-tridecatriene-7,9-diyne-1,2-diacetate, and (3Z,5E,11E)-tridecatriene-7,9-diynyl-1-O-(E)-ferulate. This study provides metabolomic information for the establishment of a comprehensive quality evaluation system for AL.