This study aims to further analyze the chemical characteristics of mountainous forest cultivated ginseng (MFCG) and garden ginseng (GG), concerning their calcium oxalate crystals, organic acids, and ginsenosides. The results demonstrate that MFCG had higher levels of non-free oxalate, calcium oxalate crystals, and most ginsenosides, while GG had higher fumaric acid/total organic acids. The content of non-free oxalate and calcium oxalate crystals in rhizome was the highest, showing a positive correlation with the growth years (5–20 years). In most cases, in MFCG, non-free oxalic acid ≥ 0.8%, calcium oxalate ≥ 160/mg, fumaric acid/total organic acids < 9%, Rb1 ≥ 6 mg/g, PPD/PPT was close to 2, and Rb1/Ro ≥ 2.5, while in GG, non-free oxalic acid < 0.8%, calcium oxalate ≤ 60/mg, fumaric acid/total organic acids ≥ 9%, Rb1 < 6 mg/g, PPD/PPT was close to 1, and Rb1/Ro < 2.5. These results can be used as the basis for distinguishing between GG and MFCG. Chemometric analysis of non-free oxalate, calcium oxalate crystals, and ginsenosides could distinguish MFCG from GG. Chemometric analysis of succinate, citrate, and malonic acids could mostly differentiate MFCG of over 15 years from that of less than 12 years. As far as we know, the present study is the first to determine the difference in the ratio of ginsenosides (Rb1/Ro, PPD/PPT) and the ratio of organic acids, which provides an innovative method for the distinction between the two and a scientific basis for effective quality control of MFCG.