Ligustici Rhizoma et Radix (LReR) is the dried rhizomes and roots of Ligusticum sinese Oliv. (LS) or Ligusticum jeholense Nakai et Kitag. (LJ). However, in the market, LS and LJ are frequently confused with each other. Since the volatile oils are both the main active components and quality control indicators of LReR, a strategy combining gas chromatography-mass spectrometry (GC-MS) and chemical pattern recognition (CPR) was used to compare the volatile components of LJ and LS. Total ion chromatography (TIC) revealed that phthalides (i.e., neocnidilide) and phenylpropanoids (i.e., myristicin) could be thought of as the most critical components in the volatile oils of LJ and LS, respectively. In addition, the chemical components of the volatile oils in LJ and LS were successfully distinguished by hierarchical cluster analysis (HCA) and principal component analysis (PCA). Moreover, two quality markers, including myristicin and neocnidilide, with a very high discriminative value for the classification of LJ and LS, were found by orthogonal partial least squares discriminant analysis (OPLS-DA). The relative contents of myristicin and neocnidilide were 10.86 ± 6.18% and 26.43 ± 19.63% for LJ, and 47.43 ± 12.66% and 2.87 ± 2.31% for LS. In conclusion, this research has developed an effective approach to discriminating LJ and LS based on volatile oils by combining GC-MS with chemical pattern recognition analysis.