Peonies are globally renowned ornamental plants, and distant hybridization is a key method for breeding new varieties, though it often faces cross-incompatibility challenges. The metabolic mechanisms underlying the crossing barrier between tree peony (Paeonia sect. Moutan) and herbaceous peony (P. lactiflora) remain unclear. To identify key metabolites involved in cross-incompatibility, we performed a cross between P. ostii ‘Fengdanbai’ (female parent) and P. lactiflora ‘Red Sara’ (male parent) and analyzed metabolites in the stigma 12 h after pollination using UPLC-MS. We identified 1242 differential metabolites, with 433 up-regulated and 809 down-regulated, including sugars, nucleotides, amino acids, lipids, organic acids, benzenoids, flavonoids, and alkaloids. Most differential metabolites were down-regulated in hybrid stigmas, potentially affecting pollen germination and pollen tube growth. Cross-pollinated stigma exhibited lower levels of high-energy nutrients (such as amino acids, nucleotides, and tricarboxylic acid cycle metabolites) compared to self-pollinated stigma, which suggests that energy deficiency is a contributing factor to the crossing barrier. Additionally, cross-pollination significantly impacted KEGG pathways such as nucleotide metabolism, purine metabolism, and vitamin B6 metabolism, with most metabolites in these pathways being down-regulated. These findings provide new insights into the metabolic basis of cross-incompatibility between tree and herbaceous peonies, offering a foundation for overcoming hybridization barriers in peony breeding.