Phenolics in the Chinese olive (Canarium album (Lour.) Raeusch) fruit significantly affect its flavor and quality. The shikimate pathway is a bridge connecting primary metabolism and secondary metabolism through which fixed carbon can be transformed into phenolics. In this study, we aimed to reveal the relationship between the shikimate pathway and phenolic compound biosynthesis. Three Chinese olive fruits (cv. Tanxiang (TX), Changying (CY) and Lingfeng (LF)) with distinct flavor were utilized as materials. The results of this study showed that the synthesis and accumulation of quinate and gallate were active in the Chinese olive fruit. The accumulation amount of phenolic compounds was significantly different among the three cultivars. TX contained the highest content of ellagate, (iso)corilagin, conjugated quercetin and conjugated kaempferol; CY contained the highest content of conjugated luteolin; and LF contained the lowest content of ellagate, conjugated gallate, hyperin, conjugated quercetin, conjugated kaempferol and conjugated luteolin during fruit development. The expression of 3-dehydroquinate/shikimate dehydrogenase gene-4 (DHD/SDH-4), 3-dehydroquinate synthase gene (DHQS), chorismate synthase gene (CS) and Chorismate mutase gene-1 (CM-1) and shikimate content increased with the maturing of fruit. The gene 3-Deoxy-D-arabino-heptulosonate-7-phosphate synthase gene-1 (DAHPS-1) was most expressed in TX, while barely expressed in LF during fruit development. The expression of CM-1 was highest in CY. Chorismate mutase gene-2 (CM-2) expression was higher in TX and CY during late fruit development. The cultivars with higher expression of DAHPS-1 and Chorismate mutase genes (CMs) accumulated more phenolic compounds in fruit. DAHPS-1 and CMs are proposed as key genes for polyphenolic synthesis in the Chinese olive fruit. These results proved that shikimate metabolism had a positive effect on the phenols’ synthesis. Our study provides new insight into the regulatory mechanism of the biosynthesis and accumulation of phenolic compounds in the fruit of Chinese olive.