A ≥ 200 °C fraction (CT200F) of low-temperature coal tar was prepared by a rotary film evaporator. The catalytic conversion experiments of CT200F and six model compounds were conducted on the pyrolysis gas chromatography–mass spectrometer. The yields of catalytic conversion products benzene, toluene, xylene, and naphthalene (BTXN) were analyzed by semi-quantitative analysis according to the chromatographic peak areas. Additionally, the possible formation pathways and mechanisms of the target products BTXN generated over different catalysts were investigated. The results show that the yield of aromatic hydrocarbons increases and the yield of acid compounds decreases during CT200F pyrolysis over ZSM-5, HY, USY, and β-zeolite compared with that of its non-catalytic pyrolysis, especially the yields of BTXN obtained over USY and β-zeolite increase by 128 and 108%, respectively. The pore structure of ZSM-5 is suitable to produce BTX, while the suitable acidity and pore structure of USY, HY, and β-zeolite are more beneficial for the selective preparation of naphthalene than that of ZSM-5. The conversion pathways of six model compounds into BTXN over zeolites were obtained, and the following conclusions can be drawn: The dehydroxylation effect of zeolites shows the order of ZSM-5 > HY > USY > β-zeolite. The catalytic effect of zeolites on the cracking and ring opening of PAHs in CT200F shows the order of β-zeolite > USY > HY > ZSM-5. The catalytic effect of catalysts on the cracking and aromatization of aliphatic compounds shows the order of ZSM-5 > β-zeolite > USY > HY. β-zeolite has an outstanding catalytic performance in the conversion of PAHs into naphthalene. ZSM-5 and HY can effectively remove phenolic hydroxyl groups in phenol and naphthol. During the catalytic conversion processes of the coal tar fraction and model compounds, the catalytic effect of the pore constructions of zeolites is more important than their acidities, which determines whether large molecules can enter and whether acid sites in non-micropores can be effectively utilized.