In this study, we analyzed the mechanism and the process of fungal-induced agarwood formation in Aquilaria sinensis and studied the functional changes in the xylem structure after the process. The microscopic structure of the white zone, transition zone, agarwood zone, and decay zone of 12-and 18-months of inoculation A. sinensis xylem was studied. The distribution of nuclei, starch grains, soluble sugars, sesquiterpenes, fungal propagules, and mycelium in xylem tissues was investigated by histochemical analysis. The results show that the process of agarwood formation was accompanied by apoptosis of parenchyma cells such as interxylary phloem, xylem rays, and axial parenchyma. Regular changes in the conversion of starch grains to soluble sugars, the production of sesquiterpenoids, and other characteristic components of agarwood in various types of parenchyma cells were also observed. The material transformation was concentrated in the interxylary phloem, providing a structural and material basis for the formation of agarwood. It is the core part of the production of sesquiterpenoids and other characteristic products of agarwood. Compared with the A. sinensis inoculated for 12 months, the xylem of the A. sinensis inoculated for 18 months was more vigorous. There were no significant differences between the 12 and 18 months of inoculation in terms of sugars and agarwood characteristic products. In production, harvesting after 12 months of inoculation can improve harvesting efficiency.