Recent studies have shown that climate change is having severe impacts on production of the tea plant [Camellia sinensis (L.) Kuntze], an important woody cash crop. Quantitative production figures are necessary to provide tea stakeholders and policy makers with evidence to justify immediate action. We used data from Zhejiang Province, China, and the economic output model of four tea plant cultivars (Jiaming 1, Longjing 43, Baiye 1, and Jiukeng) in spring and the yield output model of one tea plant cultivar (Jiukeng) in summer and autumn, and analyzed the impact of climate change on tea production from 1985 to 2018. The results showed that the effects of high temperature and drought on tea production have increased significantly. Temperature was the main factor effecting the economic output of tea plants in spring. Climate change has resulted in the start date of tea buds and leaves plucking period to have become significantly earlier in spring. The risk of frost damage and economic loss, caused by frost, decreased significantly in the period of spring tea production. Among the five bud and leaf grades, the length of plucking periods for superfine, Grades 1 and 2 decreased significantly. High temperatures and drought were the main factors impacting tea production in summer and autumn, and effected the economic output of spring tea in the following year. The economic losses caused by daytime rainfall decreased significantly in spring tea production. This study has provided essential evidence that climate change has already had a significant impact on tea plant output.