Understanding the components of water consumption plays a critical role in agricultural management in arid regions. This study aimed to analyze the characteristics of the components of the water consumption of winter wheat on the Loess Plateau in China to investigate their effects on yield and water use efficiency (WUE). Winter wheat observation data were collected from 1981 to 2020 at the Xifeng Agrometeorological Station on the Loess Plateau. The results show that over the past 40 years, the average water consumption of the winter wheat fields was 315 mm, but there were large differences between years. The soil water was first converted from precipitation (P) during the growing season, accounting for 69.4%, and then consumed via soil water storage (ΔW) in the fallow period, accounting for 30.6%. The yield of winter wheat varied from 1057 to 6914 kg·ha⁻¹. The correlation between winter wheat yield and P during the growth period was stronger than the correlation between winter wheat yield and ΔW. The average WUE was 11.0 kg·ha⁻¹·mm⁻¹ from 1981 to 2020, with the highest value of 13.7 kg·ha⁻¹·mm⁻¹ occurring in the 2010s and the lowest value of 9.6 kg·ha⁻¹·mm⁻¹ occurring in the 2000s. The WUE was positively correlated with P (p < 0.01) during the growth period and negatively correlated with ΔW (p < 0.05). Therefore, P can increase yield and WUE more effectively than ΔW. These findings provide a theoretical basis for the efficient utilization of water resources on the Loess Plateau and the sustainable development of dry farming.