To evaluate the effects of long-term fallow tillage on soil microbial community structure in different soil layers and winter wheat yield, we conducted a 5-year long-term field experiment in the Loess Plateau, China, using three fallow tillage methods: no-tillage (NT), subsoiling tillage (ST), and deep plowing (DP). The soil physical and chemical properties, community structure, and composition of soil bacteria and fungi in the 0–20 cm and 20–40 cm soil layers, and winter wheat yield were analyzed. The results showed that, compared with DP, NT and ST significantly increased soil moisture content (SWC), soil organic carbon (SOC) content, and dissolved organic carbon (DOC) contents in 0–20 cm soil layer (p < 0.05), and significantly increased soil microbial community Shannon and Simpson index in 0–40 cm soil layer (p < 0.05). Compared with NT, ST and DP significantly increased SWC and SOC contents in 20–40 cm soil layer (p < 0.05). Actinobacteria and Ascomycota were the most abundant bacteria and fungi in the soil of the experimental site. Redundancy analysis further showed that soil physicochemical properties (SWC, SOC, DOC, and DON) were closely related to the microbial community. PICRUSt2 prediction results showed that DP increased the metabolic functional diversity of bacteria and fungi. ST and DP significantly increased the yield of winter wheat, and DP had the best effect. In conclusion, subsoiling tillage and deep plowing were beneficial to the accumulation and utilization of natural precipitation and the improvement of soil microbial community structure. Deep plowing was beneficial to the decomposition and metabolism of straw and organic fertilizer, and improved the catabolic ability of microbial community, thus increasing the yield of winter wheat.