An understanding of long-term tillage and straw management impact on soil structure and productivity is necessary for the further development of conservation tillage practice in dryland farming areas. Data from a 15-year field experiment conducted in Shanxi, on the loess plateau of northern China, were used to compare the long-term effects of no-till and residue cover (NTSC) with conventional tillage (CT) in a winter wheat (Triticum aestivum L.) monoculture. Long-term CT and straw removal resulted in poor soil structure and low productivity. Mean soil bulk density in NTSC was 1.5% less than in CT and capillary porosity (2 mm was much greater for NTSC in the 0–0.20 m profile. Soil organic matter and total N and P were 27.9%, 25.6%, and 4.4% greater in NTSC, respectively, and earthworms (19/m2) were found only in the no tillage treatment. Crop yield and water use efficiency tended to be higher under NTSC than under CT, especially in the years of low rainfall, suggesting that the change in soil structure has provided a better environment for crop development. Our 15-year experimental data indicate that NTSC is a more sustainable farming system, which can improve soil structure, and increase productivity with positive environmental impacts in the rainfed dryland farming areas of northern China.