Soil degradation and erosion pose significant threats to agricultural sustainability in fragile ecosystems, such as the Loess Plateau in northern China. This study examines the long-term impacts of fertilization regimes and land-use systems on soil health, focusing on soil aggregate stability, fertility, and crop productivity. Six treatment combinations were evaluated in our study, including three continuous alfalfa fields (AL-CK, AL-P, and AL-NPM) and three continuous wheat fields (WH-NPM, WH-NP, and WH-P), each representing a combination of land use and three fertilization treatments: (1) no fertilization (CK), (2) inorganic fertilization (120 kg ha⁻¹ N, 60 kg ha⁻¹ P-NP), and (3) a combination of organic and inorganic fertilization (75 t ha⁻¹ cow manure-NPM). Soil samples were collected from three depths (0–10 cm, 10–20 cm, and 20–30 cm) to assess physical and chemical properties. We evaluated the long-term effects of different fertilization treatments on soil stability, fertility, and crop yield to explore the interactions among soil’s physical and chemical properties under two land-use types and to assess the effectiveness of combined organic and inorganic fertilization strategies in improving soil health and mitigating erosion in vulnerable landscapes. The study revealed significant depth-specific variations with surface layers (0–10 cm) showing the greatest improvement under NPM treatments, particularly in continuous alfalfa fields, which exhibited higher soil fertility, improved soil structure, and crop yield. In contrast, continuous wheat fields with minimal fertilization demonstrated significantly lower soil quality and productivity. Using the combination of mineral fertilizers and organic amendments, such as cow manure, proved to be the most effective strategy for significantly enhancing nutrient availability and overall soil health. Partial Least Squares Modeling (PLS-M) and Mantel analysis highlighted the critical role of fertilization management in maintaining soil quality, boosting crop productivity, and mitigating erosion in high-risk areas. This study emphasizes the importance of integrated nutrient management for sustainable land use and soil conservation in erosion-prone regions.