Low productivity in albic soils often results in excessive nitrogen input, while straw return further increases nitrogen accumulation through decomposition. To address this issue, a three-year field experiment was conducted in albic soils of high, medium, and low fertility. Two nitrogen management strategies were assessed: nitrogen addition and reduction. Addition treatments included conventional nitrogen application rate alone (N), straw return (8250 kg ha⁻¹) with conventional nitrogen application rate (SN), and straw return with increased nitrogen (SN₊). Reduction treatments comprised SN and straw return with 10%, 20%, and 30% reduced nitrogen (SN_0.9, SN_0.8, and SN_0.7). Soil physical properties, nutrient content, and rice yield were evaluated. Results showed that SN_0.9 exhibited advantages in high-fertility albic soils, as it increased rice yield and improved some soil quality while reducing the nitrogen input by 10%. However, yield under SN_0.9 declined progressively over the three years, indicating limitations of long-term application. SN performed better than both N and SN₊ in medium- and low-fertility albic soils, offering better yield and soil quality improvements. However, nitrogen overaccumulation risk under continuous application should not be overlooked. These findings highlight that fertility-based nitrogen adjustment combined with straw return can simultaneously improve rice productivity and soil quality while reducing nitrogen input in albic soils.