Long-term reliance on mineral fertilizers may degrade soil structure and weaken carbon and nitrogen retention in rice–wheat rotation systems. Organic amendments combined with mineral fertilizers can provide organic substrates, promote aggregate formation, and improve soil C–N retention, but the contrasting effects of plant- and animal-derived amendments remain unclear. A long-term field experiment initiated in 2012 in Chaohu, China, compared no fertilization (CK), chemical fertilizer alone (CF), chemical fertilizer plus oil cake (OC), and chemical fertilizer plus cattle manure (CM). At wheat maturity in 2025, soil aggregate distribution, stability, aggregate-associated soil organic carbon (SOC) and total nitrogen (TN), C and N contribution rates, and yield components were determined; wheat yield was further evaluated using annual records from 2021 to 2025. Compared with CK, OC and CM increased water-stable macroaggregates (>0.25 mm) by 17.35% and 17.94% and reduced aggregate destruction by 54.61% and 53.57%, respectively. In large macroaggregates (>2 mm), OC and CM increased SOC by 58.30% and 84.83% and TN by 141.76% and 200.00%, respectively. Five-year mean yield increased by 208.09%, 240.78%, and 225.15% under CF, OC, and CM, respectively, relative to CK, but did not differ significantly among fertilized treatments. Overall, oil cake showed a numerical advantage in maintaining wheat yield, whereas cattle manure had greater potential to improve aggregate-scale C–N retention and soil structural stability.