Diversified cropping systems, especially those including legumes, have been proposed to enhance food production with reduced inputs and environmental impacts. However, the impact of legume pre-crops on main crop yield and its drivers has never been systematically investigated in a global context. Here, we synthesize 11,768 yield observations from 462 field experiments comparing legume-based and non-legume cropping systems and show that legumes enhanced main crop yield by 20%. These yield advantages decline with increasing N fertilizer rates and crop diversity of the main cropping system. The yield benefits are consistent among main crops (e.g., rice, wheat, maize) and evident across pedo-climatic regions. Moreover, greater yield advantages (32% vs. 7%) are observed in low- vs. high-yielding environments, suggesting legumes increase crop production with low inputs (e.g., in Africa or organic agriculture). In conclusion, our study suggests that legume-based rotations offer a critical pathway for enhancing global crop production, especially when integrated into low-input and low-diversity agricultural systems.