The summer maize yields and partial factor productivity of nitrogen (N) fertilizer (PFP_N, grain yield per unit N fertilizer) on smallholder farms in China are low, and differ between farms due to complex, sub-optimal management practices. We collected data on management practices and yields from smallholder farms in three major summer maize-producing sites—Laoling, Quzhou and Xushui—in the North China Plain (NCP) for two growing seasons, during 2015–2016. Boundary line analysis and a Proc Mixed Model were used to evaluate the contribution of individual factors and their interactions. Summer maize grain yields and PFP_N ranged from 6.6 t ha⁻¹ to 14.2 t ha⁻¹ and 15.4 kg kg⁻¹ to 96.1 kg kg⁻¹, respectively, and averaged 10.5 t ha⁻¹ and 49.1 kg kg⁻¹, respectively. The mean total yield gap and PFP_N gap were 3.6 t ha⁻¹ and 43.3 kg kg⁻¹ in Laoling, 2.2 t ha⁻¹ and 24.5 kg kg⁻¹ in Xushui, and 2.8 t ha⁻¹ and 41.1 kg kg⁻¹ in Quzhou. A positive correlation was observed between the yield gap and PFP_N gap; the PFP_N gap could be reduced by 6.0 kg kg⁻¹ (3.6–6.6 kg kg⁻¹) by reducing the yield gap by 1 t ha⁻¹. The high yield and high PFP_N (HH) fields had a higher plant density and lower N fertilization rate than the low yield and low PFP_N (LL) fields. Our results show that multiple management factors caused the yield gap, but the relative contribution of plant density is slightly higher than that of other management practices, such as N input, the sowing date, and potassium fertilizer input. The low PFP_N was mainly attributed to an over-application of N fertilizer. To enhance the sustainable production of summer maize, the production gaps should be tackled through programs that guide smallholder farmers on the adoption of optimal management practices.