Compound microbial fertilizer is a new type of environmentally friendly slow-release fertilizer that can effectively improve the physical and chemical properties of the soil, significantly improve the ecological environment, and promote the sustainable development of agriculture. In this study, we conducted a field experiment to evaluate the impact of different applications of chemical fertilizer combined with composite microbial fertilizer on soil physical properties and soybean yields at Heshan Farm, Heilongjiang Province, China, during 2021–2022. Soybean varieties “Jinyuan 55” and “Keshan 1” were treated with three treatments implemented as follows: T1 (conventional fertilization), T2 (50% N fertilizer + compound microbial fertilizer), and T3 (0 N fertilizer + compound microbial fertilizer). Compared to conventional fertilization (T1 treatment), the application of composite microbial fertilizers (T2 and T3) resulted in a decrease in soil bulk density and an increase in porosity. Notably, we observed that moderate application of the composite microbial fertilizer (T3) led to a decrease in the volume fraction of clay particles and an increase in the volume fraction of sand particles. Furthermore, all treatments exhibited high content of agglomerates larger than 5 mm at 0–20 cm. The application of composite microbial fertilizers (T2 and T3) promoted the formation of large soil agglomerates and reduced the presence of micro-agglomerates smaller than 0.25 mm. In 2021–2022, The soybean yield increased by 13.02% in the T2 treatment compared with the T1 treatment and decreased by 9.34% in the T3 treatment. We concluded that the appropriate application of compound microbial fertilizer can help protect black soil, enhance the self-repair capability of black soil, and improve soybean quality in abnormal precipitation years. These results provide an actionable basis for constructing and developing green fertilizer systems for the soybean industry.