The present paper probes into the influence of the steel slag particle size on the mechanical properties and microstructure of concrete, with steel slag serving as the primary raw material. Steel slag with different particle sizes was selected as the partial substitute material for concrete by mechanical grinding. The influence of steel slag on the compressive strength, bending strength, and microstructure of concrete was determined by laser particle size analyzer, specific surface area analyzer, strength experiment, X-ray diffraction (XRD), and scanning electron microscope (SEM). The results show that mechanical grinding has significant effects on the particle size distribution and specific surface area of the steel slag. The optimal grinding time is 20 min and the specific surface area is 0.65 m²/g. D10, D50 and D90 are 0.91 μm, 16.57 μm and 46.40 μm, respectively. The steel slag with a fine particle size can better fill the pores in concrete and improve the compactness, thus enhancing the mechanical properties of concrete. The change in the steel slag particle size does not change the type of hydration products, but the smaller the particle size of steel slag, the better the gelling activity, the larger the hydration products, the denser the structure, and the better the mechanical properties. Therefore, the present study provides an important theoretical basis and practical guidance for the application of steel slag as an additive in the concrete industry.