This study systematically investigates the surface nanocrystallization of 35CrMo steel induced by Ultrasonic Surface Rolling Processing (USRP). It reveals the formation of a gradient nanostructure, where martensite lath fragmentation under high-frequency impacts leads to a surface layer of equiaxed nanocrystals and high-density dislocations. This novel microstructure yields exceptional surface integrity: roughness is minimized to 0.029 μm due to plastic flow, residual stress is transformed into high compressive stress, and surface microhardness is significantly enhanced by 32.3%, primarily governed by grain refinement and dislocation strengthening. Consequently, the treated material exhibits a 28.9% reduction in wear mass loss, which is directly attributed to the combined effects of the strengthened gradient layer’s improved load-bearing capacity and the effective suppression of crack initiation by compressive residual stresses. Our findings not only provide direct microstructural evidence for classic strengthening theories but also offer a practical guide for optimizing the surface performance of high-strength alloy components.