Rough surfaces have been widely considered as negative factors affecting cavitation erosion resistance. However, this study presented the opposite result. Here, 316L stainless steel substrates and the arc-sprayed 316L stainless steel coatings were subjected to a specific grinding process that introduced scratches on the surfaces. The surface hardness values of these ground specimens were measured to evaluate the influence of the grinding-induced strain hardening. The cavitation erosion performance of the specimens was evaluated. The results showed that rough surfaces with scratches could enhance the cavitation erosion resistance, particularly at the early stage of cavitation erosion. The scratches had a greater effect on the cavitation erosion resistance of the coatings than on the substrates. Moreover, rough surfaces with initial surface scratches could extend the incubation period of the 316L stainless steel substrates due to the inhibition of the plastic deformation. The SEM observation showed that the scratch structure of the coating surface inhibited the growth of cracks and the propagation of cavitation pits. This study could also serve as a reference for investigating the cavitation erosion behaviors of materials with a particular surface feature.