The selection of process parameters for an ultra-high-pressure water jet directly affects the adhesion ability of the subsequent coating on the ship’s surface. This study investigates the effect of jet pressure, standoff distance, and nozzle traverse speed on the surface roughness of grade-A marine steel, aiming to optimize the process parameters and improve the quality of surface treatment. Based on single-factor experiments and orthogonal experiments, a three-factor, three-level experimental design was employed, considering jet pressure, standoff distance, and nozzle traverse speed. Scanning electron microscopy (SEM) and a confocal microscope were used to analyze the surface morphology and roughness of grade-A marine steel. The experimental results proved that surface roughness exhibited a nonlinear relationship with jet pressure, initially increasing and then decreasing as pressure rose. Conversely, surface roughness showed negative correlations with both standoff distance and nozzle traverse speed, progressively decreasing with increases in these parameters. Through hierarchical analysis, the effect hierarchy of the three factors on surface roughness was determined as follows: jet pressure > standoff distance > nozzle traverse speed. Parametric optimization revealed that a jet pressure of 150 MPa, a standoff distance of 25 mm, and a nozzle traverse speed of 180 mm/min collectively yielded a peak surface roughness of 62.549 μm. This value aligns with the pre-coating surface preparation standards for grade-A marine steel substrates, ensuring optimal adhesion for subsequent anti-corrosion treatments.