This paper studies the influencing factors on the magnetic material (nickel-phosphorus coated SKD11 steel) polishing process were determined by investigating the distribution of magnetic iron (MIGs) and abrasive grains (AGs) in the working surface of magnetic liquid slurry (MLS). The influence of the factors, namely the diameters of the MIGs and the AGs and working distance (K), on the polishing surface quality was assessed by fuzzy gray analysis. Results showed that the values of the fuzzy gray relationships with the factors were 0.6361, 0.5181, 0.8804, and 0.6464. Result showed that the fuzzy gray relationship point of factors known as K exerted the most remarkable influence on the polishing effect, whereas the effect of MIG diameter was insignificant. The experimental results indicated that MLS containing MIGs with a diameter of 5 μm, AGs with a diameter of 1.5 μm and K = 1 mm should be prepared to create a mirror surface with nickel–phosphorus coated SKD11 steel. An extremely smooth mirror surface with Ra = 0.926 nm is produced under these experimental conditions, leaving no scratches and impurities on the machined surface. The proposed polishing method that uses MLS with MIGs and AGs (which are low-cost and commercially available) is suitable and effective for polishing the surface of magnetic materials with extremely high accuracy.