Motivated by the loss of tensile strength in 9%Ni steel arc-welded joints performed using commercially available Ni-based austenitic filler metals, the viability of retaining tensile strength using an experimentally produced matching ferritic filler metal was confirmed. Compared to the austenitic Ni-based filler metal (685 MPa), higher tensile strength in gas metal arc (GMA) welded joints was achieved using a ferritic filler metal (749 MPa) due to its microstructure being similar to the base metal (645 MPa). The microstructure of hard martensite resulted in an impact energy of 71 J (−196 °C), which was two times higher than the specified minimum value of ≥34 J. The tensile and impact strength of the welded joint is affected not only by its microstructure, but also by the degree of its mechanical mismatch depending on the type of filler metal. Welds with a harder microstructure and less mechanical mismatch are important for achieving an adequate combination of tensile strength and notched impact strength. This is achievable with the cost-effective ferritic filler metal. A more desirable combination of mechanical properties is guaranteed by applying low preheating temperature (200 °C), which is a more practicable and economical solution compared to the high post-weld heat treatment (PWHT) temperature (580 °C) suggested by other research.