To solve problems in dissimilarly light metal joints, refilled friction stir spot welding (RFSSW) is proposed instead of resistance spot welding. However, rotation speed, dwell time, plunge depth, and the diameter of welding tools all have a great influence on joints, which brings great challenges in optimizing welding parameters to ensure their mechanical properties. In this study, the 1.5 mm thick 2A12Al and 2 mm thick 7B04Al lap joints were prepared by Taguchi orthogonal experiment design and RFSSW. The welding tool (shoulder) diameters were 5 mm and 7 mm, respectively. The macro/microstructures of the cross-section, the geometrical characteristics of the effective welding depth (EWD), the stir zone area (SZA), and the stir zone volume (SZV) were characterized. The shear strength and failure mode of the lap joint were analyzed using an optical microscope. It was found that EWD, SZA, and SZV had a good correlation with tensile–shear force. The optimal welding parameters of 5 mm diameter joints are 1500 rpm of rotation speed, 2.5 mm of plunge depth, and 0 s of dwell time, which for 7 mm joints are 1200 rpm, 1.5 mm, and 2 s. The tensile–shear force of 5 mm and 7 mm joints welded with these optical parameters was 4965 N and 5920 N, respectively. At the same time, the 5 mm diameter joints had better strength and strength stability.