In November 2023, surface sediments were collected at 46 sites around the main islands of the Zhoushan Archipelago (Dinghai, Daishan, Qushan, and Shengsi) in the East China Sea. The concentrations of Cu, Zn, Cr, Pb, Cd, Hg, and As were determined, together with sediment TOC and Eh. Pollution and ecological risks were evaluated using the single-factor index (Pi), Nemerow pollution index (PN), and Hakanson’s potential ecological risk index (RI). Source apportionment was investigated using FA–PC and EPA PMF 5.0. Mean concentrations (mg/kg) were Zn 77.58, Cr 70.08, Cu 28.44, Pb 18.92, As 9.40, Cd 0.09, and Hg 0.073, with higher levels generally observed near Dinghai, Daishan, and Shengsi. The overall risk was low, whereas Cd and Hg contributed disproportionately to RI. FA–PC suggested two major source groups, and PMF resolved three factors related to (i) agriculture/aquaculture (As), (ii) industrial and domestic effluents (Hg), and (iii) port and ship-related activities (Cd, Cu, Cr, Pb, Zn). The results support targeted management focusing on Cd, Hg, Cu, and As in identified hotspots.