Seven heavy metals (Cr, Mn, Co, Ni, Cu, Zn, Pb) were measured in surface sediments from the Lishui River watershed, an area with increased soil erosion in China. The mean concentrations of heavy metals were 61.20 mg/kg (Cr), 757.15 mg/kg (Mn), 9.39 mg/kg (Co), 25.31 mg/kg (Ni), 22.84 mg/kg (Cu), 91.66 mg/kg (Zn), and 40.19 mg/kg (Pb), respectively. The spatial distribution of heavy metals was site-specific, exhibiting a remarkably high level in the sampling stations with intense agricultural activities (Lixian) and industrial activities (Jinshi). Contamination indexes including contamination factor, pollution load index, nemerow multi-factor index, potential ecological risk index, and human health risk were used to assess the pollution degree of the river sediments. The results indicated the pollution degree of heavy metals decreased in the order of Mn > Pb > Zn > Cr > Cu > Ni > Co. Heavy metals resulted in non-pollution to moderate pollution, with low ecological risk and an acceptable carcinogenic risk caused by Cr and Ni for children and adults. Person’s correlation analysis and principal component analysis, coupled with cluster analysis, revealed that the sediments from the Lishui River were mainly influenced by two sources. Cr, Co, Ni, and Cu were mainly derived from natural sources, while Mn, Zn, and Pb originated from agricultural and industrial activities, mining, and vehicular traffic.