Heavy metal pollution in sediment is one of the most serious problems in water bodies, including rivers, which can cause secondary pollution when environmental conditions change. In this study, surface sediment samples collected from the four main tributaries of Dongting Lake (i.e., Xiangjiang River (XR), Zishui River (ZR), Yuanjiang River (YR), and Lishui River (LR)) were analyzed for concentrations of Zn, Cr, Cu, As, Cd, and Pb. The spatial distribution, source, and potential ecological risk of these metals were determined. The results suggest a great spatial heterogeneity of heavy metals in the sediment of the studied rivers. Heavy metals had highest concentrations in the sediment of XR, especially midstream and downstream. A principal component analysis (PCA) and correlation analysis indicated that Cd and As were mainly from industrial wastewater and mineral mining, Cr came from natural process and agricultural activities, and Zn and Cu potentially from both. Pb was originated from atmospheric deposition and river inflow transportation. According to the geo-accumulation index ( ${\mathrm{I}}_{geo}$ ), enrichment factor (EF), and risk index (RI) assessment, heavy metals pollution was highest in the sediment of XR, and Cd was the main pollutant in the sediment of XR, presenting considerable potential ecological risk. This may contribute to heavy metal pollution in Dongting Lake. This paper provides a reference for the aquatic environmental management of heavy metals in Dongting Lake area and its tributaries.