With regard to the size of China’s freshwater lakes, Taihu Lake ranks third and it plays an important role in the supply of drinking water, flood prevention, farming and navigation, as well as in the travelling industry. The problem of environmental pollution has attracted widespread attention in recent years. In order to understand the levels, distribution and sources of heavy metals in sediments of Taihu Lake, random selection was carried out to obtain 59 samples of surface sediment from the entire lake and study the concentrations of Pb, Cd, Cu, Zn, Cr and Ni. Toxic units were also calculated to normalize the toxicities caused by various heavy metals. As a result, Cd and Cu in sediment were considered lower than the effect range low (ERL) at all regions where samples were gathered, while Pb and Ni were categorized into ERL-effect range median (ERM) at over 22% of the regions where samples were obtained. Nevertheless, all average concentrations of the samples were below the level of potential effect. According to the findings of this research, significant spatial heterogeneity existed in the above heavy metals. In conclusion, the distribution areas of heavy metals with higher concentrations were mainly the north bays, namely Zhushan Bay, Meiliang Bay as well as Gonghu Bay. The distribution areas of Cu, Zn, Cr and Ni with higher concentration also included the lake’s central region, whereas the uniform distribution areas of those with lower concentrations were the lake’s southeast region. In addition, it was most probable that the spatial distribution of heavy metals was determined by river inputs, whereas atmospheric precipitation caused by urban and traffic contamination also exerted considerable effects on the higher concentrations of Pb and Cd. Through evaluating the total amount of toxic units (ΣTU), it was found that higher toxicity existed primarily in the north bays and central region of the lake. If the heavy metals were sorted by the reduction of mean heavy metal toxic units in Taihu Lake in descending order, it would be Pb, Cr, Ni, Cd, Zn and Cu. Generally speaking, these result of analyses are conducive to alleviating the contamination of heavy metals in Taihu Lake.