The primary goals of this study were to reveal the environmental status of potentially toxic elements (PTEs) and their ecological risks, as well as their associated health risks in the Baoshan area, southwest China, which has been surveyed with the scale of 1:250,000 geochemical mapping. Based on a comparison of the PTE concentrations with the soil environmental quality of China and the enrichment factor (EF), geo-accumulation index (I_geo), contamination factor (C_f), and potential ecological risk indexes ($E_r^i$ and PERI), as well as the potential non-carcinogenic hazard indices (HI and CHI) and carcinogenic risks indices (TCR and CTCR), the following conclusions were drawn: The PTE concentrations in the surface soil samples that were collected from the investigated area (1.65% sites) exceeded the risk intervention values (RIV) for soil contamination of agricultural land of China. Cadmium (Cd) and mercury (Hg) posed higher ecological risks than other PTEs (arsenic (As), chromium (Cr), lead (Pb), copper (Cu), nickel (Ni), and zinc (Zn)), which was highlighted by their toxic response factor. Arsenic was the main PTE with a non-carcinogenic risk (19.57% sites for children and 0.25% sites for adults) and the only PTE that carries a carcinogenic risk (2.67% sites for Children and 0.76% sites for adults) to humans in the research area. Children are more vulnerable to health risks when compared to adults because of their behavioral and physiological traits. Geological genesis was responsible for the high concentrations, ecological risk, and health risk distribution patterns of the examined PTEs. Even though the present research highlights several important aspects related to PTE pollution in the research area, further investigations are needed, especially in mining areas.