This study features a survey of the concentrations of aluminum (Al) and heavy metals (Mn, Pb, Cd, Hg, As, Cr, Ni, Cu, and Zn) in tea leaves and the corresponding cultivation soils (0–30 cm), carried out in Puan County (Guizhou Province, China). The average concentrations of Al, Mn, Pb, Cd, Hg, As, Cr, Ni, Cu, and Zn in the soil were 106 × 10³, 214, 20.9, 0.09, 0.12, 17.5, 121, 27.8, 131.2, and 64 mg·kg⁻¹, respectively. The heavy metals’ pollution indexes in the soil can be ranked as follows: Cu > Cr > Hg > As > Ni > Zn > Pb > Mn > Cd. The soil was moderately polluted by Cu because of the high geochemical background value of Cu in the area. The potential environment risk index (RI) showed that 7.69% out of the total sample sites were within the moderate level. Moreover, the ranges of Al, Mn, Pb, Cd, Hg, As, Cr, Ni, Cu, and Zn concentrations in young tea leaves were 250–660, 194–1130, 0.107–0.400, 0.012–0.092, 0.014–0.085, 0.073–0.456, 0.33–1.26, 6.33–14.90, 14.90–26.10, and 35.8–50.3 mg·kg⁻¹, respectively. While in mature tea leaves, they were 4300–10,400, 536–4610, 0.560–1.265, 0.040–0.087, 0.043–0.089, 0.189–0.453, 0.69–2.91, 3.43–14.20, 6.17–16.25, and 9.1–20.0 mg·kg⁻¹, respectively. Furthermore, the concentrations of Pb, Cu, As, Hg, Cd, and Cr in young tea leaves and mature tea leaves were all lower than the standard limit values (5.0, 30, 2.0, 0.3, 1.0, and 5.0 mg·kg⁻¹ for Pb, Cu, As, Hg, Cd, and Cr, respectively) in China. Besides, the accumulation ability of tea leaves to Mn was the strongest, and the average bioconcentration factor (BCF) of Mn in mature tea leaves was 12.5. In addition, the average target hazard quotients (THQ) were all less than one for the young tea leaves and the average aggregate risk hazard index (HI) to adults was 0.272, indicating that there was not a potential health risk for adults through the consumption of the infusions brewed by young tea leaves. However, for mature tea leaves, the percentage which HI values were above one was 38.46%, and the risk to adults via the consumption of mature tea infusions were mainly contributed by Mn and Al.