This study aims to quantify concentrations of minerals and trace elements in human milk (HM) and infant formula (IF) and evaluate associations with medical, social, environmental, and demographic variables. A prospective, case series study of 170 nursing mothers was made. HM samples were obtained from full-term (colostrum, intermediate and mature HM) and preterm (mature HM) mothers. Variables of interest were assessed by a questionnaire. For comparison, IF samples (n = 30) were analyzed in a cross-sectional study. Concentrations of 35 minerals, essential and toxic trace elements were quantified, 5 for the first time: thallium in HM and IF; strontium in preterm HM; and gallium, lithium and uranium in IF. In preterm and full-term HM, levels of selenium (p < 0.001) were significantly lower than recommended and were associated with low birth weight (p < 0.002). Cesium and strontium concentrations were significantly higher than recommended (p < 0.001). Associations were observed between arsenic and residence in an urban area (p = 0.013), and between lead and smoking (p = 0.024) and well-water consumption (p = 0.046). In IF, aluminum, vanadium, and uranium levels were higher than in HM (p < 0.001); uranium, quantified for the first time, was 100 times higher in all types of IF than in HM. Our results indicate that concentrations of most trace elements were within internationally accepted ranges for HM and IF. However, preterm infants are at increased risk of nutritional deficiencies and toxicity. IF manufacturers should reduce the content of toxic trace elements.