This study aimed to examine the physical fitness (PF) levels of primary school children and to determine the associations among PF, concentration, and health-related quality of life (HRQOL) in a subcohort. PF was assessed in 6533 healthy primary school-age children (aged 6–10 years, 3248 boys and 3285 girls) via standardized test batteries. Concentration was measured with the d2-R test, and KINDL questionnaires were used to determine HRQOL. Analysis of variance showed an increase in PF with age in all PF dimensions (all p < 0.001), except cardiopulmonary fitness (estimated VO2max) in girls (p = 0.129). Boys performed better in nearly all PF dimensions, except curl-ups, in all children aged ≥7 years (p < 0.05). Concentration levels increased in boys and girls aged 7–9 years (p < 0.001), whereas HRQOL did not (p = 0.179). The estimated VO2max had a strong impact on concentration (β = 0.16, p < 0.001) and HRQOL (β = 0.21, p < 0.001) in 9- to 10-year-olds. Cardiopulmonary fitness is important for improved concentration and better HRQOL in primary school-age children. However, longitudinal data are needed to provide further insight into the intraindividual relationships of PF and concentration over the course of child development and set up targeted prevention programs.