The association of metabolic syndrome (MetS) components with elevated serum alanine aminotransferase (ALT) levels, a marker of hepatic injury, may differ between men and women. However, the sex-specific association in a military young population which has a low prevalence of MetS was unclear. We conducted a cross-sectional examination in 6738 men and 766 women, aged 18–50 years, from the cardiorespiratory fitness study in armed forces (CHIEF) in eastern Taiwan. The components of MetS were defined according to the updated International Diabetes Federation (IDF) ethnic criteria for Asians. Elevated ALT levels were defined as ≥40 U/L for both sexes and ≥30 U/L for women alternatively. Multivariate logistic regression analysis was performed to determine the sex-specific association between MetS components and elevated ALT. The prevalence of MetS and elevated ALT in men were 11.9% and 12.7% respectively, and in women were 3.5%, and 3.8% respectively. In men, high-density lipoprotein < 40 mg/dL, blood pressures ≥ 130/85 mmHg, serum triglycerides ≥ 150 mg/dL, and waist size ≥ 90 cm were associated with elevated ALT (odds ratios (OR) and 95% confidence intervals: 1.59 (1.34–1.90), 1.40 (1.19–1.65), 2.00 (1.68–2.39), and 1.68 (1.38–2.04); all p < 0.001); whereas in women, only fasting plasma glucose ≥ 100 mg/dL was associated with elevated ALT ≥ 40 U/L (OR: 7.59 (2.35–24.51), p = 0.001) and ALT ≥ 30 U/L (2.67 (0.89–7.95), p = 0.08). Our findings suggest that the relationship between metabolic abnormalities and elevated ALT may differ by sex, possibly due to the MetS more prevalent in young adult men than in women.