The protective effect of physical activity on breast cancer recurrence may be mediated changes in by sex hormone levels. In this study, we examined the association between habitual physical activity and estrogen and androgen plasma levels in postmenopausal women with localised breast cancer. We conducted a cross-sectional study among 47 postmenopausal women who were breast cancer survivors with estrogen receptor-positive tumours (enrolled at the Medical Oncology Department of University Hospital Dr. Peset, Valencia, Spain). Habitual physical activity was assessed using the International Physical Activity Questionnaire (IPAQ), and a weighted estimate of total physical activity per week (MET∙min∙wk⁻¹) was calculated. Total plasma levels of estrone, 17β-estradiol, progesterone, androstenedione, testosterone, and dehydroepiandrosterone-sulphate (DHEA-sulphate) were measured. Bivariate analyses by the Spearman correlation test were done between physical activity and each hormone concentration. Multivariate analyses (linear regression) using concentration of each hormone as the dependent variable and physical activity, age, marital status, BMI, Charlson Comorbidity Index, tumour stage, previous radiotherapy, or previous chemotherapy as predictor variables. Estrone concentration was positively and significantly correlated with BMI (ρ = 0.332, p = 0.022), but no other correlations were found between BMI and the other hormone concentrations, nor were concentrations of any hormone associated with age or Charlson Comorbidity Index (p > 0.05 in all cases). Physical activity was significantly and inversely correlated with estrone concentration (ρ = −0.308; p = 0.035). Linear regression analysis confirmed a statistically significant association between estrone concentration and BMI and physical activity, after adjusting for all potential confounders (for BMI: standardised β coefficient = 0.407; non-standardised β coefficient = 1.054; t = 2.898; p = 0.006; 95% CI for non-standardised beta: 0.318- to 1.790; for physical activity: standardised β coefficient = −0.300; non-standardised β coefficient = −0.005; t = −2.135; p = 0.039; 95% CI for non-standardised beta: −0.010- to 0.000). The relationship between estrone concentration and physical activity may be further explored as a biomarker for evaluating the protective effect of physical activity against breast cancer recurrence in women receiving anti-estrogen therapies.