Background: Breast cancer is rare in men. This population-based study aimed to determine outcomes of male breast cancer in relation to residence and other variables. Methods: In this retrospective cohort study, men diagnosed with breast cancer in Saskatchewan during 2000–2019 were evaluated. Cox proportional multivariable regression analyses were performed to determine the correlation between survival and clinicopathological and contextual factors. Results: One hundred-eight eligible patients with a median age of 69 years were identified. Of them, 16% had WHO performance status ≥ 2 and 61% were rural residents. The stage at diagnosis was as follows: stage 0, 7%; I, 31%; II, 42%; III, 11%; IV, 8%. Ninety-eight percent had hormone receptor-positive breast cancer. The median disease-free survival of urban patients was 97 (95% CI: 50–143) vs. 64 (46–82) months of rural patients (p = 0.29). The median OS of urban patients was 127 (94–159) vs. 93 (32–153) months for rural patients (p = 0.27). On multivariable analysis, performance status ≥ 2, hazard ratio (HR) 2.82 (1.14–6.94), lack of adjuvant systemic therapy, HR 2.47 (1.03–5.92), and node-positive disease, HR 2.32 (1.22–4.40) were significantly correlated with inferior disease-free survival in early-stage invasive breast cancer. Whereas stage IV disease, HR 7.8 (3.1–19.5), performance status ≥ 2, HR 3.25 (1.57–6.71), and age ≥ 65 years, HR 2.37 (1.13–5.0) were correlated with inferior overall survival in all stages. Conclusions: Although residence was not significantly correlated with outcomes, rural men had numerically inferior survival. Poor performance status, node-positive disease, and lack of adjuvant systemic therapy were correlated with inferior disease-free survival.