The COVID-19 pandemic is a significant worldwide health crisis. Breast cancer patients with COVID-19 are fragile and require particular clinical care. This study aimed to identify the clinical characteristics of breast cancer patients with COVID-19 and the risks associated with anti-cancer treatment. The medical records of breast cancer patients with laboratory-confirmed COVID-19 were collected among 9559 COVID-19 patients from seven designated hospitals from 13th January to 18th March 2020 in Hubei, China. Univariate and multivariate analyses were performed to assess risk factors for COVID-19 severity. Of the 45 breast cancer patients with COVID-19, 33 (73.3%) developed non-severe COVID-19, while 12 (26.7%) developed severe COVID-19, of which 3 (6.7%) patients died. The median age was 62 years, and 3 (6.7%) patients had stage IV breast cancer. Univariate analysis showed that age over 75 and the Eastern Cooperative Oncology Group (ECOG) score were associated with COVID-19 disease severity (P < 0.05). Multivariate analysis showed that patients who received chemotherapy within 7 days had a significantly higher risk for severe COVID-19 (logistic regression model: RR = 13.886, 95% CI 1.014–190.243, P = 0.049; Cox proportional hazards model: HR = 13.909, 95% CI 1.086–178.150, P = 0.043), with more pronounced neutropenia and higher LDH, CRP and procalcitonin levels than other patients (P < 0.05). In our breast cancer cohort, the severity of COVID-19 could be associated with baseline factors such as age over 75 and ECOG scores. Chemotherapy within 7 days before symptom onset could be a risk factor for severe COVID-19, reflected by neutropenia and elevated LDH, CRP and procalcitonin levels.