Histology is an important predictor of the behavior of breast cancer. We aim to study the impact of histology on the overall survival (OS) of breast cancer patients. We studied 11,085 breast cancer patients diagnosed with T1–T2 tumors, clinically node-negative and non-metastatic, from 2004 to 2019 included in the National Cancer Database. Kaplan–Meier curves, log-rank tests and Cox regression models were used to study the impact of histology and other variables on OS. In our patient population, 8678 (78.28%) had ductal cancer (IDC), while 2407 (21.71%) had lobular cancer (ILC). ILC patients were significantly more likely to be older, Caucasian, have a lower grade at diagnosis and be hormone receptor-positive compared to IDC patients. There was no statistically significant difference in the 5-year OS of early stage ductal (16.8%) and lobular cancer patients (16.7%) (p = 0.200). Patients of Hispanic and African American origin had worse OS rates compared to non-Hispanic and Caucasian patients, respectively. For node-positive disease, HER2+ tumors and triple-negative tumors, chemotherapy had a positive influence on OS (HR 0.85, 95% CI 0.77–0.93, p = 0.0012). Histology did not have a significant impact on the 5-year OS of early stage breast cancer patients.