A substantial number of patients with life-threatening illnesses like cancer receive inappropriate end-of-life care. Improving their quality of end-of-life care is a priority for patients and their families and for public health. To investigate the association between provision, timing, and initial setting of hospital-based specialist palliative care and potentially inappropriate end-of-life care for patients with cancer in two acute care hospitals in the Netherlands, we conducted a retrospective observational study using hospital administrative databases. All adults diagnosed with or treated for cancer in the year preceding their death in 2018 or 2019 were included. The main exposure was hospital-based specialist palliative care initiated >30 days before death. The outcome measures in the last 30 days of life were six quality indicators for inappropriate end-of-life care (≥2 ED-visits, ≥2 hospital admissions, >14 days hospitalization, ICU-admission, chemotherapy, hospital death). We identified 2603 deceased patients, of whom 14% (n = 359) received specialist palliative care >30 days before death (exposure group). Overall, 27% (n = 690) received potentially inappropriate end-of-life care: 19% in the exposure group, versus 28% in the non-exposure group (p < 0.001). The exposure group was 45% less likely to receive potentially inappropriate end-of-life care (AOR 0.55; 95% CI 0.41 to 0.73). Early (>90 days) and late (≤90 and >30 days) initiation of specialist palliative care, as well as outpatient and inpatient initiation, were all associated with less potentially inappropriate end-of-life care (AOR 0.49; 0.62; 0.32; 0.64, respectively). Thus, timely access to hospital-based specialist palliative care is associated with less potentially inappropriate end-of-life care for patients with cancer. The outpatient initiation of specialist palliative care seems to enhance this result.