Recently, a global phase III study demonstrated that nivolumab markedly improved patient outcomes in recurrent or metastatic head and neck carcinoma (RMHNC). However, the efficacy of nivolumab in patients who are ineligible for clinical trials is unknown. We investigated nivolumab efficacy in real-world patients and prognostic factors associated with the response to nivolumab. This study was conducted at 11 institutes associated with Kyoto University and its Affiliated Hospitals-Head and Neck Oncology Group. In total, 93 patients with RMHNC who received nivolumab between May 2017 and May 2018 were retrospectively reviewed. Objective response rate (ORR), overall survival, and progression-free survival (PFS) were evaluated. Univariate and multivariate analyses were performed to identify prognostic factors. The ORRs in patients with squamous cell carcinoma (SCC) and non-SCC were 21.8% and 0%, respectively. In patients with SCC and non-SCC, the 1-year PFS rates were 28.7% and 8.9%, respectively. The hazard ratio (HR) for risk of PFS events (SCC versus non-SCC) was 2.28 (95% confidence interval: 1.21–4.1; log-rank p = 0.007). Univariate and multivariate analyses revealed radiotherapy history, platinum-refractory carcinoma, and treatment-related adverse events (TRAEs) as important prognostic factors associated with PFS in patients with SCC. In a real-world setting, non-SCC and platinum-refractory carcinoma were associated with a poorer prognosis, and a history of radiotherapy to the primary tumor, and the occurrence of TRAEs were associated with a better prognosis. These findings could be useful for clinicians and patients when selecting a treatment strategy.