Predicting the mortality risk of patients with Coronavirus Disease 2019 (COVID-19) can be valuable in allocating limited medical resources in the setting of outbreaks. This study assessed the role of a chest X-ray (CXR) scoring system in a multivariable model in predicting the mortality of COVID-19 patients by performing a single-center, retrospective, observational study including consecutive patients admitted with a confirmed diagnosis of COVID-19 and an initial CXR. The CXR severity score was calculated by three radiologists with 12 to 15 years of experience in thoracic imaging, based on the extent of lung involvement and density of lung opacities. Logistic regression analysis was used to identify independent predictive factors for mortality to create a predictive model. A validation dataset was used to calculate its predictive value as the AUROC. A total of 628 patients (58.1% male) were included in this study. Age (p < 0.001), sepsis (p < 0.001), S/F ratio (p < 0.001), need for mechanical ventilation (p < 0.001), and the CXR severity score (p = 0.005) were found to be independent predictive factors for mortality. We used these variables to develop a predictive model with an AUROC of 0.926 (0.891, 0.962), which was significantly higher than that of the WHO COVID severity classification, 0.853 (0.798, 0.909) (one-tailed p-value = 0.028), showing that our model can accurately predict mortality of hospitalized COVID-19 patients.