Background: Chest Computed Tomography (CT) imaging has played a central role in the diagnosis of interstitial pneumonia in patients affected by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and can be used to obtain the extent of lung involvement in COVID-19 pneumonia patients either qualitatively, via visual inspection, or quantitatively, via AI-based software. This study aims to compare the qualitative/quantitative pathological lung extension data on COVID-19 patients. Secondly, the quantitative data obtained were compared to verify their concordance since they were derived from three different lung segmentation software. Methods: This double-center study includes a total of 120 COVID-19 patients (60 from each center) with positive reverse-transcription polymerase chain reaction (RT-PCR) who underwent a chest CT scan from November 2020 to February 2021. CT scans were analyzed retrospectively and independently in each center. Specifically, CT images were examined manually by two different and experienced radiologists for each center, providing the qualitative extent score of lung involvement, whereas the quantitative analysis was performed by one trained radiographer for each center using three different software: 3DSlicer, CT Lung Density Analysis, and CT Pulmo 3D. Results: The agreement between radiologists for visual estimation of pneumonia at CT can be defined as good (ICC 0.79, 95% CI 0.73–0.84). The statistical tests show that 3DSlicer overestimates the measures assessed; however, ICC index returns a value of 0.92 (CI 0.90–0.94), indicating excellent reliability within the three software employed. ICC was also performed between each single software and the median of the visual score provided by the radiologists. This statistical analysis underlines that the best agreement is between 3D Slicer “LungCTAnalyzer” and the median of the visual score (0.75 with a CI 0.67–82 and with a median value of 22% of disease extension for the software and 25% for the visual values). Conclusions: This study provides for the first time a direct comparison between the actual gold standard, which is represented by the qualitative information described by radiologists, and novel quantitative AI-based techniques, here represented by three different commonly used lung segmentation software, underlying the importance of these specific values that in the future could be implemented as consistent prognostic and clinical course parameters.

(1) Background: Computed tomography (CT) plays a paramount role in the characterization and follow-up of COVID-19. Several score systems have been implemented to properly assess the lung parenchyma involved in patients suffering from SARS-CoV-2 infection, such as the visual quantitative assessment score (VQAS) and software-based quantitative assessment score (SBQAS) to help in managing patients with SARS-CoV-2 infection. This study aims to investigate and compare the diagnostic accuracy of the VQAS and SBQAS with two different types of software based on artificial intelligence (AI) in patients affected by SARS-CoV-2. (2) Methods: This is a retrospective study; a total of 90 patients were enrolled with the following criteria: patients’ age more than 18 years old, positive test for COVID-19 and unenhanced chest CT scan obtained between March and June 2021. The VQAS was independently assessed, and the SBQAS was performed with two different artificial intelligence-driven software programs (Icolung and CT-COPD). The Intraclass Correlation Coefficient (ICC) statistical index and Bland–Altman Plot were employed. (3) Results: The agreement scores between radiologists (R1 and R2) for the VQAS of the lung parenchyma involved in the CT images were good (ICC = 0.871). The agreement score between the two software types for the SBQAS was moderate (ICC = 0.584). The accordance between Icolung and the median of the visual evaluations (Median R1–R2) was good (ICC = 0.885). The correspondence between CT-COPD and the median of the VQAS (Median R1–R2) was moderate (ICC = 0.622). (4) Conclusions: This study showed moderate and good agreement upon the VQAS and the SBQAS; enhancing this approach as a valuable tool to manage COVID-19 patients and the combination of AI tools with physician expertise can lead to the most accurate diagnosis and treatment plans for patients.