Healthcare datasets are becoming larger and more complex, necessitating the development of accurate and generalizable AI models for medical applications. Unstructured datasets, including medical imaging, electrocardiograms, and natural language data, are gaining attention with advancements in deep convolutional neural networks and large language models. However, estimating the generalizability of these models to new healthcare settings without extensive validation on external data remains challenging. In experiments across 13 datasets including X-rays, CTs, ECGs, clinical discharge summaries, and lung auscultation data, our results demonstrate that model performance is frequently overestimated by up to 20% on average due to shortcut learning of hidden data acquisition biases (DAB). Shortcut learning refers to a phenomenon in which an AI model learns to solve a task based on spurious correlations present in the data as opposed to features directly related to the task itself. We propose an open source, bias-corrected external accuracy estimate, P_Est, that better estimates external accuracy to within 4% on average by measuring and calibrating for DAB-induced shortcut learning.

BACKGROUND AND PURPOSE: Addressing the performance of an imaging-based parameter compared to a “gold standard” pathologic measurement is essential to achieve accurate clinical T-classification. Our aim was to determine the radiologic-pathologic tumor thickness correlation and its prognostic value in oral squamous cell carcinoma. MATERIALS AND METHODS: All pathologic T1–T3 (seventh edition of the Cancer Staging Manual of the American Joint Committee on Cancer) oral squamous cell carcinomas diagnosed between 2010 and 2015 were reviewed. Radiologic tumor thickness was measured on preoperative CT or MR imaging blinded to pathology. The radiologic-pathologic tumor thickness correlation was calculated. The impact of the imaging-to-surgery time interval and imaging technique on the correlation was explored. Intra-/interrater reliability on radiologic tumor thickness was calculated. The correlation of radiologic-versus-pathologic tumor thickness and its performance as the seventh edition T-category modifier was evaluated. Multivariable analysis assessed the prognostic value of the radiologic tumor thickness for overall survival adjusted for age, seventh edition T-category, and performance status. RESULTS: For 354 consecutive patients, the radiologic-pathologic tumor thickness correlation was similar for the image-to-surgery interval of ≤4.0 weeks (ρ = 0.76) versus 4–8 weeks (ρ = 0.80) but lower in those with more than an 8-week interval (ρ = 0.62). CT and MR imaging had similar correlations (0.76 and 0.80). Intrarater and interrater reliability was excellent (0.88 and 0.84). Excluding 19 cases with an imaging-to-surgery interval of >8 weeks, 335 patients were eligible for further analysis. The radiologic-pathologic tumor thickness correlation was 0.78. The accuracy for upstaging the T-classification based on radiologic tumor thickness was 83% for pathologic T1 and 74% for pathologic T2 tumors. Multivariable analysis confirmed the prognostic value of radiologic tumor thickness (hazard ratio = 1.5, P = .02) for overall survival. CONCLUSIONS: This study demonstrates a good radiologic-pathologic tumor thickness correlation. Intrarater and interrater reliability for radiologic tumor thickness was excellent. Radiologically thicker tumor was predictive of inferior survival.

Artificial intelligence (AI) and machine learning (ML) are becoming critical in developing and deploying personalized medicine and targeted clinical trials. Recent advances in ML have enabled the integration of wider ranges of data including both medical records and imaging (radiomics). However, the development of prognostic models is complex as no modeling strategy is universally superior to others and validation of developed models requires large and diverse datasets to demonstrate that prognostic models developed (regardless of method) from one dataset are applicable to other datasets both internally and externally. Using a retrospective dataset of 2,552 patients from a single institution and a strict evaluation framework that included external validation on three external patient cohorts (873 patients), we crowdsourced the development of ML models to predict overall survival in head and neck cancer (HNC) using electronic medical records (EMR) and pretreatment radiological images. To assess the relative contributions of radiomics in predicting HNC prognosis, we compared 12 different models using imaging and/or EMR data. The model with the highest accuracy used multitask learning on clinical data and tumor volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction, outperforming models relying on clinical data only, engineered radiomics, or complex deep neural network architecture. However, when we attempted to extend the best performing models from this large training dataset to other institutions, we observed significant reductions in the performance of the model in those datasets, highlighting the importance of detailed population-based reporting for AI/ML model utility and stronger validation frameworks. We have developed highly prognostic models for overall survival in HNC using EMRs and pretreatment radiological images based on a large, retrospective dataset of 2,552 patients from our institution.Diverse ML approaches were used by independent investigators. The model with the highest accuracy used multitask learning on clinical data and tumor volume.External validation of the top three performing models on three datasets (873 patients) with significant differences in the distributions of clinical and demographic variables demonstrated significant decreases in model performance. Significance: ML combined with simple prognostic factors outperformed multiple advanced CT radiomics and deep learning methods. ML models provided diverse solutions for prognosis of patients with HNC but their prognostic value is affected by differences in patient populations and require extensive validation.

PURPOSE: Osteoradionecrosis of the jaw (ORN) can manifest in varying severity. The aim of this study is to identify ORN risk factors and develop a novel classification to depict the severity of ORN. METHODS: Consecutive patients with head and neck cancer (HNC) treated with curative-intent intensity-modulated radiation therapy (IMRT) (≥45 Gy) from 2011 to 2017 were included. Occurrence of ORN was identified from in-house prospective dental and clinical databases and charts. Multivariable logistic regression model was used to identify risk factors and stratify patients into high-risk and low-risk groups. A novel ORN classification system was developed to depict ORN severity by modifying existing systems and incorporating expert opinion. The performance of the novel system was compared with 15 existing systems for their ability to identify and predict serious ORN event (jaw fracture or requiring jaw resection). RESULTS: ORN was identified in 219 of 2,732 (8%) consecutive patients with HNC. Factors associated with high risk of ORN were oral cavity or oropharyngeal primaries, received IMRT dose ≥60 Gy, current/ex-smokers, and/or stage III to IV periodontal condition. The ORN rate for high-risk versus low-risk patients was 12.7% versus 3.1% ( P < .001) with an AUC of 0.71. Existing ORN systems overclassified serious ORN events and failed to recognize maxillary ORN. A novel ORN classification system, ClinRad, was proposed on the basis of vertical extent of bone necrosis and presence/absence of exposed bone/fistula. This system detected serious ORN events in 5.7% of patients and statistically outperformed existing systems. CONCLUSION: We identified risk factors for ORN and proposed a novel ORN classification system on the basis of vertical extent of bone necrosis and presence/absence of exposed bone/fistula. It outperformed existing systems in depicting the seriousness of ORN and may facilitate clinical care and clinical trials.

The purpose of this study was to identify clinical and dosimetric factors associated with radiotherapy induced bone injury (RIBI) following stereotactic lung radiotherapy. Inoperable patients with early stage non-small cell lung cancer, treated with SBRT, who received 54 or 60 Gy in 3 fractions, and had a minimum of 6 months follow up were reviewed. Archived treatment plans were retrieved, ribs delineated individually and treatment plans re-computed using heterogeneity correction. Clinical and dosimetric factors were evaluated for their association with rib fracture using logistic regression analysis; a dose-event curve and nomogram were created. 46 consecutive patients treated between Oct 2004 and Dec 2008 with median follow-up 25 months (m) (range 6 – 51 m) were eligible. 41 fractured ribs were detected in 17 patients; median time to fracture was 21 m (range 7 – 40 m). The mean maximum point dose in non-fractured ribs (n = 1054) was 10.5 Gy ± 10.2 Gy, this was higher in fractured ribs (n = 41) 48.5 Gy ± 24.3 Gy (p < 0.0001). On univariate analysis, age, dose to 0.5 cc of the ribs (D_0.5), and the volume of the rib receiving at least 25 Gy (V₂₅), were significantly associated with RIBI. As D_0.5 and V₂₅ were cross-correlated (Spearman correlation coefficient: 0.57, p < 0.001), we selected D_0.5 as a representative dose parameter. On multivariate analysis, age (odds ratio: 1.121, 95% CI: 1.04 – 1.21, p = 0.003), female gender (odds ratio: 4.43, 95% CI: 1.68 – 11.68, p = 0.003), and rib D_0.5 (odds ratio: 1.0009, 95% CI: 1.0007 – 1.001, p < 0.0001) were significantly associated with rib fracture. Using D_0.5, a dose-event curve was constructed estimating risk of fracture from dose at the median follow up of 25 months after treatment. In our cohort, a 50% risk of rib fracture was associated with a D_0.5 of 60 Gy. Dosimetric and clinical factors contribute to risk of RIBI and both should be included when modeling risk of toxicity. A nomogram is presented using D_0.5, age, and female gender to estimate risk of RIBI following SBRT. This requires validation.