The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. We identified 56 articles published from 2015 to 2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50 %), followed by image-synthesis (13 %), and multiple applications simultaneously (11 %). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32 %). Imaging data was used in 91 % of studies, while only 13 % incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60 %), with Monte Carlo dropout being the most commonly implemented UQ method (32 %) followed by ensembling (16 %). 55 % of studies did not share code or datasets. Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, we identified a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.

Background Clinical trial matching, essential for advancing medical research, involves detailed screening of potential participants to ensure alignment with specific trial requirements. Research staff face challenges due to the high volume of eligible patients and the complexity of varying eligibility criteria. The traditional manual process, both time-consuming and error-prone, often leads to missed opportunities. Recently, large language models (LLMs), specifically generative pre-trained transformers (GPTs), have become impressive and impactful tools. Utilizing such tools from artificial intelligence (AI) and natural language processing (NLP) may enhance the accuracy and efficiency of this process through automated patient screening against established criteria. Methods Utilizing data from the National NLP Clinical Challenges (n2c2) 2018 Challenge, we utilized 202 longitudinal patient records. These records were annotated by medical professionals and evaluated against 13 selection criteria encompassing various health assessments. Our approach involved embedding medical documents into a vector database to determine relevant document sections and then using an LLM (OpenAI's GPT-3.5 Turbo and GPT-4) in tandem with structured and chain-of-thought prompting techniques for systematic document assessment against the criteria. Misclassified criteria were also examined to identify classification challenges. Results This study achieved an accuracy of 0.81, sensitivity of 0.80, specificity of 0.82, and a micro F1 score of 0.79 using GPT-3.5 Turbo, and an accuracy of 0.87, sensitivity of 0.85, specificity of 0.89, and micro F1 score of 0.86 using GPT-4. Notably, some criteria in the ground truth appeared mislabeled, an issue we couldn’t explore further due to insufficient label generation guidelines on the website. Conclusion Our findings underscore the potential of AI and NLP technologies, including LLMs, in the clinical trial matching process. The study demonstrated strong capabilities in identifying eligible patients and minimizing false inclusions. Such automated systems promise to alleviate the workload of research staff and improve clinical trial enrollment, thus accelerating the process and enhancing the overall feasibility of clinical research. Further work is needed to determine the potential of this approach when implemented on real clinical data.

There can be significant uncertainty when identifying cervical lymph node (LN) metastases in patients with oropharyngeal squamous cell carcinoma (OPSCC) despite the use of modern imaging modalities such as positron emission tomography (PET) and computed tomography (CT) scans. Grossly involved LNs are readily identifiable during routine imaging, but smaller and less PET-avid LNs are harder to classify. We trained a convolutional neural network (CNN) to detect malignant LNs in patients with OPSCC and used quantitative measures of uncertainty to identify the most reliable predictions. Our dataset consisted of images of 791 LNs from 129 patients with OPSCC who had preoperative PET/CT imaging and detailed pathological reports after neck dissections. These LNs were segmented on PET/CT imaging and then labeled according to the pathology reports. An AlexNet-like CNN was trained to classify LNs as malignant or benign. We estimated epistemic and aleatoric uncertainty by using dropout variational inference and test-time augmentation, respectively. CNN performance was stratified according to the median epistemic and aleatoric uncertainty values calculated using the validation cohort. Our model achieved an area under the receiver operating characteristic (ROC) curve (AUC) of 0.99 on the testing dataset. Sensitivity and specificity were 0.94 and 0.90, respectively. Epistemic and aleatoric uncertainty values were statistically larger for false negative and false positive predictions than for true negative and true positive predictions (p < 0.001). Model sensitivity and specificity were 1.0 and 0.98, respectively, for cases with epistemic uncertainty lower than the median value of the incorrect predictions in the validation dataset. For cases with higher epistemic uncertainty, sensitivity and specificity were 0.67 and 0.41, respectively. Model sensitivity and specificity were 1.0 and 0.98, respectively, for cases with aleatoric uncertainty lower than the median value of the incorrect predictions in the validation dataset. For cases with higher aleatoric uncertainty, sensitivity and specificity were 0.67 and 0.37, respectively. We used a CNN to predict the malignant status of LNs in patients with OPSCC with high accuracy, and we showed that uncertainty can be used to quantify a prediction’s reliability. Assigning measures of uncertainty to predictions could improve the accuracy of LN classification by efficiently identifying instances where expert evaluation is needed to corroborate a model’s prediction.

Purpose: Elective neck irradiation (ENI) has long been considered mandatory when treating head and neck squamous cell carcinoma (HNSCC) with definitive radiotherapy, but it is associated with significant dose to normal organs-at-risk (OAR). In this prospective phase II study, we investigated the efficacy and tolerability of eliminating ENI and strictly treating involved and suspicious lymph nodes (LN) with intensity-modulated radiotherapy. Patients and Methods: Patients with newly diagnosed HNSCC of the oropharynx, larynx, and hypopharynx were eligible for enrollment. Each LN was characterized as involved or suspicious based on radiologic criteria and an in-house artificial intelligence (AI)–based classification model. Gross disease received 70 Gray (Gy) in 35 fractions and suspicious LNs were treated with 66.5 Gy, without ENI. The primary endpoint was solitary elective volume recurrence, with secondary endpoints including patterns-of-failure and patient-reported outcomes. Results: Sixty-seven patients were enrolled, with 18 larynx/hypopharynx and 49 oropharynx cancer. With a median follow-up of 33.4 months, the 2-year risk of solitary elective nodal recurrence was 0%. Gastrostomy tubes were placed in 14 (21%), with median removal after 2.9 months for disease-free patients; no disease-free patient is chronically dependent. Grade I/II dermatitis was seen in 90%/10%. There was no significant decline in composite MD Anderson Dysphagia Index scores after treatment, with means of 89.1 and 92.6 at 12 and 24 months, respectively. Conclusions: These results suggest that eliminating ENI is oncologically sound for HNSCC, with highly favorable quality-of-life outcomes. Additional prospective studies are needed to support this promising paradigm before implementation in any nontrial setting.

BACKGROUND: While International Classification of Diseases, Tenth Revision codes suffice for identifying stroke events in surveillance, accurately classifying stroke types and subtypes using electronic health records remains challenging due to limitations in structured data. This often necessitates manual review of clinical documentation. This study evaluated whether a large language model, Generative Pre-Trained Transformer 4 Omni (GPT-4o), can accurately identify stroke types and subtypes from unstructured clinical notes. METHODS: We implemented a retrieval-augmented generation framework with GPT-4o to classify stroke types (ischemic versus hemorrhagic) and ischemic stroke subtypes using electronic health records data. The American Heart Association Get With The Guidelines–Stroke registry served as the gold standard. Model development used a 20% subset of Get With The Guidelines–Stroke–linked data from UT Southwestern Medical Center (UTSW), with the remaining 80% reserved for testing. External validation used data from the Parkland Health and Hospital System (PHHS). A total of 4123 stroke hospitalizations from January 2019 to August 2023 were included (UTSW: n=2047; PHHS: n=2076). Three prompting strategies—zero-shot chain-of-thought, expert-guided, and instruction-based—were evaluated. Predictions of GPT-4os were compared with classifications made by trained abstractors contributing to the Get With The Guidelines–Stroke registry. RESULTS: In the external validation set, 79.6% of patients had ischemic stroke and 20.4% hemorrhagic. GPT-4o achieved 98% accuracy (95% CI, 0.97–0.99) in classifying stroke type, where accuracy reflects the overall proportion of correctly classified patients. Sensitivity was 0.98 (95% CI, 0.97–0.99), and specificity was 0.97 (95% CI, 0.96–0.98). For ischemic stroke subtypes, sensitivity ranged from 0.40 (95% CI, 0.31–0.49) for cryptogenic to 0.95 (95% CI, 0.93–0.97) for small-vessel occlusion. Specificity ranged from 0.94 (95% CI, 0.92–0.96) for large-artery atherosclerosis to 0.98 (95% CI, 0.97–0.99) for cardioembolism. Zero-shot chain-of-thought prompting—requiring minimal human input—performed comparably to more labor-intensive strategies. Consistency analysis revealed > 99% agreement across repeated queries. CONCLUSIONS: GPT-4o demonstrated strong accuracy in classifying stroke types but faced challenges with ischemic subtypes.

(1) We hypothesized that adding concurrent stereotactic ablative radiotherapy (SAbR) would increase the time to progression in patients with metastatic castrate-resistant prostate cancer (mCRPCA) treated with sipuleucel-T. (2) Patients with a history of prostate cancer (PC), radiographic evidence of metastatic disease, and rising prostate-specific antigen (PSA) > 0.2 ng/dL on castrate testosterone levels were enrolled in this single-arm phase II clinical trial and treated with sipuleucel-T and SAbR. The primary endpoint was time to progression (TTP). Cellular and humoral responses were measured using ELISpot and Luminex multiplex assays, respectively. (3) Twenty patients with mCRPC were enrolled and treated with SAbR to 1–3 sites. Treatment was well tolerated with 51, 8, and 4 treatment-related grade 1, 2, and 3 toxicities, respectively, and no grade 4 or 5 adverse events. At a median follow-up of 15.5 months, the median TTP was 11.2 weeks (95% CI; 6.8–14.0 weeks). Median OS was 76.8 weeks (95% CI; 41.6–130.8 weeks). This regimen induced both humoral and cellular immune responses. Baseline M-MDSC levels were elevated in mCRPC patients compared to healthy donors (p = 0.004) and a decline in M-MDSC was associated with biochemical response (p = 0.044). Responders had lower baseline uric acid levels (p = 0.05). No clear correlation with radiographic response was observed. (4) While the regimen was safe, the PC-antigen-specific immune response induced by SAbR did not yield a synergistic clinical benefit for patients treated with sipuleucel-T compared to the historically reported outcomes.