22 September 2025 in Journal of the American Medical Informatics Association
NINU SCRIPT Study Investigators
NU SCRIPT Study Investigators
+11
Objective: Rule-based structured data algorithms and natural language processing (NLP) approaches applied to unstructured clinical notes have limited accuracy and poor generalizability for identifying immunosuppression. Large language models (LLMs) may effectively identify patients with heterogenous types of immunosuppression from unstructured clinical notes. We compared the performance of LLMs applied to unstructured notes for identifying patients with immunosuppressive conditions or immunosuppressive medication use against 2 baselines: (1) structured data algorithms using diagnosis codes and medication orders and (2) NLP approaches applied to unstructured notes. Materials and Methods: We used hospital admission notes from a primary cohort of 827 intensive care unit (ICU) patients at Northwestern Memorial Hospital and a validation cohort of 200 ICU patients at Beth Israel Deaconess Medical Center, along with diagnosis codes and medication orders from the primary cohort. We evaluated the performance of structured data algorithms, NLP approaches, and LLMs in identifying 7 immunosuppressive conditions and 6 immunosuppressive medications. Results: In the primary cohort, structured data algorithms achieved peak F1 scores ranging from 0.30 to 0.97 for identifying immunosuppressive conditions and medications. NLP approaches achieved peak F1 scores ranging from 0 to 1. GPT-4o outperformed or matched structured data algorithms and NLP approaches across all conditions and medications, with F1 scores ranging from 0.51 to 1. GPT-4o also performed impressively in our validation cohort (F1 = 1 for 8/13 variables). Discussion: LLMs, particularly GPT-4o, outperformed structured data algorithms and NLP approaches in identifying immunosuppressive conditions and medications with robust external validation. Conclusion: LLMs can be applied for improved cohort identification for research purposes.
8 August 2025 in PLOS ONE
Background: Adults with immunosuppression are more likely to develop severe COVID-19 than adults without immunosuppression. Less is known about differences in outcomes for adults with immunosuppression who are hospitalized with COVID-19. Methods: A retrospective cohort study of adults hospitalized with COVID-19 at Northwestern Medicine hospitals between 03/01/2020 and 05/31/2022 was performed. Regression analyses were performed comparing in-hospital mortality, intensive care unit (ICU) admission, oxygenation requirements, and hospital/ICU length of stay among patients without immunosuppression (n = 9079) and patients with immunosuppression (n = 873). Results: Patients with immunosuppression had significantly higher mortality than patients without immunosuppression (OR: 1.33, 95% CI: 1.11–1.60). This effect was even stronger when controlling for age at admission, diabetes, obesity, SARS-CoV-2 variant era, and COVID-19 medication use (adjusted OR: 1.78, 95% CI: 1.46–2.16). ICU admission (adjusted OR: 1.64, 95% CI: 1.41–1.90) and invasive ventilation (adjusted OR: 1.68, 95% CI: 1.36–2.06) were also significantly higher in patients with immunosuppression. Hospitalization length (median: 7 days) and ICU length of stay (median: 2.5 days) were longer in patients with immunosuppression compared to patients without immunosuppression (median: 5 days, adjusted p < 0.001; median: 2 days, adjusted p = 0.04). Subgroup analyses showed that patients with solid organ transplant, HIV with low CD4 cell count, and secondary immunodeficiency had significantly higher adjusted mortality and ICU admission compared to patients without immunosuppression. Patients with solid organ transplant also had significantly higher invasive ventilation and ICU length of stay. Conclusions: Patients with immunosuppression had worse outcomes than patients without immunosuppression. Subgroup analyses showed that patients with solid organ transplant had the worst outcomes overall. Patients with HIV had similar outcomes as patients without immunosuppression unless CD4 cell count was low.
17 January 2025 in medRxiv
TIThe NU SCRIPT Study Investigators
The NU SCRIPT Study Investigators
+11
Identifying immunosuppressed patients using structured data can be challenging. Large language models effectively extract structured concepts from unstructured clinical text. Here we show that GPT-4o outperforms traditional approaches in identifying immunosuppressive conditions and medication use by processing hospital admission notes. We also demonstrate the extensibility of our approach in an external dataset. Cost-effective models like GPT-4o mini and Llama 3.1 also perform well, but not as well as GPT-4o.
6 November 2025 in medRxiv
TIThe NU SCRIPT Study Investigators
The NU SCRIPT Study Investigators
+9
Background: Despite sophisticated standard of care (SOC) testing (including bronchoalveolar lavage (BAL) fluid culture and multiplex PCR), the etiology of pneumonia in immunocompromised patients is frequently unknown. Microbial cell-free DNA (mcfDNA) sequencing increases diagnostic yield in plasma but remains understudied in BAL fluid. Methods: This was a single center, retrospective, observational cohort study. Residual BAL fluid collected from immunocompromised, mechanically ventilated patients with suspected pneumonia was sent to Karius® for mcfDNA sequencing. Given the retrospective design, mcfDNA sequencing results were unavailable to clinicians. SOC testing results were compared to Karius® BAL (KT-BAL) test reports that classified identified microorganisms as Category One (always pathogenic), Category Two (usually pathogenic), or Category Three (rarely pathogenic). Findings: 228 BAL fluid samples from 155 patients were analyzed. In pneumonia patients, KT-BAL yielded 18 additional Category One organisms, 138 additional Category Two organisms, and 313 additional Category Three organisms compared to SOC. Organisms missed by SOC and identified by KT-BAL included bacterial pathogens (99 patients), clinically relevant DNA viruses (34 patients), non- Candida fungi (11 patients), and parasites (1 patient). Compared to patients with Category One or Two organisms identified by both SOC and KT-BAL (n = 68), patients with organisms identified by KT-BAL alone (n = 87) had significantly more cumulative intubation days (11 [5, 20] days vs. 15 [8, 32] days, p = 0.035). Interpretation: KT-BAL increased diagnostic yield for immunocompromised patients with suspected pneumonia with suggestion of prolonged respiratory failure in patients with pathogens missed by SOC.