Aaron E. Kornblith

Aaron E. Kornblith

Auto-generated profile, based on publicly available information

Aaron E. Kornblith

Department of Emergency Medicine University of California San Francisco San Francisco California USA

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 October 2024 in Nature Communications
Verified
Journal Article
Open access
PDF file
CWChristopher Y. K. Williams
Christopher Y. K. Williams
BMBrenda Y. Miao
Brenda Y. Miao
AKAaron E. Kornblith
Aaron E. Kornblith
ABAtul J Butte
Atul J Butte
The release of GPT-4 and other large language models (LLMs) has the potential to transform healthcare. However, existing research evaluating LLM performance on real-world clinical notes is limited. Here, we conduct a highly-powered study to determine whether LLMs can provide clinical recommendations for three tasks (admission status, radiological investigation(s) request status, and antibiotic prescription status) using clinical notes from the Emergency Department. We randomly selected 10,000 Emergency Department visits to evaluate the accuracy of zero-shot, GPT-3.5-turbo- and GPT-4-turbo-generated clinical recommendations across four different prompting strategies. We found that both GPT-4-turbo and GPT-3.5-turbo performed poorly compared to a resident physician, with accuracy scores 8% and 24%, respectively, lower than physician on average. Both LLMs tended to be overly cautious in its recommendations, with high sensitivity at the cost of specificity. Our findings demonstrate that, while early evaluations of the clinical use of LLMs are promising, LLM performance must be significantly improved before their deployment as decision support systems for clinical recommendations and other complex tasks.

29 October 2018 in IEEE Reviews in Biomedical Engineering
Verified
Journal Article
Closed access
ARAdam Rao
Adam Rao
EHEmily Huynh
Emily Huynh
TRThomas J. Royston
Thomas J. Royston
AKAaron Kornblith
Aaron Kornblith
Shuvo RoyShuvo Roy
Shuvo Roy
Recent developments in sensor technology and computational analysis methods enable new strategies to measure and interpret lung acoustic signals that originate internally, such as breathing or vocal sounds, or are externally introduced, such as in chest percussion or airway insonification. A better understanding of these sounds has resulted in new instrumentation that allows for highly accurate as well as portable options for measurement in the hospital, in the clinic, and even at home. This review outlines instrumentation for acoustic stimulation and measurement of the lungs. We first review the fundamentals of acoustic lung signals and the pathophysiology of the diseases that these signals are used to detect. Then, we focus on different methods of measuring and creating signals that have been used in recent research for pulmonary disease diagnosis. These new methods, combined with signal processing and modeling techniques, lead to a reduction in noise, and allow improved feature extraction and signal classification. We conclude by presenting the results of human subject studies taking advantage of both the instrumentation and signal processing tools to accurately diagnose common lung diseases. This paper emphasizes active areas of research within modern lung acoustics, and encourages the standardization of future work in this field.

17 June 2025 in PLOS Digital Health
Verified
Journal Article
Open access
PDF file
CWChristopher Y. K. Williams
Christopher Y. K. Williams
JBJaskaran Bains
Jaskaran Bains
TTTianyu Tang
Tianyu Tang
KPKishan Patel
Kishan Patel
ALAlexa N. Lucas
Alexa N. Lucas
AKAaron E. Kornblith
Aaron E. Kornblith
Large language models (LLMs) possess a range of capabilities which may be applied to the clinical domain, including text summarization. As ambient artificial intelligence scribes and other LLM-based tools begin to be deployed within healthcare settings, rigorous evaluations of the accuracy of these technologies are urgently needed. In this cross-sectional study of 100 randomly sampled adult Emergency Department (ED) visits from 2012 to 2023 at the University of California, San Francisco ED, we sought to investigate the performance of GPT-4 and GPT-3.5-turbo in generating ED encounter summaries and evaluate the prevalence and type of errors for each section of the encounter summary across three evaluation criteria: 1) Inaccuracy of LLM-summarized information; 2) Hallucination of information; 3) Omission of relevant clinical information. In total, 33% of summaries generated by GPT-4 and 10% of those generated by GPT-3.5-turbo were entirely error-free across all evaluated domains. Summaries generated by GPT-4 were mostly accurate, with inaccuracies found in only 10% of cases, however, 42% of the summaries exhibited hallucinations and 47% omitted clinically relevant information. Inaccuracies and hallucinations were most commonly found in the Plan sections of LLM-generated summaries, while clinical omissions were concentrated in text describing patients’ Physical Examination findings or History of Presenting Complaint. The potential harmfulness score across errors was low, with a mean score of 0.57 (SD 1.11) out of 7 and only three errors scoring 4 (‘Potential for permanent harm’) or greater. In summary, we found that LLMs could generate accurate encounter summaries but were liable to hallucination and omission of clinically relevant information. Individual errors on average had a low potential for harm. A comprehensive understanding of the location and type of errors found in LLM-generated clinical text is important to facilitate clinician review of such content and prevent patient harm.

6 May 2024 in Annals of Surgery
Verified
Journal Article
Open access
JSJason L. Sperry
Jason L. Sperry
FGFrancis X. Guyette
Francis X. Guyette
BRBedda L. Rosario-Rivera
Bedda L. Rosario-Rivera
MKMatthew E. Kutcher
Matthew E. Kutcher
LKLucy Z. Kornblith
Lucy Z. Kornblith
DRDoris Roldan
Doris Roldan
Objective: To determine the feasibility, efficacy, and safety of early cold stored platelet transfusion compared with standard care resuscitation in patients with hemorrhagic shock. Background: Data demonstrating the safety and efficacy of early cold stored platelet transfusion are lacking following severe injury. Methods: A phase 2, multicenter, randomized, open label, clinical trial was performed at 5 US trauma centers. Injured patients at risk of large volume blood transfusion and the need for hemorrhage control procedures were enrolled and randomized. The intervention was the early transfusion of a single apheresis cold stored platelet unit, stored for up to 14 days versus standard care resuscitation. The primary outcome was feasibility and the principal clinical outcome for efficacy and safety was 24-hour mortality. Results: Mortality at 24 hours was 5.9% in patients who were randomized to early cold stored platelet transfusion compared with 10.2% in the standard care arm (difference, −4.3%; 95% CI, −12.8% to 3.5%; P=0.26). No significant differences were found for any of the prespecified ancillary outcomes. Rates of arterial and/or venous thromboembolism and adverse events did not differ across treatment groups. Conclusions and Relevance: In severely injured patients, early cold stored platelet transfusion is feasible, safe and did not result in a significant lower rate of 24-hour mortality. Early cold stored platelet transfusion did not result in a higher incidence of arterial and/or venous thrombotic complications or adverse events. The storage age of the cold stored platelet product was not associated with significant outcome differences. Trial Registration: ClinicalTrials.gov identifier: NCT 04667468.

5 November 2020 in Journal of Trauma and Acute Care Surgery
Verified
Journal Article
Closed access
ZMZachary A. Matthay
Zachary A. Matthay
AKAaron E. Kornblith
Aaron E. Kornblith
EMEllicott C. Matthay
Ellicott C. Matthay
MSMahsa Sedaghati
Mahsa Sedaghati
SPSue Rn Peterson
Sue Rn Peterson
LKLucy Z. Kornblith
Lucy Z. Kornblith
BACKGROUND: The large-scale social distancing efforts to reduce SARS-CoV-2 transmission have dramatically changed human behaviors associated with traumatic injuries. Trauma centers have reported decreases in trauma volume, paralleled by changes in injury mechanisms. We aimed to quantify changes in trauma epidemiology at an urban Level I trauma center in a county that instituted one of the earliest shelter-in-place orders to inform trauma care during future pandemic responses. METHODS: A single-center interrupted time-series analysis was performed to identify associations of shelter-in-place with trauma volume, injury mechanisms, and patient demographics in San Francisco, California. To control for short-term trends in trauma epidemiology, weekly level data were analyzed 6 months before shelter-in-place. To control for long-term trends, monthly level data were analyzed 5 years before shelter-in-place. RESULTS: Trauma volume decreased by 50% in the week following shelter-in-place (p < 0.01), followed by a linear increase each successive week (p < 0.01). Despite this, trauma volume for each month (March–June 2020) remained lower compared with corresponding months for all previous 5 years (2015–2019). Pediatric trauma volume showed similar trends with initial decreases (p = 0.02) followed by steady increases (p = 0.05). Reductions in trauma volumes were due entirely to changes in nonviolent injury mechanisms, while violence-related injury mechanisms remained unchanged (p < 0.01). CONCLUSION: Although the shelter-in-place order was associated with an overall decline in trauma volume, violence-related injuries persisted. Delineating and addressing underlying factors driving persistent violence-related injuries during shelter-in-place orders should be a focus of public health efforts in preparation for future pandemic responses. LEVEL OF EVIDENCE: Epidemiological study, level III.

14 February 2025 in Proceedings of the National Academy of Sciences
Verified
Journal Article
Open access
YTYan Shuo Tan
Yan Shuo Tan
CSChandan Singh
Chandan Singh
KNKeyan Nasseri
Keyan Nasseri
AAAbhineet Agarwal
Abhineet Agarwal
JDJames Duncan
James Duncan
BYBin Yu
Bin Yu
Modern machine learning has achieved impressive prediction performance, but often sacrifices interpretability, a critical consideration in high-stakes domains such as medicine. In such settings, practitioners often use highly interpretable decision tree models, but these suffer from inductive bias against additive structure. To overcome this bias, we propose Fast Interpretable Greedy-Tree Sums (FIGS), which generalizes the Classification and Regression Trees (CART) algorithm to simultaneously grow a flexible number of trees in summation. By combining logical rules with addition, FIGS adapts to additive structure while remaining highly interpretable. Experiments on real-world datasets show FIGS achieves state-of-the-art prediction performance. To demonstrate the usefulness of FIGS in high-stakes domains, we adapt FIGS to learn clinical decision instruments (CDIs), which are tools for guiding decision-making. Specifically, we introduce a variant of FIGS known as Group Probability-Weighted Tree Sums (G-FIGS) that accounts for heterogeneity in medical data. G-FIGS derives CDIs that reflect domain knowledge and enjoy improved specificity (by up to 20% over CART) without sacrificing sensitivity or interpretability. Theoretically, we prove that FIGS learns components of additive models, a property we refer to as disentanglement. Further, we show (under oracle conditions) that tree-sum models leverage disentanglement to generalize more efficiently than single tree models when fitted to additive regression functions. Finally, to avoid overfitting with an unconstrained number of splits, we develop Bagging-FIGS, an ensemble version of FIGS that borrows the variance reduction techniques of random forests. Bagging-FIGS performs competitively with random forests and XGBoost on real-world datasets.