Emily Alsentzer

Emily Alsentzer

Auto-generated profile, based on publicly available information

Emily Alsentzer

Brigham and Women’s Hospital, Harvard University , Boston, MA 02115

Research Focus & Metrics

Representative Publications

View All Publications
18 December 2023 in The Lancet Digital Health
Verified
Journal Article
Open access
TZTravis Zack
Travis Zack
ELEric Lehman
Eric Lehman
MSMirac Suzgun
Mirac Suzgun
JRJorge A Rodriguez
Jorge A Rodriguez
LCLeo Anthony Celi
Leo Anthony Celi
EAEmily Alsentzer
Emily Alsentzer
Large language models (LLMs) such as GPT-4 hold great promise as transformative tools in health care, ranging from automating administrative tasks to augmenting clinical decision making. However, these models also pose a danger of perpetuating biases and delivering incorrect medical diagnoses, which can have a direct, harmful impact on medical care. We aimed to assess whether GPT-4 encodes racial and gender biases that impact its use in health care. Using the Azure OpenAI application interface, this model evaluation study tested whether GPT-4 encodes racial and gender biases and examined the impact of such biases on four potential applications of LLMs in the clinical domain-namely, medical education, diagnostic reasoning, clinical plan generation, and subjective patient assessment. We conducted experiments with prompts designed to resemble typical use of GPT-4 within clinical and medical education applications. We used clinical vignettes from NEJM Healer and from published research on implicit bias in health care. GPT-4 estimates of the demographic distribution of medical conditions were compared with true US prevalence estimates. Differential diagnosis and treatment planning were evaluated across demographic groups using standard statistical tests for significance between groups. We found that GPT-4 did not appropriately model the demographic diversity of medical conditions, consistently producing clinical vignettes that stereotype demographic presentations. The differential diagnoses created by GPT-4 for standardised clinical vignettes were more likely to include diagnoses that stereotype certain races, ethnicities, and genders. Assessment and plans created by the model showed significant association between demographic attributes and recommendations for more expensive procedures as well as differences in patient perception. Our findings highlight the urgent need for comprehensive and transparent bias assessments of LLM tools such as GPT-4 for intended use cases before they are integrated into clinical care. We discuss the potential sources of these biases and potential mitigation strategies before clinical implementation. Priscilla Chan and Mark Zuckerberg.

30 November 2023 in npj Digital Medicine
Verified
Journal Article
Open access
PDF file
EAEmily Alsentzer
Emily Alsentzer
MRMatthew J. Rasmussen
Matthew J. Rasmussen
RFRomy Fontoura
Romy Fontoura
ACAlexis L. Cull
Alexis L. Cull
BBBrett Beaulieu-Jones
Brett Beaulieu-Jones
VKVesela P. Kovacheva
Vesela P. Kovacheva
Many areas of medicine would benefit from deeper, more accurate phenotyping, but there are limited approaches for phenotyping using clinical notes without substantial annotated data. Large language models (LLMs) have demonstrated immense potential to adapt to novel tasks with no additional training by specifying task-specific instructions. Here we report the performance of a publicly available LLM, Flan-T5, in phenotyping patients with postpartum hemorrhage (PPH) using discharge notes from electronic health records (n = 271,081). The language model achieves strong performance in extracting 24 granular concepts associated with PPH. Identifying these granular concepts accurately allows the development of interpretable, complex phenotypes and subtypes. The Flan-T5 model achieves high fidelity in phenotyping PPH (positive predictive value of 0.95), identifying 47% more patients with this complication compared to the current standard of using claims codes. This LLM pipeline can be used reliably for subtyping PPH and outperforms a claims-based approach on the three most common PPH subtypes associated with uterine atony, abnormal placentation, and obstetric trauma. The advantage of this approach to subtyping is its interpretability, as each concept contributing to the subtype determination can be evaluated. Moreover, as definitions may change over time due to new guidelines, using granular concepts to create complex phenotypes enables prompt and efficient updating of the algorithm. Using this language modelling approach enables rapid phenotyping without the need for any manually annotated training data across multiple clinical use cases.

20 June 2019 in arXiv
Verified
Preprint
Open access
PDF file
EAEmily Alsentzer
Emily Alsentzer
JMJohn R. Murphy
John R. Murphy
WBWillie Boag
Willie Boag
WWWei-Hung Weng
Wei-Hung Weng
DJDi Jin
Di Jin
MMMatthew B. A. McDermott
Matthew B. A. McDermott
Contextual word embedding models such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018) have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these models have been minimally explored on specialty corpora, such as clinical text; moreover, in the clinical domain, no publicly-available pre-trained BERT models yet exist. In this work, we address this need by exploring and releasing BERT models for clinical text: one for generic clinical text and another for discharge summaries specifically. We demonstrate that using a domain-specific model yields performance improvements on three common clinical NLP tasks as compared to nonspecific embeddings. These domain-specific models are not as performant on two clinical de-identification tasks, and argue that this is a natural consequence of the differences between de-identified source text and synthetically non de-identified task text.

23 January 2025 in NEJM AI
Verified
Journal Article
Closed access
IRInioluwa Deborah Raji
Inioluwa Deborah Raji
RDRoxana Daneshjou
Roxana Daneshjou
EAEmily Alsentzer
Emily Alsentzer

17 November 2015 in The ISME Journal
Verified
Journal Article
Closed access
JLJoshua S Lichtman
Joshua S Lichtman
EAEmily Alsentzer
Emily Alsentzer
MJMia Jaffe
Mia Jaffe
DSDaniel Sprockett
Daniel Sprockett
EMEvan Masutani
Evan Masutani
JEJoshua E Elias
Joshua E Elias
Endogenous intestinal microbiota have wide-ranging and largely uncharacterized effects on host physiology. Here, we used reverse-phase liquid chromatography-coupled tandem mass spectrometry to define the mouse intestinal proteome in the stomach, jejunum, ileum, cecum and proximal colon under three colonization states: germ-free (GF), monocolonized with Bacteroides thetaiotaomicron and conventionally raised (CR). Our analysis revealed distinct proteomic abundance profiles along the gastrointestinal (GI) tract. Unsupervised clustering showed that host protein abundance primarily depended on GI location rather than colonization state and specific proteins and functions that defined these locations were identified by random forest classifications. K-means clustering of protein abundance across locations revealed substantial differences in host protein production between CR mice relative to GF and monocolonized mice. Finally, comparison with fecal proteomic data sets suggested that the identities of stool proteins are not biased to any region of the GI tract, but are substantially impacted by the microbiota in the distal colon.

16 July 2023 in medRxiv
Verified
Preprint
Open access
PDF file
TZTravis Zack
Travis Zack
ELEric Lehman
Eric Lehman
MSMirac Suzgun
Mirac Suzgun
JRJorge A. Rodriguez
Jorge A. Rodriguez
LCLeo Anthony Celi
Leo Anthony Celi
EAEmily Alsentzer
Emily Alsentzer
Background: Large language models (LLMs) such as GPT-4 hold great promise as transformative tools in healthcare, ranging from automating administrative tasks to augmenting clinical decision- making. However, these models also pose a serious danger of perpetuating biases and delivering incorrect medical diagnoses, which can have a direct, harmful impact on medical care. Methods: Using the Azure OpenAI API, we tested whether GPT-4 encodes racial and gender biases and examined the impact of such biases on four potential applications of LLMs in the clinical domain—namely, medical education, diagnostic reasoning, plan generation, and patient assessment. We conducted experiments with prompts designed to resemble typical use of GPT-4 within clinical and medical education applications. We used clinical vignettes from NEJM Healer and from published research on implicit bias in healthcare. GPT-4 estimates of the demographic distribution of medical conditions were compared to true U.S. prevalence estimates. Differential diagnosis and treatment planning were evaluated across demographic groups using standard statistical tests for significance between groups. Findings: We find that GPT-4 does not appropriately model the demographic diversity of medical conditions, consistently producing clinical vignettes that stereotype demographic presentations. The differential diagnoses created by GPT-4 for standardized clinical vignettes were more likely to include diagnoses that stereotype certain races, ethnicities, and gender identities. Assessment and plans created by the model showed significant association between demographic attributes and recommendations for more expensive procedures as well as differences in patient perception. Interpretation: Our findings highlight the urgent need for comprehensive and transparent bias assessments of LLM tools like GPT-4 for every intended use case before they are integrated into clinical care. We discuss the potential sources of these biases and potential mitigation strategies prior to clinical implementation.