1 February 2024 in Annals of Internal Medicine
Large language models (LLMs) are artificial intelligence models trained on vast text data to generate humanlike outputs. They have been applied to various tasks in health care, ranging from answering medical examination questions to generating clinical reports. With increasing institutional partnerships between companies producing LLMs and health systems, the real-world clinical application of these models is nearing realization. As these models gain traction, health care practitioners must understand what LLMs are, their development, their current and potential applications, and the associated pitfalls in a medical setting. This review, coupled with a tutorial, provides a comprehensive yet accessible overview of these areas with the aim of familiarizing health care professionals with the rapidly changing landscape of LLMs in medicine. Furthermore, the authors highlight active research areas in the field that promise to improve LLMs' usability in health care contexts.
7 March 2025 in npj Digital Medicine
Red teaming, the practice of adversarially exposing unexpected or undesired model behaviors, is critical towards improving equity and accuracy of large language models, but non-model creator-affiliated red teaming is scant in healthcare. We convened teams of clinicians, medical and engineering students, and technical professionals (80 participants total) to stress-test models with real-world clinical cases and categorize inappropriate responses along axes of safety, privacy, hallucinations/accuracy, and bias. Six medically-trained reviewers re-analyzed prompt-response pairs and added qualitative annotations. Of 376 unique prompts (1504 responses), 20.1% were inappropriate (GPT-3.5: 25.8%; GPT-4.0: 16%; GPT-4.0 with Internet: 17.8%). Subsequently, we show the utility of our benchmark by testing GPT-4o, a model released after our event (20.4% inappropriate). 21.5% of responses appropriate with GPT-3.5 were inappropriate in updated models. We share insights for constructing red teaming prompts, and present our benchmark for iterative model assessments.
4 March 2024 in Journal of Investigative Dermatology
17 February 2021 in eLife
Life in a three-dimensional biofilm is typical for many bacteria, yet little is known about how strains interact in this context. Here, we created essential gene CRISPR interference knockdown libraries in biofilm-forming Bacillus subtilis and measured competitive fitness during colony co-culture with wild type. Partial knockdown of some translation-related genes reduced growth rates and led to out-competition. Media composition led some knockdowns to compete differentially as biofilm versus non-biofilm colonies. Cells depleted for the alanine racemase AlrA died in monoculture but survived in a biofilm colony co-culture via nutrient sharing. Rescue was enhanced in biofilm colony co-culture with a matrix-deficient parent due to a mutualism involving nutrient and matrix sharing. We identified several examples of mutualism involving matrix sharing that occurred in three-dimensional biofilm colonies but not when cultured in two dimensions. Thus, growth in a three-dimensional colony can promote genetic diversity through sharing of secreted factors and may drive evolution of mutualistic behavior.
9 July 2022 in JAAD Case Reports
MPMarlyanne Pol-Rodriguez
+2