Suhana Bedi

Suhana Bedi

Auto-generated profile, based on publicly available information

Suhana Bedi

Stanford University School of Medicine, Stanford, USA

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 January 2026 in Nature Medicine
Verified
Journal Article
Closed access
PDF file
SBSuhana Bedi
Suhana Bedi
HCHejie Cui
Hejie Cui
MFMiguel Fuentes
Miguel Fuentes
AUAlyssa Unell
Alyssa Unell
MWMichael Wornow
Michael Wornow
NSNigam H. Shah
Nigam H. Shah

1 August 2025 in JAMA Network Open
Verified
Journal Article
Open access
SBSuhana Bedi
Suhana Bedi
YJYixing Jiang
Yixing Jiang
PCPhilip Chung
Philip Chung
SKSanmi Koyejo
Sanmi Koyejo
NSNigam Shah
Nigam Shah
This cross-sectional study evaluates whether the performance of large language models on medical benchmarks reflects logical reasoning or pattern recognition.

16 April 2024 in medRxiv
Verified
Preprint
Open access
PDF file
SBSuhana Bedi
Suhana Bedi
YLYutong Liu
Yutong Liu
LOLucy Orr-Ewing
Lucy Orr-Ewing
DDDebadutta Dash
Debadutta Dash
SKSanmi Koyejo
Sanmi Koyejo
OKOluwasanmi Koyejo
Oluwasanmi Koyejo
1 Importance: Large Language Models (LLMs) can assist in a wide range of healthcare-related activities. Current approaches to evaluating LLMs make it difficult to identify the most impactful LLM application areas. Objective: To summarize the current evaluation of LLMs in healthcare in terms of 5 components: evaluation data type, healthcare task, Natural Language Processing (NLP)/Natural Language Understanding (NLU) task, dimension of evaluation, and medical specialty. Data Sources: A systematic search of PubMed and Web of Science was performed for studies published between 01-01-2022 and 02-19-2024. Study Selection: Studies evaluating one or more LLMs in healthcare. Data Extraction and Synthesis: Three independent reviewers categorized 519 studies in terms of data used in the evaluation, the healthcare tasks (the what) and the NLP/NLU tasks (the how) examined, the dimension(s) of evaluation, and the medical specialty studied. Results: Only 5% of reviewed studies utilized real patient care data for LLM evaluation. The most popular healthcare tasks were assessing medical knowledge (e.g. answering medical licensing exam questions, 44.5%), followed by making diagnoses (19.5%), and educating patients (17.7%). Administrative tasks such as assigning provider billing codes (0.2%), writing prescriptions (0.2%), generating clinical referrals (0.6%) and clinical notetaking (0.8%) were less studied. For NLP/NLU tasks, the vast majority of studies examined question answering (84.2%). Other tasks such as summarization (8.9%), conversational dialogue (3.3%), and translation (3.1%) were infrequent. Almost all studies (95.4%) used accuracy as the primary dimension of evaluation; fairness, bias and toxicity (15.8%), robustness (14.8%), deployment considerations (4.6%), and calibration and uncertainty (1.2%) were infrequently measured. Finally, in terms of medical specialty area, most studies were in internal medicine (42%), surgery (11.4%) and ophthalmology (6.9%), with nuclear medicine (0.6%), physical medicine (0.4%) and medical genetics (0.2%) being the least represented. Conclusions and Relevance: Existing evaluations of LLMs mostly focused on accuracy of question answering for medical exams, without consideration of real patient care data. Dimensions like fairness, bias and toxicity, robustness, and deployment considerations received limited attention. To draw meaningful conclusions and improve LLM adoption, future studies need to establish a standardized set of LLM applications and evaluation dimensions, perform evaluations using data from routine care, and broaden testing to include administrative tasks as well as multiple medical specialties. Key Points: Question: How are healthcare applications of large language models (LLMs) currently evaluated? Findings: Studies rarely used real patient care data for LLM evaluation. Administrative tasks such as generating provider billing codes and writing prescriptions were understudied. Natural Language Processing (NLP)/Natural Language Understanding (NLU) tasks like summarization, conversational dialogue, and translation were infrequently explored. Accuracy was the predominant dimension of evaluation, while fairness, bias and toxicity assessments were neglected. Evaluations in specialized fields, such as nuclear medicine and medical genetics were rare. Meaning: Current LLM assessments in healthcare remain shallow and fragmented. To draw concrete insights on their performance, evaluations need to use real patient care data across a broad range of healthcare and NLP/NLU tasks and medical specialties with standardized dimensions of evaluation.

28 April 2023 in medRxiv
Verified
Preprint
Open access
PDF file
SBSuhana Bedi
Suhana Bedi
SFScott L. Fleming
Scott L. Fleming
CCChia-Chun Chiang
Chia-Chun Chiang
KMKeith Morse
Keith Morse
AKAswathi Kumar
Aswathi Kumar
EBEmma Brunskill
Emma Brunskill
Prior work has shown that large language models like GPT-4 and Med-PaLM 2 can answer sample questions from the USMLE Step 2 Clinical Knowledge (CK) exam with greater than 80% accuracy. But can these generative AI create USMLE-like exam questions? This capability could augment humans in writing or preparing for such exams. Here we assess the ability of GPT-4 to generate realistic exam questions by asking licensed physicians to (1) distinguish AI-generated questions from genuine USMLE Step 2 CK questions, and (2) assess the validity of AI-generated questions and answers. We find that GPT-4 can generate question/answer pairs that are largely indistinguishable from human-generated ones, with a majority (64%) deemed “valid” by a panel of licensed physicians.

21 November 2024 in Biocomputing 2020
Verified
Conference Paper
Closed access
SBSuhana Bedi
Suhana Bedi
SFScott L. Fleming
Scott L. Fleming
CCChia-Chun Chiang
Chia-Chun Chiang
KMKeith Morse
Keith Morse
AKAswathi Kumar
Aswathi Kumar
NSNigam H. Shah
Nigam H. Shah

7 January 2026 in Nature Medicine
Verified
Journal Article
Closed access
PDF file
SBSuhana Bedi
Suhana Bedi
JFJason Alan Fries
Jason Alan Fries
NSNigam H. Shah
Nigam H. Shah