Jeff Zhang

Jeff Zhang

Auto-generated profile, based on publicly available information

Jeff Zhang

New York University Langone Health Department of Neurosurgery

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 January 2025 in Nature Medicine
Verified
Journal Article
Open access
PDF file
DADaniel Alexander Alber
Daniel Alexander Alber
ZYZihao Yang
Zihao Yang
AAAnton Alyakin
Anton Alyakin
EYEunice Yang
Eunice Yang
SRSumedha Rai
Sumedha Rai
EOEric Karl Oermann
Eric Karl Oermann
The adoption of large language models (LLMs) in healthcare demands a careful analysis of their potential to spread false medical knowledge. Because LLMs ingest massive volumes of data from the open Internet during training, they are potentially exposed to unverified medical knowledge that may include deliberately planted misinformation. Here, we perform a threat assessment that simulates a data-poisoning attack against The Pile, a popular dataset used for LLM development. We find that replacement of just 0.001% of training tokens with medical misinformation results in harmful models more likely to propagate medical errors. Furthermore, we discover that corrupted models match the performance of their corruption-free counterparts on open-source benchmarks routinely used to evaluate medical LLMs. Using biomedical knowledge graphs to screen medical LLM outputs, we propose a harm mitigation strategy that captures 91.9% of harmful content (F1 = 85.7%). Our algorithm provides a unique method to validate stochastically generated LLM outputs against hard-coded relationships in knowledge graphs. In view of current calls for improved data provenance and transparent LLM development, we hope to raise awareness of emergent risks from LLMs trained indiscriminately on web-scraped data, particularly in healthcare where misinformation can potentially compromise patient safety.

4 April 2025 in Journal of Neuro-Oncology
Verified
Journal Article
Closed access
PDF file
AFAllen Ye Fu
Allen Ye Fu
KBKenneth Bernstein
Kenneth Bernstein
JZJeff Zhang
Jeff Zhang
JSJoshua Silverman
Joshua Silverman
JMJanice Mehnert
Janice Mehnert
DKDouglas Kondziolka
Douglas Kondziolka

29 January 2025 in Research Square
Verified
Preprint
Open access
PDF file
AAAnton Alyakin
Anton Alyakin
JSJaden Stryker
Jaden Stryker
DADaniel Alber
Daniel Alber
KSKarl Sangwon
Karl Sangwon
BDBrandon Duderstadt
Brandon Duderstadt
EOEric Oermann
Eric Oermann
The authors have requested that this preprint be removed from Research Square.

9 July 2024 in Journal of the American Geriatrics Society
Verified
Journal Article
Closed access
JZJeff Zhang
Jeff Zhang
RKRania Kanchi
Rania Kanchi
SCSarah Conderino
Sarah Conderino
NLNatalie K. Levy
Natalie K. Levy
SASamrachana Adhikari
Samrachana Adhikari
JDJohn A. Dodson
John A. Dodson

19 March 2026 in Research Square
Verified
Preprint
Open access
PDF file
EOEric Oermann
Eric Oermann
LJLavender Jiang
Lavender Jiang
ACAngelica Chen
Angelica Chen
XHXu Han
Xu Han
CLChris Liu
Chris Liu
KCKyunghyun Cho
Kyunghyun Cho
Operational decisions governing patient flow, cost, and quality of care demand specialized predictive models, yet most clinical NLP efforts focus on medical knowledge benchmarks. We introduce Lang1, a family of language models (100M-7B parameters) pretrained on 80 billion clinical tokens from NYU Langone Health electronic health records blended with 627 billion internet tokens. We evaluate Lang1 on the REalistic Medical Evaluation (ReMedE), an evaluation suite derived from 668,331 Electronic Health Records (EHR) notes spanning five tasks: readmission, mortality prediction, length of stay, comorbidity coding, and insurance denial. In zero-shot settings, both general-purpose and biomedical models underperform on four of five tasks. After finetuning, Lang1-1B outperforms finetuned generalist models up to 70x larger and zero-shot models up to 671x larger. Joint multi-task finetuning yields cross-task transfer, and Lang1-1B transfers effectively to unseen tasks and an external health system. These results demonstrate that effective healthcare AI requires in-domain pretraining, supervised finetuning, and evaluation beyond proxy benchmarks.