Jack Gallifant

Jack Gallifant

Auto-generated profile, based on publicly available information

Jack Gallifant

Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 January 2025 in Nature Medicine
Verified
Journal Article
Open access
PDF file
JGJack Gallifant
Jack Gallifant
MAMajid Afshar
Majid Afshar
SASaleem Ameen
Saleem Ameen
YAYindalon Aphinyanaphongs
Yindalon Aphinyanaphongs
SCShan Chen
Shan Chen
DBDanielle S. Bitterman
Danielle S. Bitterman
Large language models (LLMs) are rapidly being adopted in healthcare, necessitating standardized reporting guidelines. We present transparent reporting of a multivariable model for individual prognosis or diagnosis (TRIPOD)-LLM, an extension of the TRIPOD + artificial intelligence statement, addressing the unique challenges of LLMs in biomedical applications. TRIPOD-LLM provides a comprehensive checklist of 19 main items and 50 subitems, covering key aspects from title to discussion. The guidelines introduce a modular format accommodating various LLM research designs and tasks, with 14 main items and 32 subitems applicable across all categories. Developed through an expedited Delphi process and expert consensus, TRIPOD-LLM emphasizes transparency, human oversight and task-specific performance reporting. We also introduce an interactive website (https://tripod-llm.vercel.app/) facilitating easy guideline completion and PDF generation for submission. As a living document, TRIPOD-LLM will evolve with the field, aiming to enhance the quality, reproducibility and clinical applicability of LLM research in healthcare through comprehensive reporting.

18 January 2024 in PLOS Digital Health
Verified
Journal Article
Open access
PDF file
JGJack Gallifant
Jack Gallifant
AFAmelia Fiske
Amelia Fiske
YSYulia A. Levites Strekalova
Yulia A. Levites Strekalova
JOJuan S. Osorio-Valencia
Juan S. Osorio-Valencia
RPRachael Parke
Rachael Parke
RPRobin Pierce
Robin Pierce
The study provides a comprehensive review of OpenAI’s Generative Pre-trained Transformer 4 (GPT-4) technical report, with an emphasis on applications in high-risk settings like healthcare. A diverse team, including experts in artificial intelligence (AI), natural language processing, public health, law, policy, social science, healthcare research, and bioethics, analyzed the report against established peer review guidelines. The GPT-4 report shows a significant commitment to transparent AI research, particularly in creating a systems card for risk assessment and mitigation. However, it reveals limitations such as restricted access to training data, inadequate confidence and uncertainty estimations, and concerns over privacy and intellectual property rights. Key strengths identified include the considerable time and economic investment in transparent AI research and the creation of a comprehensive systems card. On the other hand, the lack of clarity in training processes and data raises concerns about encoded biases and interests in GPT-4. The report also lacks confidence and uncertainty estimations, crucial in high-risk areas like healthcare, and fails to address potential privacy and intellectual property issues. Furthermore, this study emphasizes the need for diverse, global involvement in developing and evaluating large language models (LLMs) to ensure broad societal benefits and mitigate risks. The paper presents recommendations such as improving data transparency, developing accountability frameworks, establishing confidence standards for LLM outputs in high-risk settings, and enhancing industry research review processes. It concludes that while GPT-4’s report is a step towards open discussions on LLMs, more extensive interdisciplinary reviews are essential for addressing bias, harm, and risk concerns, especially in high-risk domains. The review aims to expand the understanding of LLMs in general and highlights the need for new reflection forms on how LLMs are reviewed, the data required for effective evaluation, and addressing critical issues like bias and risk.

25 October 2023 in The Lancet Digital Health
Verified
Journal Article
Open access
JGJack Gallifant
Jack Gallifant
EKEmmett Alexander Kistler
Emmett Alexander Kistler
LNLuis Filipe Nakayama
Luis Filipe Nakayama
CZChloe Zera
Chloe Zera
SKSunil Kripalani
Sunil Kripalani
LCLeo Anthony Celi
Leo Anthony Celi
The growing recognition of differences in health outcomes across populations has led to a slow but increasing shift towards transparent reporting of patient outcomes. In addition, pay-for-equity initiatives, such as those proposed by the Centers for Medicare and Medicaid, will require the reporting of health outcomes across subgroups over time. Dashboards offer one means of visualising data in the health-care context that can highlight essential disparities in clinical outcomes, guide targeted quality-improvement efforts, and ultimately improve health equity. In this Viewpoint, we evaluate all studies that have reported the successful development of a disparity dashboard and share the data collected and unintended consequences reported. We propose a framework for systematic equality improvement through incentivisation of the collecting and reporting of health data and through implementation of reward systems to reduce health disparities.

24 November 2023 in BMJ Health & Care Informatics
Verified
Journal Article
Open access
PDF file
JZJoe Zhang
Joe Zhang
JGJack Gallifant
Jack Gallifant
RPRobin L Pierce
Robin L Pierce
AFAoife Fordham
Aoife Fordham
JTJames Teo
James Teo
HAHutan Ashrafian
Hutan Ashrafian
Objectives Digital health inequality, observed as differential utilisation of digital tools between population groups, has not previously been quantified in the National Health Service (NHS). Deployment of universal digital health interventions, including a national smartphone app and online primary care services, allows measurement of digital inequality across a nation. We aimed to measure population factors associated with digital utilisation across 6356 primary care providers serving the population of England. Methods We used multivariable regression to test association of population and provider characteristics (including patient demographics, socioeconomic deprivation, disease burden, prescribing burden, geography and healthcare provider resource) with activation of two independent digital services during 2021/2022. Results We find a significant adjusted association between increased population deprivation and reduced digital utilisation across both interventions. Multivariable regression coefficients for most deprived quintiles correspond to 4.27 million patients across England where deprivation is associated with non-activation of the NHS App. Conclusion Results are concerning for technologically driven widening of healthcare inequalities. Targeted incentive to digital is necessary to prevent digital disparity from becoming health outcomes disparity.

27 March 2024 in Journal of Biomedical Informatics
Verified
Journal Article
Closed access
JEJacob G. Ellen
Jacob G. Ellen
João MatosJoão Matos
João Matos
MVMartin Viola
Martin Viola
JGJack Gallifant
Jack Gallifant
JQJustin Quion
Justin Quion
NHNebal S. Abu Hussein
Nebal S. Abu Hussein

21 June 2023 in BMJ Health & Care Informatics
Verified
Journal Article
Open access
PDF file
HWHope Watson
Hope Watson
JGJack Gallifant
Jack Gallifant
YLYuan Lai
Yuan Lai
ARAlexander P Radunsky
Alexander P Radunsky
CVCleva Villanueva
Cleva Villanueva
LCLeo Anthony Celi
Leo Anthony Celi
Introduction In January, the National Institutes of Health (NIH) implemented a Data Management and Sharing Policy aiming to leverage data collected during NIH-funded research. The COVID-19 pandemic illustrated that this practice is equally vital for augmenting patient research. In addition, data sharing acts as a necessary safeguard against the introduction of analytical biases. While the pandemic provided an opportunity to curtail critical research issues such as reproducibility and validity through data sharing, this did not materialise in practice and became an example of ‘Open Data in Appearance Only’ (ODIAO). Here, we define ODIAO as the intent of data sharing without the occurrence of actual data sharing (eg, material or digital data transfers). Objective Propose a framework that states the main risks associated with data sharing, systematically present risk mitigation strategies and provide examples through a healthcare lens. Methods This framework was informed by critical aspects of both the Open Data Institute and the NIH’s 2023 Data Management and Sharing Policy plan guidelines. Results Through our examination of legal, technical, reputational and commercial categories, we find barriers to data sharing ranging from misinterpretation of General Data Privacy Rule to lack of technical personnel able to execute large data transfers. From this, we deduce that at numerous touchpoints, data sharing is presently too disincentivised to become the norm. Conclusion In order to move towards Open Data, we propose the creation of mechanisms for incentivisation, beginning with recentring data sharing on patient benefits, additional clauses in grant requirements and committees to encourage adherence to data reporting practices.