Karim Hanna

Karim Hanna

Auto-generated profile, based on publicly available information

Karim Hanna

Morsani College of Medicine, University of South Florida, Tampa, FL, United States

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 April 2024 in Journal of Medical Internet Research
Verified
Journal Article
Open access
ZHZhe He
Zhe He
Balu BhasuranBalu Bhasuran
Balu Bhasuran
QJQiao Jin
Qiao Jin
STShubo Tian
Shubo Tian
KHKarim Hanna
Karim Hanna
ZLZhiyong Lu
Zhiyong Lu
Background: Although patients have easy access to their electronic health records and laboratory test result data through patient portals, laboratory test results are often confusing and hard to understand. Many patients turn to web-based forums or question-and-answer (Q&A) sites to seek advice from their peers. The quality of answers from social Q&A sites on health-related questions varies significantly, and not all responses are accurate or reliable. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to have their questions answered. Objective: We aimed to assess the feasibility of using LLMs to generate relevant, accurate, helpful, and unharmful responses to laboratory test–related questions asked by patients and identify potential issues that can be mitigated using augmentation approaches. Methods: We collected laboratory test result–related Q&A data from Yahoo! Answers and selected 53 Q&A pairs for this study. Using the LangChain framework and ChatGPT web portal, we generated responses to the 53 questions from 5 LLMs: GPT-4, GPT-3.5, LLaMA 2, MedAlpaca, and ORCA_mini. We assessed the similarity of their answers using standard Q&A similarity-based evaluation metrics, including Recall-Oriented Understudy for Gisting Evaluation, Bilingual Evaluation Understudy, Metric for Evaluation of Translation With Explicit Ordering, and Bidirectional Encoder Representations from Transformers Score. We used an LLM-based evaluator to judge whether a target model had higher quality in terms of relevance, correctness, helpfulness, and safety than the baseline model. We performed a manual evaluation with medical experts for all the responses to 7 selected questions on the same 4 aspects. Results: Regarding the similarity of the responses from 4 LLMs; the GPT-4 output was used as the reference answer, the responses from GPT-3.5 were the most similar, followed by those from LLaMA 2, ORCA_mini, and MedAlpaca. Human answers from Yahoo data were scored the lowest and, thus, as the least similar to GPT-4–generated answers. The results of the win rate and medical expert evaluation both showed that GPT-4’s responses achieved better scores than all the other LLM responses and human responses on all 4 aspects (relevance, correctness, helpfulness, and safety). LLM responses occasionally also suffered from lack of interpretation in one’s medical context, incorrect statements, and lack of references. Conclusions: By evaluating LLMs in generating responses to patients’ laboratory test result–related questions, we found that, compared to other 4 LLMs and human answers from a Q&A website, GPT-4’s responses were more accurate, helpful, relevant, and safer. There were cases in which GPT-4 responses were inaccurate and not individualized. We identified a number of ways to improve the quality of LLM responses, including prompt engineering, prompt augmentation, retrieval-augmented generation, and response evaluation.

5 May 2021 in Journal of Medical Internet Research
Verified
Journal Article
Open access
ONOliver T Nguyen
Oliver T Nguyen
ATAmir Alishahi Tabriz
Amir Alishahi Tabriz
JHJin Huo
Jin Huo
KHKarim Hanna
Karim Hanna
CSChristopher M Shea
Christopher M Shea
KTKea L. Turner
Kea L. Turner
Background: Electronic visits (e-visits) involve asynchronous communication between clinicians and patients through a secure web-based platform, such as a patient portal, to elicit symptoms and determine a diagnosis and treatment plan. E-visits are now reimbursable through Medicare due to the COVID-19 pandemic. The state of evidence regarding e-visits, such as the impact on clinical outcomes and health care delivery, is unclear. Objective: To address this gap, we examine how e-visits have impacted clinical outcomes and health care quality, access, utilization, and costs. Methods: We conducted a systematic review; MEDLINE, Embase, and Web of Science were searched from January 2000 through October 2020 for peer-reviewed studies that assessed e-visits’ impacts on clinical and health care delivery outcomes. Results: Out of 1859 papers, 19 met the inclusion criteria. E-visit usage was associated with improved or comparable clinical outcomes, especially for chronic disease management (eg, diabetes care, blood pressure management). The impact on quality of care varied across conditions. Quality of care was equivalent or better for chronic conditions, but variable quality was observed in infection management (eg, appropriate antibiotic prescribing). Similarly, the impact on health care utilization varied across conditions (eg, lower utilization for dermatology but mixed impact in primary care). Health care costs were lower for e-visits than those for in-person visits for a wide range of conditions (eg, dermatology and acute visits). No studies examined the impact of e-visits on health care access. It is difficult to draw firm conclusions about effectiveness or impact on care delivery from the studies that were included because many used observational designs. Conclusions: Overall, the evidence suggests e-visits may provide clinical outcomes that are comparable to those provided by in-person care and reduce health care costs for certain health care conditions. At the same time, there is mixed evidence on health care quality, especially regarding infection management (eg, sinusitis, urinary tract infections, conjunctivitis). Further studies are needed to test implementation strategies that might improve delivery (eg, clinical decision support for antibiotic prescribing) and to assess which conditions can be managed via e-visits.

11 February 2025 in JMIR Medical Education
Verified
Journal Article
Open access
TITsunagu Ichikawa
Tsunagu Ichikawa
EOElizabeth Olsen
Elizabeth Olsen
AVArathi Vinod
Arathi Vinod
NGNoah Glenn
Noah Glenn
KHKarim Hanna
Karim Hanna
SPStacey Pierce-Talsma
Stacey Pierce-Talsma
Background: Interest has recently increased in generative artificial intelligence (GenAI), a subset of artificial intelligence that can create new content. Although the publicly available GenAI tools are not specifically trained in the medical domain, they have demonstrated proficiency in a wide range of medical assessments. The future integration of GenAI in medicine remains unknown. However, the rapid availability of GenAI with a chat interface and the potential risks and benefits are the focus of great interest. As with any significant medical advancement or change, medical schools must adapt their curricula to equip students with the skills necessary to become successful physicians. Furthermore, medical schools must ensure that faculty members have the skills to harness these new opportunities to increase their effectiveness as educators. How medical schools currently fulfill their responsibilities is unclear. Colleges of Osteopathic Medicine (COMs) in the United States currently train a significant proportion of the total number of medical students. These COMs are in academic settings ranging from large public research universities to small private institutions. Therefore, studying COMs will offer a representative sample of the current GenAI integration in medical education. Objective: This study aims to describe the policies and training regarding the specific aspect of GenAI in US COMs, targeting students, faculty, and administrators. Methods: Web-based surveys were sent to deans and Student Government Association (SGA) presidents of the main campuses of fully accredited US COMs. The dean survey included questions regarding current and planned policies and training related to GenAI for students, faculty, and administrators. The SGA president survey included only those questions related to current student policies and training. Results: Responses were received from 81% (26/32) of COMs surveyed. This included 47% (15/32) of the deans and 50% (16/32) of the SGA presidents (with 5 COMs represented by both the deans and the SGA presidents). Most COMs did not have a policy on the student use of GenAI, as reported by the dean (14/15, 93%) and the SGA president (14/16, 88%). Of the COMs with no policy, 79% (11/14) had no formal plans for policy development. Only 1 COM had training for students, which focused entirely on the ethics of using GenAI. Most COMs had no formal plans to provide mandatory (11/14, 79%) or elective (11/15, 73%) training. No COM had GenAI policies for faculty or administrators. Eighty percent had no formal plans for policy development. Furthermore, 33.3% (5/15) of COMs had faculty or administrator GenAI training. Except for examination question development, there was no training to increase faculty or administrator capabilities and efficiency or to decrease their workload. Conclusions: The survey revealed that most COMs lack GenAI policies and training for students, faculty, and administrators. The few institutions with policies or training were extremely limited in scope. Most institutions without current training or policies had no formal plans for development. The lack of current policies and training initiatives suggests inadequate preparedness for integrating GenAI into the medical school environment, therefore, relegating the responsibility for ethical guidance and training to the individual COM member.

1 June 2022 in JCO Oncology Practice
Verified
Journal Article
Closed access
KHKarim Hanna
Karim Hanna
BABrandy L. Arredondo
Brandy L. Arredondo
MCMelody N. Chavez
Melody N. Chavez
CGCarley Geiss
Carley Geiss
EHEmma Hume
Emma Hume
KTKea L. Turner
Kea L. Turner
PURPOSE: The effects of COVID-19 have been understudied in rural areas. This study sought to (1) identify cancer screening barriers and facilitators during the pandemic in rural and urban primary care practices, (2) describe implementation strategies to support cancer screening, and (3) provide recommendations. METHODS: A qualitative study was conducted (N = 42) with primary care staff across 20 sites. Individual interviews were conducted through videoconference from August 2020 to April 2021 and recorded, transcribed, and analyzed using deductive and inductive coding (hybrid approach) in NVivo 12 Plus. Practices included federally qualified health centers, tribal health centers, rural health clinics, hospital/health system–owned clinics, and academic medical centers across 10 states including urban (55%) and rural (45%) sites. Staff included individuals serving in the dual role of health care provider and administrator (21.4%), health care administrator (23.8%), physician (19.0%), advanced practice provider (11.9%), or resident (23.8%). The interviews assessed perceptions about cancer screening barriers and facilitators, implementation strategies, and future recommendations. RESULTS: Participants reported multilevel barriers to cancer screening including policy-level (eg, elective procedure delays), organizational (eg, backlogs), and individual (eg, patient cancellation). Several facilitators to screening were noted, such as home-based testing, using telehealth, and strong partnerships with referral sites. Practices used strategies to encourage screening, such as incentivizing patients and providers and expanding outreach. Rural clinics reported challenges with backlogs, staffing, telehealth implementation, and patient outreach. CONCLUSION: Primary care staff used innovative strategies during the pandemic to promote cancer screening. Unresolved challenges (eg, backlogs and inability to implement telehealth) disproportionately affected rural clinics.

1 July 2024 in The Journal of the American Board of Family Medicine
Verified
Journal Article
Closed access
PDF file
KHKarim Hanna
Karim Hanna
DCDavid Chartash
David Chartash
WLWinston Liaw
Winston Liaw
DADamian Archer
Damian Archer
DPDaniel Parente
Daniel Parente
WAWayne Altman
Wayne Altman
Artificial Intelligence (AI) is poised to revolutionize family medicine, offering a transformative approach to achieving the Quintuple Aim. This article examines the imperative for family medicine to adapt to the rapidly evolving field of AI, with an emphasis on its integration in clinical practice. AI9s recent advancements have the potential to significantly transform health care. We argue for the proactive engagement of family medicine in directing AI technologies toward enhancing the “Quintuple Aim.” The article highlights potential benefits of AI, such as improved patient outcomes through enhanced diagnostic tools, clinician well-being through reduced administrative burdens, and the promotion of health equity by analyzing diverse data sets. However, we also acknowledge the risks associated with AI, including the potential for automation to diverge from patient-centered care and exacerbate health care disparities. Our recommendations stress the need for family medicine education to incorporate AI literacy, the development of a collaborative for AI integration, and the establishment of guidelines and standards through interdisciplinary cooperation. We conclude that although AI poses challenges, its responsible and ethical implementation can revolutionize family medicine, optimizing patient care and enhancing the role of clinicians in a technology-driven future.

2 October 2024 in Family Medicine
Verified
Journal Article
Open access
PDF file
RHRana E. Hanna
Rana E. Hanna
LSLogan R. Smith
Logan R. Smith
RMRahul Mhaskar
Rahul Mhaskar
KHKarim Hanna
Karim Hanna
Background and Objectives: Artificial intelligence (AI), such as ChatGPT and Bard, has gained popularity as a tool in medical education. The use of AI in family medicine has not yet been assessed. The objective of this study is to compare the performance of three large language models (LLMs; ChatGPT 3.5, ChatGPT 4.0, and Google Bard) on the family medicine in-training exam (ITE). Methods: The 193 multiple-choice questions of the 2022 ITE, written by the American Board of Family Medicine, were inputted in ChatGPT 3.5, ChatGPT 4.0, and Bard. The LLMs’ performance was then scored and scaled. Results: ChatGPT 4.0 scored 167/193 (86.5%) with a scaled score of 730 out of 800. According to the Bayesian score predictor, ChatGPT 4.0 has a 100% chance of passing the family medicine board exam. ChatGPT 3.5 scored 66.3%, translating to a scaled score of 400 and an 88% chance of passing the family medicine board exam. Bard scored 64.2%, with a scaled score of 380 and an 85% chance of passing the boards. Compared to the national average of postgraduate year 3 residents, only ChatGPT 4.0 surpassed the residents’ mean of 68.4%. Conclusions: ChatGPT 4.0 was the only LLM that outperformed the family medicine postgraduate year 3 residents’ national averages on the 2022 ITE, providing robust explanations and demonstrating its potential use in delivering background information on common medical concepts that appear on board exams.