Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V’s rationales of image comprehension, recall of medical knowledge, and step-by-step multimodal reasoning when solving New England Journal of Medicine (NEJM) Image Challenges—an imaging quiz designed to test the knowledge and diagnostic capabilities of medical professionals. Evaluation results confirmed that GPT-4V performs comparatively to human physicians regarding multi-choice accuracy (81.6% vs. 77.8%). GPT-4V also performs well in cases where physicians incorrectly answer, with over 78% accuracy. However, we discovered that GPT-4V frequently presents flawed rationales in cases where it makes the correct final choices (35.5%), most prominent in image comprehension (27.2%). Regardless of GPT-4V’s high accuracy in multi-choice questions, our findings emphasize the necessity for further in-depth evaluations of its rationales before integrating such multimodal AI models into clinical workflows.

Background: The Coronavirus Disease 2019 (COVID-19) pandemic has necessitated a sudden transition to remote learning in medical schools. We aimed to assess perceptions of remote learning among pre-clinical medical students and subsequently to identify pros and cons of remote learning, as well as uncover gaps to address in ongoing curricular development.Methods: A survey was distributed to first- and second-year medical students at the University of California San Diego School of Medicine in March 2020. Frequencies of responses to structured multiple-choice questions were compared regarding impacts of remote learning on quality of instruction and ability to participate, value of various remote learning resources, living environment, and preparedness for subsequent stages of training. Responses to open-ended questions about strengths and weaknesses of the remote curriculum and overall reflections were coded for thematic content.Results: Of 268 students enrolled, 104 responded (53.7% of first-year students and 23.9% of second-year students). Overall, students felt that remote learning had negatively affected the quality of instruction and their ability to participate. Most (64.1%) preferred the flexibility of learning material at their own pace. Only 25.5% of respondents still felt connected to the medical school or classmates, and feelings of anxiety and isolation were noted negatives of remote learning. Most second-year students (56.7%) felt their preparation for the United States Medical Licensing Examination Step 1 exam was negatively affected, and 43.3% felt unprepared to begin clerkships. In narrative responses, most students appreciated the increased flexibility of remote learning, but they also identified several deficits that still need to be addressed, including digital fatigue, decreased ability to participate, and lack of clinical skills, laboratory, and hands-on learning.Conclusions: Videocasted lectures uploaded in advance, electronic health record and telehealth training for students, and training for teaching faculty to increase technological fluency may be considered to optimize remote learning curricula.

Electronic health record (EHR) log data have shown promise in measuring physician time spent on clinical activities, contributing to deeper understanding and further optimization of the clinical environment. In this article, we propose 7 core measures of EHR use that reflect multiple dimensions of practice efficiency: total EHR time, work outside of work, time on documentation, time on prescriptions, inbox time, teamwork for orders, and an aspirational measure for the amount of undivided attention patients receive from their physicians during an encounter, undivided attention. We also illustrate sample use cases for these measures for multiple stakeholders. Finally, standardization of EHR log data measure specifications, as outlined here, will foster cross-study synthesis and comparative research.