2 November 2023 in JMIR Medical Education
Background: Whether GPT-4, the conversational artificial intelligence, can accurately diagnose and triage health conditions and whether it presents racial and ethnic biases in its decisions remain unclear. Objective: We aim to assess the accuracy of GPT-4 in the diagnosis and triage of health conditions and whether its performance varies by patient race and ethnicity. Methods: We compared the performance of GPT-4 and physicians, using 45 typical clinical vignettes, each with a correct diagnosis and triage level, in February and March 2023. For each of the 45 clinical vignettes, GPT-4 and 3 board-certified physicians provided the most likely primary diagnosis and triage level (emergency, nonemergency, or self-care). Independent reviewers evaluated the diagnoses as “correct” or “incorrect.” Physician diagnosis was defined as the consensus of the 3 physicians. We evaluated whether the performance of GPT-4 varies by patient race and ethnicity, by adding the information on patient race and ethnicity to the clinical vignettes. Results: The accuracy of diagnosis was comparable between GPT-4 and physicians (the percentage of correct diagnosis was 97.8% (44/45; 95% CI 88.2%-99.9%) for GPT-4 and 91.1% (41/45; 95% CI 78.8%-97.5%) for physicians; P=.38). GPT-4 provided appropriate reasoning for 97.8% (44/45) of the vignettes. The appropriateness of triage was comparable between GPT-4 and physicians (GPT-4: 30/45, 66.7%; 95% CI 51.0%-80.0%; physicians: 30/45, 66.7%; 95% CI 51.0%-80.0%; P=.99). The performance of GPT-4 in diagnosing health conditions did not vary among different races and ethnicities (Black, White, Asian, and Hispanic), with an accuracy of 100% (95% CI 78.2%-100%). P values, compared to the GPT-4 output without incorporating race and ethnicity information, were all .99. The accuracy of triage was not significantly different even if patients’ race and ethnicity information was added. The accuracy of triage was 62.2% (95% CI 46.5%-76.2%; P=.50) for Black patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for White patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for Asian patients, and 62.2% (95% CI 46.5%-76.2%; P=.69) for Hispanic patients. P values were calculated by comparing the outputs with and without conditioning on race and ethnicity. Conclusions: GPT-4’s ability to diagnose and triage typical clinical vignettes was comparable to that of board-certified physicians. The performance of GPT-4 did not vary by patient race and ethnicity. These findings should be informative for health systems looking to introduce conversational artificial intelligence to improve the efficiency of patient diagnosis and triage.
27 October 2022 in Acute Medicine & Surgery
Aim: Hip fracture is one of the most common fall‐related injuries in the elderly population. Although falls may cause multiple types of injuries, no study has investigated the details of fall‐related injuries accompanied by hip fractures. This study aimed to characterize the features of such injuries.Methods: This is a cross‐sectional study using data from four tertiary emergency departments in Japan. We identified patients diagnosed with hip fracture including femoral neck fracture, trochanter fracture, or subtrochanteric fracture from May 12, 2014 to July 12, 2021. Among patients with hip fracture, we included those with fall‐related hip fracture. We excluded patients ages <40 years old and whose fall was high energy onset, defined as fall from more than three steps or 1 m.Results: Among 326 emergency departments patients diagnosed with fall‐related hip fracture, 288 patients were eligible for the analysis. Seventeen patients (6%) had injuries in addition to hip fractures. The most frequent injury was upper limb injury (e.g., distal radial fracture; n = 5, 30%), followed by head injury (e.g., subdural hematoma; n = 4, 24%), chest injury (e.g., pneumothorax; n = 2, 12%), and trunk injury (vertebral compression fracture; n = 2, 12%). There were no significantly different clinical characteristics between patients with hip injuries and those without.Conclusion: A total of 6% of patients diagnosed with hip fracture had other fall‐related injuries. The most frequent were upper limb injury and head injury. Our findings underscore the importance of whole‐body assessment in patients with fall‐related hip fracture in the emergency department.
15 April 2026 in Resuscitation Plus
As of yet, there are few studies of Asian patients with pseudo-pulseless electrical activity (PEA). Japanese patients with out-of-hospital cardiac arrest (OHCA) tend to be elderly, experience cardiac arrest while at home, and present with severe illness. The present study evaluated the prognostic implications of pseudo-PEA in a Japanese cohort with a high risk of OHCA. The present study is a post hoc analysis of SOS-KANTO 2017, a prospective, multicentric study which included OHCA patients with PEA on hospital arrival who underwent echocardiographic assessment during resuscitation. The patients were categorized into a pseudo-PEA and a true PEA group. The primary outcome was the return of spontaneous circulation (ROSC). The secondary outcomes included survival to hospital admission, 30-day survival, and a favorable neurological status at discharge. Logistic regression was performed with cardiac activity as the primary predictor after adjusting for covariates. Of 894 eligible patients, 345 had pseudo-PEA, and 549 had true PEA. In total, 49.3% achieved ROSC. The pseudo-PEA group had a significantly higher rate of ROSC (87.2% vs. 25.5%; p < 0.01). On multivariable logistic regression, the pseudo-PEA group was independently associated with a high ROSC rate (odds ratio [OR]: 22.1; 95% confidence interval [CI]: 14.2–34.3), survival to hospital admission (OR: 17.8; 95% CI: 11.2–28.2), 30-day survival (OR: 51.9; 95% CI: 7.33–367), and favorable neurological outcome at discharge (OR: 14.3; 95% CI: 1.62–125). Pseudo-PEA may be associated with a higher rate of ROSC and favorable, short and middle-term outcomes in Japanese patients with OHCA. Early echocardiographic assessment of cardiac activity may help prognostication and guide resuscitation even in high-risk populations.
23 March 2023 in JMIR Preprints
BACKGROUND Whether GPT-4, the conversational artificial intelligence, can accurately diagnose and triage health conditions and whether it presents racial and ethnic biases in its decisions remain unclear. OBJECTIVE We aim to assess the accuracy of GPT-4 in the diagnosis and triage of health conditions and whether its performance varies by patient race and ethnicity. METHODS We compared the performance of GPT-4 and physicians, using 45 typical clinical vignettes, each with a correct diagnosis and triage level, in February and March 2023. For each of the 45 clinical vignettes, GPT-4 and 3 board-certified physicians provided the most likely primary diagnosis and triage level (emergency, nonemergency, or self-care). Independent reviewers evaluated the diagnoses as “correct” or “incorrect.” Physician diagnosis was defined as the consensus of the 3 physicians. We evaluated whether the performance of GPT-4 varies by patient race and ethnicity, by adding the information on patient race and ethnicity to the clinical vignettes. RESULTS The accuracy of diagnosis was comparable between GPT-4 and physicians (the percentage of correct diagnosis was 97.8% (44/45; 95% CI 88.2%-99.9%) for GPT-4 and 91.1% (41/45; 95% CI 78.8%-97.5%) for physicians; P=.38). GPT-4 provided appropriate reasoning for 97.8% (44/45) of the vignettes. The appropriateness of triage was comparable between GPT-4 and physicians (GPT-4: 30/45, 66.7%; 95% CI 51.0%-80.0%; physicians: 30/45, 66.7%; 95% CI 51.0%-80.0%; P=.99). The performance of GPT-4 in diagnosing health conditions did not vary among different races and ethnicities (Black, White, Asian, and Hispanic), with an accuracy of 100% (95% CI 78.2%-100%). P values, compared to the GPT-4 output without incorporating race and ethnicity information, were all .99. The accuracy of triage was not significantly different even if patients’ race and ethnicity information was added. The accuracy of triage was 62.2% (95% CI 46.5%-76.2%; P=.50) for Black patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for White patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for Asian patients, and 62.2% (95% CI 46.5%-76.2%; P=.69) for Hispanic patients. P values were calculated by comparing the outputs with and without conditioning on race and ethnicity. CONCLUSIONS GPT-4’s ability to diagnose and triage typical clinical vignettes was comparable to that of board-certified physicians. The performance of GPT-4 did not vary by patient race and ethnicity. These findings should be informative for health systems looking to introduce conversational artificial intelligence to improve the efficiency of patient diagnosis and triage.