Background: Assessment of medical information provided by artificial intelligence (AI) chatbots like ChatGPT and Google’s Gemini and comparison with international guidelines is a burgeoning area of research. These AI models are increasingly being considered for their potential to support clinical decision-making and patient education. However, their accuracy and reliability in delivering medical information that aligns with established guidelines remain under scrutiny. Objective: This study aims to assess the accuracy of medical information generated by ChatGPT and Gemini and its alignment with international guidelines for sepsis management. Methods: ChatGPT and Gemini were asked 18 questions about the Surviving Sepsis Campaign guidelines, and the responses were evaluated by 7 independent intensive care physicians. The responses generated were scored as follows: 3=correct, complete, and accurate; 2=correct but incomplete or inaccurate; and 1=incorrect. This scoring system was chosen to provide a clear and straightforward assessment of the accuracy and completeness of the responses. The Fleiss κ test was used to assess the agreement between evaluators, and the Mann-Whitney U test was used to test for the significance of differences between the correct responses generated by ChatGPT and Gemini. Results: ChatGPT provided 5 (28%) perfect responses, 12 (67%) nearly perfect responses, and 1 (5%) low-quality response, with substantial agreement among the evaluators (Fleiss κ=0.656). Gemini, on the other hand, provided 3 (17%) perfect responses, 14 (78%) nearly perfect responses, and 1 (5%) low-quality response, with moderate agreement among the evaluators (Fleiss κ=0.582). The Mann-Whitney U test revealed no statistically significant difference between the two platforms (P=.48). Conclusions: ChatGPT and Gemini both demonstrated potential for generating medical information. Despite their current limitations, both showed promise as complementary tools in patient education and clinical decision-making. The medical information generated by ChatGPT and Gemini still needs ongoing evaluation regarding its accuracy and alignment with international guidelines in different medical domains, particularly in the sepsis field.
BACKGROUND Assessment of medical information provided by artificial intelligence (AI) chatbots like ChatGPT and Google’s Gemini and comparison with international guidelines is a burgeoning area of research. These AI models are increasingly being considered for their potential to support clinical decision-making and patient education. However, their accuracy and reliability in delivering medical information that aligns with established guidelines remain under scrutiny. OBJECTIVE This study aims to assess the accuracy of medical information generated by ChatGPT and Gemini and its alignment with international guidelines for sepsis management. METHODS ChatGPT and Gemini were asked 18 questions about the Surviving Sepsis Campaign guidelines, and the responses were evaluated by 7 independent intensive care physicians. The responses generated were scored as follows: 3=correct, complete, and accurate; 2=correct but incomplete or inaccurate; and 1=incorrect. This scoring system was chosen to provide a clear and straightforward assessment of the accuracy and completeness of the responses. The Fleiss κ test was used to assess the agreement between evaluators, and the Mann-Whitney U test was used to test for the significance of differences between the correct responses generated by ChatGPT and Gemini. RESULTS ChatGPT provided 5 (28%) perfect responses, 12 (67%) nearly perfect responses, and 1 (5%) low-quality response, with substantial agreement among the evaluators (Fleiss κ=0.656). Gemini, on the other hand, provided 3 (17%) perfect responses, 14 (78%) nearly perfect responses, and 1 (5%) low-quality response, with moderate agreement among the evaluators (Fleiss κ=0.582). The Mann-Whitney U test revealed no statistically significant difference between the two platforms (P=.48). CONCLUSIONS ChatGPT and Gemini both demonstrated potential for generating medical information. Despite their current limitations, both showed promise as complementary tools in patient education and clinical decision-making. The medical information generated by ChatGPT and Gemini still needs ongoing evaluation regarding its accuracy and alignment with international guidelines in different medical domains, particularly in the sepsis field.
This investigation examines the independent contributions of artificial intelligence (AI) capabilities and organizational creativity to sustainable development outcomes within Saudi Arabia’s healthcare sector. Despite theoretical predictions of synergistic interactions between technological and creative capabilities, empirical evidence from institutionalized healthcare contexts remains limited. This study, employing structural equation modelling with data from 304 healthcare professionals across two major healthcare institutions, reveals unexpected findings that challenge prevailing theoretical assumptions. Contrary to established technology management literature, the analysis demonstrates that AI capabilities and organizational creativity function as parallel, non-interactive contributors to sustainability rather than exhibiting the synergistic relationships hypothesized in existing theoretical frameworks. All five dimensions of AI capabilities examined, Data Infrastructure, Technology, Technical Skills, Business Skills, and Organizational Change Capacity, demonstrate strong positive direct effects on sustainable development outcomes, with standardized coefficients ranging from 0.307 to 0.590. Organizational creativity similarly exhibits positive direct effects, albeit with more modest magnitudes (β = 0.202-0.244). However, the comprehensive absence of significant moderation effects across all hypothesized interactions challenges conventional assumptions about technology-creativity synergies, suggesting that highly regulated healthcare environments may fundamentally alter how organizational capabilities translate into sustainable outcomes.