Background: As health care continues to evolve with technological advancements, the integration of artificial intelligence into clinical practices has shown promising potential to enhance patient care and operational efficiency. Among the forefront of these innovations are large language models (LLMs), a subset of artificial intelligence designed to understand, generate, and interact with human language at an unprecedented scale. Objective: This systematic review describes the role of LLMs in improving diagnostic accuracy, automating documentation, and advancing specialist education and patient engagement within the field of gastroenterology and gastrointestinal endoscopy. Methods: Core databases including MEDLINE through PubMed, Embase, and Cochrane Central registry were searched using keywords related to LLMs (from inception to April 2024). Studies were included if they satisfied the following criteria: (1) any type of studies that investigated the potential role of LLMs in the field of gastrointestinal endoscopy or gastroenterology, (2) studies published in English, and (3) studies in full-text format. The exclusion criteria were as follows: (1) studies that did not report the potential role of LLMs in the field of gastrointestinal endoscopy or gastroenterology, (2) case reports and review papers, (3) ineligible research objects (eg, animals or basic research), and (4) insufficient data regarding the potential role of LLMs. Risk of Bias in Non-Randomized Studies—of Interventions was used to evaluate the quality of the identified studies. Results: Overall, 21 studies on the potential role of LLMs in gastrointestinal disorders were included in the systematic review, and narrative synthesis was done because of heterogeneity in the specified aims and methodology in each included study. The overall risk of bias was low in 5 studies and moderate in 16 studies. The ability of LLMs to spread general medical information, offer advice for consultations, generate procedure reports automatically, or draw conclusions about the presumptive diagnosis of complex medical illnesses was demonstrated by the systematic review. Despite promising benefits, such as increased efficiency and improved patient outcomes, challenges related to data privacy, accuracy, and interdisciplinary collaboration remain. Conclusions: We highlight the importance of navigating these challenges to fully leverage LLMs in transforming gastrointestinal endoscopy practices. Trial Registration: PROSPERO 581772; https://www.crd.york.ac.uk/prospero/

Background: The large language model (LLM) has the potential to be applied to clinical practice. However, there has been scarce study on this in the field of gastroenterology. Aim: This study explores the potential clinical utility of two LLMs in the field of gastroenterology: a customized GPT model and a conventional GPT-4o, an advanced LLM capable of retrieval-augmented generation (RAG). Method: We established a customized GPT with the BM25 algorithm using Open AI’s GPT-4o model, which allows it to produce responses in the context of specific documents including textbooks of internal medicine (in English) and gastroenterology (in Korean). Also, we prepared a conventional ChatGPT 4o (accessed on 16 October 2024) access. The benchmark (written in Korean) consisted of 15 clinical questions developed by four clinical experts, representing typical questions for medical students. The two LLMs, a gastroenterology fellow, and an expert gastroenterologist were tested to assess their performance. Results: While the customized LLM correctly answered 8 out of 15 questions, the fellow answered 10 correctly. When the standardized Korean medical terms were replaced with English terminology, the LLM’s performance improved, answering two additional knowledge-based questions correctly, matching the fellow’s score. However, judgment-based questions remained a challenge for the model. Even with the implementation of ‘Chain of Thought’ prompt engineering, the customized GPT did not achieve improved reasoning. Conventional GPT-4o achieved the highest score among the AI models (14/15). Although both models performed slightly below the expert gastroenterologist’s level (15/15), they show promising potential for clinical applications (scores comparable with or higher than that of the gastroenterology fellow). Conclusions: LLMs could be utilized to assist with specialized tasks such as patient counseling. However, RAG capabilities by enabling real-time retrieval of external data not included in the training dataset, appear essential for managing complex, specialized content, and clinician oversight will remain crucial to ensure safe and effective use in clinical practice.

BACKGROUND Running-related injuries are common among recreational runners, and access to sports medicine consultation in Indonesia remains limited, particularly outside large cities. Artificial intelligence (AI)–based mobile health apps that use large language models could provide personalized, low-cost injury prevention and management guidance, but their usability and market acceptance have rarely been evaluated in non-English-speaking populations. OBJECTIVE This study aimed to evaluate the usability, perceived usefulness, perceived ease of use, product-market fit, and willingness to recommend and pay for Roopin, an AI-based running injury prevention and management app, among recreational runners in Indonesia, and to explore users’ experiences and suggestions for improvement. METHODS We conducted a sequential explanatory mixed methods study. Adult runners using iOS devices were recruited online and used the Bahasa Indonesia version of Roopin for 4 weeks, then completed the System Usability Scale (SUS), perceived usefulness (PU) and perceived ease of use (PEOU) scales, the Sean Ellis product-market fit question, the Net Promoter Score (NPS), and willingness-to-pay items; engagement was recorded through usage logs. Prespecified hypotheses were tested using one-sample tests, rank-based tests, and logistic regression. A purposive subsample took part in focus group interviews, analyzed using reflexive thematic analysis and integrated with the quantitative findings in a joint display. RESULTS Of 478 registrations, 300 participants entered the cohort (mean age 30.9, SD 8.2 years; 93/300, 31.0% women) and 171 (57.0%) completed the end-of-study questionnaire. The median SUS score was 75.0, above the benchmark of 68 (Wilcoxon signed rank test, P=.01; mean 71.2, SD 18.3). Mean PU was 4.19 (SD 0.74) and mean PEOU was 4.16 (SD 0.74), both above the 3.5 benchmark (both P<.001), and each independently predicted intention to continue use (PU: odds ratio 2.37; PEOU: odds ratio 2.96). Market indicators were weaker: 17 of 171 participants (9.9%) would be very disappointed without the app (vs the 40% benchmark; P<.001), the NPS was +10.5, and 38 (22.2%) were willing to pay (vs 30%; P=.03). Overall, 194 of 300 participants (64.7%) opened the app at least once, with a median of 11 days of use (IQR 7-17); 106 (35.3%) never opened it. Ten focus groups (33 participants) highlighted the need for automatic data synchronization for non-Apple Watch devices, more concise AI responses, and stronger engagement features. CONCLUSIONS Among participants who completed the study, Roopin showed above-average usability and high perceived usefulness and ease of use, and its AI injury guidance was rated among its strongest features. Early market indicators, however, did not reach prespecified thresholds: product-market fit, the Net Promoter Score, and willingness to pay all fell short, and engagement was low. Reliable data synchronization for non-Apple Watch devices, more concise AI responses, and stronger reasons to return are priorities for refinement before a larger effectiveness evaluation.

BACKGROUND As health care continues to evolve with technological advancements, the integration of artificial intelligence into clinical practices has shown promising potential to enhance patient care and operational efficiency. Among the forefront of these innovations are large language models (LLMs), a subset of artificial intelligence designed to understand, generate, and interact with human language at an unprecedented scale. OBJECTIVE This systematic review describes the role of LLMs in improving diagnostic accuracy, automating documentation, and advancing specialist education and patient engagement within the field of gastroenterology and gastrointestinal endoscopy. METHODS Core databases including MEDLINE through PubMed, Embase, and Cochrane Central registry were searched using keywords related to LLMs (from inception to April 2024). Studies were included if they satisfied the following criteria: (1) any type of studies that investigated the potential role of LLMs in the field of gastrointestinal endoscopy or gastroenterology, (2) studies published in English, and (3) studies in full-text format. The exclusion criteria were as follows: (1) studies that did not report the potential role of LLMs in the field of gastrointestinal endoscopy or gastroenterology, (2) case reports and review papers, (3) ineligible research objects (eg, animals or basic research), and (4) insufficient data regarding the potential role of LLMs. Risk of Bias in Non-Randomized Studies—of Interventions was used to evaluate the quality of the identified studies. RESULTS Overall, 21 studies on the potential role of LLMs in gastrointestinal disorders were included in the systematic review, and narrative synthesis was done because of heterogeneity in the specified aims and methodology in each included study. The overall risk of bias was low in 5 studies and moderate in 16 studies. The ability of LLMs to spread general medical information, offer advice for consultations, generate procedure reports automatically, or draw conclusions about the presumptive diagnosis of complex medical illnesses was demonstrated by the systematic review. Despite promising benefits, such as increased efficiency and improved patient outcomes, challenges related to data privacy, accuracy, and interdisciplinary collaboration remain. CONCLUSIONS We highlight the importance of navigating these challenges to fully leverage LLMs in transforming gastrointestinal endoscopy practices. CLINICALTRIAL PROSPERO 581772; https://www.crd.york.ac.uk/prospero/