Background: With the increasing interest in the application of large language models (LLMs) in the medical field, the feasibility of its potential use as a standardized patient in medical assessment is rarely evaluated. Specifically, we delved into the potential of using ChatGPT, a representative LLM, in transforming medical education by serving as a cost-effective alternative to standardized patients, specifically for history-taking tasks. Objective: The study aims to explore ChatGPT’s viability and performance as a standardized patient, using prompt engineering to refine its accuracy and use in medical assessments. Methods: A 2-phase experiment was conducted. The first phase assessed feasibility by simulating conversations about inflammatory bowel disease (IBD) across 3 quality groups (good, medium, and bad). Responses were categorized based on their relevance and accuracy. Each group consisted of 30 runs, with responses scored to determine whether they were related to the inquiries. For the second phase, we evaluated ChatGPT’s performance against specific criteria, focusing on its anthropomorphism, clinical accuracy, and adaptability. Adjustments were made to prompts based on ChatGPT’s response shortcomings, with a comparative analysis of ChatGPT’s performance between original and revised prompts. A total of 300 runs were conducted and compared against standard reference scores. Finally, the generalizability of the revised prompt was tested using other scripts for another 60 runs, together with the exploration of the impact of the used language on the performance of the chatbot. Results: The feasibility test confirmed ChatGPT’s ability to simulate a standardized patient effectively, differentiating among poor, medium, and good medical inquiries with varying degrees of accuracy. Score differences between the poor (74.7, SD 5.44) and medium (82.67, SD 5.30) inquiry groups (P<.001), between the poor and good (85, SD 3.27) inquiry groups (P<.001) were significant at a significance level (α) of .05, while the score differences between the medium and good inquiry groups were not statistically significant (P=.16). The revised prompt significantly improved ChatGPT’s realism, clinical accuracy, and adaptability, leading to a marked reduction in scoring discrepancies. The score accuracy of ChatGPT improved 4.926 times compared to unrevised prompts. The score difference percentage drops from 29.83% to 6.06%, with a drop in SD from 0.55 to 0.068. The performance of the chatbot on a separate script is acceptable with an average score difference percentage of 3.21%. Moreover, the performance differences between test groups using various language combinations were found to be insignificant. Conclusions: ChatGPT, as a representative LLM, is a viable tool for simulating standardized patients in medical assessments, with the potential to enhance medical training. By incorporating proper prompts, ChatGPT’s scoring accuracy and response realism significantly improved, approaching the feasibility of actual clinical use. Also, the influence of the adopted language is nonsignificant on the outcome of the chatbot.

Background: Emergency departments (EDs) face significant challenges due to overcrowding, prolonged waiting times, and staff shortages, leading to increased strain on health care systems. Efficient triage systems and accurate departmental guidance are critical for alleviating these pressures. Recent advancements in large language models (LLMs), such as ChatGPT, offer potential solutions for improving patient triage and outpatient department selection in emergency settings. Objective: The study aimed to assess the accuracy, consistency, and feasibility of GPT-4–based ChatGPT models (GPT-4o and GPT-4-Turbo) for patient triage using the Modified Early Warning Score (MEWS) and evaluate GPT-4o’s ability to provide accurate outpatient department guidance based on simulated patient scenarios. Methods: A 2-phase experimental study was conducted. In the first phase, 2 ChatGPT models (GPT-4o and GPT-4-Turbo) were evaluated for MEWS-based patient triage accuracy using 1854 simulated patient scenarios. Accuracy and consistency were assessed before and after prompt engineering. In the second phase, GPT-4o was tested for outpatient department selection accuracy using 264 scenarios sourced from the Chinese Medical Case Repository. Each scenario was independently evaluated by GPT-4o thrice. Data analyses included Wilcoxon tests, Kendall correlation coefficients, and logistic regression analyses. Results: In the first phase, ChatGPT’s triage accuracy, based on MEWS, improved following prompt engineering. Interestingly, GPT-4-Turbo outperformed GPT-4o. GPT-4-Turbo achieved an accuracy of 100% compared to GPT-4o’s accuracy of 96.2%, despite GPT-4o initially showing better performance prior to prompt engineering. This finding suggests that GPT-4-Turbo may be more adaptable to prompt optimization. In the second phase, GPT-4o, with superior performance on emotional responsiveness compared to GPT-4-Turbo, demonstrated an overall guidance accuracy of 92.63% (95% CI 90.34%-94.93%), with the highest accuracy in internal medicine (93.51%, 95% CI 90.85%-96.17%) and the lowest in general surgery (91.46%, 95% CI 86.50%-96.43%). Conclusions: ChatGPT demonstrated promising capability for supporting patient triage and outpatient guidance in EDs. GPT-4-Turbo showed greater adaptability to prompt engineering, whereas GPT-4o exhibited superior responsiveness and emotional interaction, which are essential for patient-facing tasks. Future studies should explore real-world implementation and address the identified limitations to enhance ChatGPT’s clinical integration.

Background: Electronic health record (EHR) patient portals are designed to provide medical health records to patients. Using an EHR portal is expected to contribute to positive health outcomes and facilitate patient-provider communication. Objective: Our objective was to examine how portal users report using their portals and the factors associated with obtaining health information from the internet. We also examined the desired portal features, factors impacting users’ trust in portals, and barriers to using portals. Methods: An internet-based survey study was conducted using Amazon Mechanical Turk. All the participants were adults in the United States who used patient portals. The survey included questions about how the participants used their portals, what factors acted as barriers to using their portals, and how they used and how much they trusted other web-based health information sources as well as their portals. A logistic regression model was used to examine the factors influencing the participants’ trust in their portals. Additionally, the desired features and design characteristics were identified to support the design of future portals. Results: A total of 394 participants completed the survey. Most of the participants were less than 35 years old (212/394, 53.8%), with 36.3% (143/394) aged between 35 and 55 years, and 9.9% (39/394) aged above 55 years. Women accounted for 48.5% (191/394) of the survey participants. More than 78% (307/394) of the participants reported using portals at least monthly. The most common portal features used were viewing lab results, making appointments, and paying bills. Participants reported some barriers to portal use including data security and limited access to the internet. The results of a logistic regression model used to predict the trust in their portals suggest that those comfortable using their portals (odds ratio [OR] 7.97, 95% CI 1.11-57.32) thought that their portals were easy to use (OR 7.4, 95% CI 1.12-48.84), and frequent internet users (OR 43.72, 95% CI 1.83-1046.43) were more likely to trust their portals. Participants reporting that the portals were important in managing their health (OR 28.13, 95% CI 5.31-148.85) and that their portals were a valuable part of their health care (OR 6.75, 95% CI 1.51-30.11) were also more likely to trust their portals. Conclusions: There are several factors that impact the trust of EHR patient portal users in their portals. Designing easily usable portals and considering these factors may be the most effective approach to improving trust in patient portals. The desired features and usability of portals are critical factors that contribute to users’ trust in EHR portals.

Background: The internet has enabled convenient and efficient health information searching which is valuable for individuals with chronic conditions requiring some level of self-management. However, there is little research evaluating what factors may impact the use of the internet for health-related tasks for specific clinical populations, such as individuals with inflammatory bowel diseases. Objective: Our goal was to investigate the factors that influence internet use in acquiring health information by individuals with inflammatory bowel diseases. Specifically, we identified factors associated with internet searching behavior and using the internet for completing health-related tasks. Methods: We used 2016 National Health Interview Survey weighted data to develop logistic regression models to predict the likelihood that individuals with inflammatory bowel diseases would use the internet for 2 types of tasks: seeking health information through online searches and using the internet to perform health-related tasks including scheduling appointments and emailing care providers. Results: 2016 National Health Interview Survey weighted data include more than 3 million weighted adult respondents with inflammatory bowel diseases (approximately 1.29% of adults in the weighted data set). Our results suggest that approximately 66.3% of those with inflammatory bowel diseases reported using the internet at least once a day, and approximately 14.7% reported being dissatisfied with their current health care. About 62.3% of those with inflammatory bowel diseases reported that they had looked up health information online, 16.3% of those with inflammatory bowel diseases reported that they had scheduled an appointment with a health care provider online, and 21.6% reported having used a computer to communicate with a health provider by email. We found that women who were self-regulating their care were more likely to look up health information online than others. Both middle-aged and older adults with inflammatory bowel diseases who were unsatisfied with their current health care were less likely to look up health information online. Frequent internet users who were worried about medical costs were more likely to look up health information online. Similarly, the results from our statistical models suggest that individuals with inflammatory bowel diseases who were frequent internet users were more likely to use the internet for specific health-related tasks. Additionally, women with inflammatory bowel diseases who reported being married were less likely to use the internet for specific health-related tasks. Conclusions: For those with inflammatory bowel diseases, there are additional socioeconomic and behavioral factors that impact the use of the internet for health information and health-related tasks. Future research should evaluate how these factors moderate the use of the internet and identify how online resources can support clinical populations in ways that improve access to information, support health self-management, and subsequently improve health outcomes.

Although beneficial for patients through its minimally invasive nature, laparoscopic surgery creates unique training challenges due to limited instrument maneuverability, absence of stereovision, and inadequate real-time feedback. Traditional training models rely on subjective instructor evaluations, which are time-consuming and lack objective error detection. This study evaluates the efficacy of an Automated Error Detection System (AEDS), designed to provide real-time feedback on mistouch error counts, in improving laparoscopic skill acquisition compared to conventional methods. Forty novice participants were recruited and randomized into Group A (AEDS-enhanced training) and Group B (traditional training). Group A underwent a crossover design: 10 min of baseline training without AEDS followed by 10 min with AEDS. Group B completed 20 min of traditional training. The training program encompassed standardized laparoscopic tasks designed to simulate real surgical procedures. Performance metrics, including task completion time and the number of errors made, were recorded for each participant through AEDS. Confidence levels were assessed through self-reported questionnaires. Furthermore, statistical analysis was performed to evaluate the effectiveness of AEDS. A paired t-test was utilized to assess error reductions within the AEDS group, and Bland-Altman analysis was used to analyze the self-estimate error bias. Also, a Wilcoxon signed-rank test evaluated improvements in confidence levels attributable to the system, while a Mann-Whitney U test was conducted to compare performance metrics between the AEDS and traditional training groups. Group A demonstrated a 24% reduction in errors post-AEDS (mean: 78.1 to 59.4, p < 0.001), outperforming Group B (mean: 67.4, p < 0.001). Participants significantly underestimated errors without AEDS (mean bias: +9.9 errors). Confidence levels in Group A increased from 2.4 to 3.6, significantly surpassing Group B’s improvement (median: 3) (p < 0.001). Real-time feedback bridged perceptual gaps, enhancing both technical precision and self-assessment accuracy. The integration of AEDS into laparoscopic training significantly reduces operational errors, accelerates skill acquisition, and boosts trainee confidence by providing objective feedback. These findings advocate for adopting AEDS in surgical education to standardize training outcomes, mitigate overconfidence, and improve patient safety. Future studies should explore AEDS scalability across advanced procedural modules and diverse trainee cohorts. Not applicable.