Background: The advancement of large language models (LLMs) offers significant opportunities for health care, particularly in the generation of medical documentation. However, challenges related to ensuring the accuracy and reliability of LLM outputs, coupled with the absence of established quality standards, have raised concerns about their clinical application. Objective: This study aimed to develop and validate an evaluation framework for assessing the accuracy and clinical applicability of LLM-generated emergency department (ED) records, aiming to enhance artificial intelligence integration in health care documentation. Methods: We organized the Healthcare Prompt-a-thon, a competitive event designed to explore the capabilities of LLMs in generating accurate medical records. The event involved 52 participants who generated 33 initial ED records using HyperCLOVA X, a Korean-specialized LLM. We applied a dual evaluation approach. First, clinical evaluation: 4 medical professionals evaluated the records using a 5-point Likert scale across 5 criteria—appropriateness, accuracy, structure/format, conciseness, and clinical validity. Second, quantitative evaluation: We developed a framework to categorize and count errors in the LLM outputs, identifying 7 key error types. Statistical methods, including Pearson correlation and intraclass correlation coefficients (ICC), were used to assess consistency and agreement among evaluators. Results: The clinical evaluation demonstrated strong interrater reliability, with ICC values ranging from 0.653 to 0.887 (P<.001), and a test-retest reliability Pearson correlation coefficient of 0.776 (P<.001). Quantitative analysis revealed that invalid generation errors were the most common, constituting 35.38% of total errors, while structural malformation errors had the most significant negative impact on the clinical evaluation score (Pearson r=–0.654; P<.001). A strong negative correlation was found between the number of quantitative errors and clinical evaluation scores (Pearson r=–0.633; P<.001), indicating that higher error rates corresponded to lower clinical acceptability. Conclusions: Our research provides robust support for the reliability and clinical acceptability of the proposed evaluation framework. It underscores the framework’s potential to mitigate clinical burdens and foster the responsible integration of artificial intelligence technologies in health care, suggesting a promising direction for future research and practical applications in the field.

Mobile and tablet apps, ubiquitous and pervasive computing, wearable computing and domotics for health.

Objective We evaluated the performance of diastolic shock index (DSI) and lactate in predicting vasopressor requirement among hypotensive patients with suspected infection in an emergency department.Methods This was a single-center, retrospective observational study for adult patients with suspected infection and hypotension in the emergency department from 2018 to 2019. The study population was split into derivation and validation cohorts (70/30). We derived a simple risk score to predict vasopressor requirement using DSI and lactate cutoff values determined by Youden index. We tested the score by the area under the receiver operating characteristic curve (AUC). We performed a multivariable regression analysis to evaluate the association between the timing of vasopressor treatment and 28-day mortality.Results A total of 1,917 patients were included. We developed a score, assigning 1 point each for the high DSI (≥2.0) and high lactate (≥2.5 mmol/L) criteria. The AUCs of the score were 0.741 (95% confidence interval [CI], 0.715–0.768) at hypotension and 0.736 (95% CI, 0.708–0.763) after initial fluid challenge in the derivation cohort and 0.676 (95% CI, 0.631–0.719) at hypotension and 0.688 (95% CI, 0.642–0.733) after initial fluid challenge in the validation cohort, respectively. In patients with scores of 2 points, early vasopressor therapy initiation was significantly associated with decreased 28-day mortality (adjusted odds ratio, 0.37; 95% CI, 0.14–0.94).Conclusion A prediction model with DSI and lactate levels might be useful to identify patients who are more likely to need vasopressor administration among hypotensive patients with suspected infection.

Objective: This study analyzed trends in emergency department (ED) visits in South Korea using the National Emergency Department Information System (NEDIS) data from 2018 to 2022. Methods: This was a retrospective observational study using data from the NEDIS database from 2018 to 2022. Age- and sex-standardized ED visits per 100,000 population, as well as age- and sex-standardized rates for mortality, admission, and transfer, were calculated. Results: The standardized ED visits per 100,000 population was approximately 20,000 from 2018 to 2019 and decreased to about 18,000 in 2022. The standardized mortality rate ranged from 1.4% to 1.7%. The admission rate (18.4%–19.4%) and the transfer rates (1.6%–1.8%) were similar during the study period. Approximately 5.5% of patients were triaged as Korean Triage and Acuity Scale score 1 or 2. About 91% of patients visited the ED directly and 21.7% of patients visited the ED with an ambulance. The ED length of stay was less than 6 hours in 90.3% of patients and the ED mortality rate was 0.6%. Acute gastroenteritis was the most common diagnosis. Respiratory virus symptoms, such as fever and sore throat, were also common chief complaints.Conclusion: ED visits decreased during the 5-year period, while admission, transfer, and death rates remained relatively stable.

Background and Objectives: Ultrasonography is useful in evaluating patients with renal colic and it has high sensitivity and specificity for diagnosing ureter stones by revealing hydronephrosis. We evaluated the efficacy of point-of-care ultrasonography protocol in managing patients with acute renal colic who visited the emergency department (ED). Materials and Methods: Between March 2019 and July 2019, patients who visited the ED because of renal colic were randomly assigned by date of visit either to the conventional group (CG), who underwent routine diagnostic work-up without ultrasonography, or to the ultrasonography group (UG), who underwent bedside ultrasonography as an initial diagnostic testing. When hydronephrosis was detected in the UG group, a confirmatory non-contrast abdomen computed tomography scan was promptly performed. The ED length of stay, complications, and missed or delayed high-risk diagnosis were evaluated. Results: In total, 128 of 147 analyzed patients were confirmed to have ureter stones. The ED length of stay was significantly lower in the UG group than in the CG group (mean 172 min; 95% confidence interval (CI): 151–194 min vs. mean 234 min; 95% CI: 216–252 min). The medical cost was also remarkably lower in the UG group than in the CG group (259 USD vs. 319 USD; p < 0.001). The incidence of complications within 30 days after visiting ED and missed or delayed high-risk diagnosis were not significantly different between the two groups. Conclusions: We found that protocolized point-of-care ultrasonography in patients with acute renal colic who visited the ED can more effectively reduce the length of stay and medical cost without 30-day complication than usual clinical practice.

This study aims to introduce a novel blockchain-applied personal health records (PHR) application and validate its user experience. The system transmits the part corresponding to the patient’s personal information off-chain and prevents data forgery and falsification by storing encrypted data on-chain. Patients may easily trace the opt-in and opt-out history of their consent data and dynamically store the consent system for data exchange on the blockchain. A mixed-method study using a questionnaire, in-depth interviews, and usability evaluation were conducted for 30 participants. The system usability score was 74.0, indicating the high usability of the application. Those who were familiar with blockchain showed confidence in the application, but those unfamiliar wanted their data to be safe using another way. Most of the participants were interested in exchanging and using their medical data and considered security important but those unfamiliar wanted their data to be safe using another way. We found that participants were concerned about data security and considered a blockchain-based PHR as a novel way to store and exchange their medical information securely. Blockchain is not a visible technology. However, a blockchain-applied PHR must be able to win user trust through visualizations, certificates, and system descriptions.