This study aimed to develop a machine learning-based clinical decision support system for emergency departments based on the decision-making framework of physicians. We extracted 27 fixed and 93 observation features using data on vital signs, mental status, laboratory results, and electrocardiograms during emergency department stay. Outcomes included intubation, admission to the intensive care unit, inotrope or vasopressor administration, and in-hospital cardiac arrest. eXtreme gradient boosting algorithm was used to learn and predict each outcome. Specificity, sensitivity, precision, F1 score, area under the receiver operating characteristic curve (AUROC), and area under the precision-recall curve were assessed. We analyzed 303,345 patients with 4,787,121 input data, resampled into 24,148,958 1 h-units. The models displayed a discriminative ability to predict outcomes (AUROC > 0.9), and the model with lagging 6 and leading 0 displayed the highest value. The AUROC curve of in-hospital cardiac arrest had the smallest change, with increased lagging for all outcomes. With inotropic use, intubation, and intensive care unit admission, the range of AUROC curve change with the leading 6 was the highest according to different amounts of previous information (lagging). In this study, a human-centered approach to emulate the clinical decision-making process of emergency physicians has been adopted to enhance the use of the system. Machine learning-based clinical decision support systems customized according to clinical situations can help improve the quality of care.

The array of complex and evolving patient data has limited clinical decision making in the emergency department (ED). This study introduces an advanced deep learning algorithm designed to enhance real-time prediction accuracy for integration into a novel Clinical Decision Support System (CDSS). A retrospective study was conducted using data from a level 1 tertiary hospital. The algorithm’s predictive performance was evaluated based on in-hospital cardiac arrest, inotropic circulatory support, advanced airway, and intensive care unit admission. We developed an artificial intelligence (AI) algorithm for CDSS that integrates multiple data modalities, including vitals, laboratory, and imaging results from electronic health records. The AI model was trained and tested on a dataset of 237,059 ED visits. The algorithm’s predictions, based solely on triage information, significantly outperformed traditional logistic regression models, with notable improvements in the area under the precision-recall curve (AUPRC). Additionally, predictive accuracy improved with the inclusion of continuous data input at shorter intervals. This study suggests the feasibility of using AI algorithms in diverse clinical scenarios, particularly for earlier detection of clinical deterioration. Future work should focus on expanding the dataset and enhancing real-time data integration across multiple centers to further optimize its application within the novel CDSS.

Importance: Emergency department (ED) discharge documentation is time-consuming and often incomplete.Objective: To develop a large language model (LLM) assistant that generates ED discharge notes and to evaluate its effectiveness on documentation quality and workflow efficiency.Design, Setting, and Participants: This comparative effectiveness study, which was conducted at a 2400-bed tertiary care hospital in South Korea, consisted of 2 primary phases: a development phase and sequential validation of the LLM assistant. In the randomized sequential prospective validation, 6 emergency physicians first wrote discharge notes manually (session 1), then edited LLM-generated drafts after a 1-hour washout period (session 2). Three independent physicians evaluated 300 note sets (each containing a manual note, an LLM draft, and an LLM-assisted note). For model development and validation, patient records from ED visits between September 1, 2022, and August 31, 2023, were used. The inclusion criteria encompassed adult patients (aged ≥17 years) and pediatric patients with nondisease conditions (eg, trauma, poisoning, or burns). Emergency physicians selected 592 representative cases for training and 50 for validation.Exposure: A commercially available text generation transformer model was used as a core LLM, fine-tuned using the 592 training cases. Two distinct processing pipelines were implemented within the LLM assistant due to different input data: (1) for patients managed solely by emergency physicians, using the ED initial record and prescription list, and (2) for those requiring specialty consultations, using the ED initial record and consultation request form.Main Outcomes and Measures: Quality of notes using 4C metrics (completeness, correctness, conciseness, and clinical utility) on a Likert scale ranging from 1 to 5 and time taken to complete the notes manually and with the LLM assistant.Results: Of the 50 test cases, the mean (SD) patient age was 57.7 (23.1) years, and 28 patients (56%) were female. LLM-assisted notes achieved higher scores than manual notes in completeness (4.23 [95% CI, 4.17-4.28] vs 4.03 [95% CI, 3.96-4.09]), correctness (4.38 [95% CI, 4.33-4.42] vs 4.20 [95% CI, 4.14-4.26]), conciseness (4.23 [95% CI, 4.18-4.28] vs 4.11 [95% CI, 4.05-4.17]), and clinical utility (4.17 [95% CI, 4.11-4.23] vs 3.85 [95% CI, 3.78-3.91]) (all P < .001). When compared with LLM drafts, LLM-assisted notes excelled in conciseness (4.23 vs 3.98 [95% CI, 3.91-4.04]; P < .001) and maintained equivalent clinical utility (4.17 vs 4.16 [95% CI, 4.11-4.21]; P > .99), but scored lower in completeness (4.23 vs 4.34 [95% CI, 4.29-4.39]; P = .001) and correctness (4.38 vs 4.45 [95% CI, 4.41-4.49]; P < .001). The median documentation time per note dropped from 69.5 (95% CI, 65.5-78.0) seconds for manual notes to 32.0 (95% CI, 29.5-36.0) seconds for LLM-assisted notes (P < .001).Conclusion: In this comparative effectiveness study, use of an on-site LLM assistant was associated with reduced writing time for ED discharge notes compared with manual note-taking, without compromising documentation quality, representing a critical advancement in the use of artificial intelligence for clinical practice.

The Korean Triage and Acuity Scale (KTAS) was developed based on the Canadian Emergency Department Triage and Acuity Scale. In patients with pain, to determine the KTAS level, the pain scale is considered; however, since the degree of pain is subjective, this may affect the accuracy of KTAS. The purpose of this study was to evaluate the accuracy of KTAS in predicting patient's severity with the degree of pain used as a modifier. A retrospective observational cohort study was conducted in an urban tertiary hospital emergency department (ED). We investigated patients over 16 years old from January to June 2016. The patients were divided into the pain and non-pain groups according to whether the degree of pain was used as a modifier or not. We compared the predictive power of KTAS on the urgency of patients between the two groups. Acute area registration in the ED, emergency procedure, emergency operation, hospitalization, intensive care unit admission, and 7-day mortality were used as markers to determine urgent patients. Overall, 24,253 patients were included in the study, with 9,175 (37.8%) in the pain group. The proportions of patients with KTAS 1–3 were 61.4% in the pain and 75.6% in the non-pain groups. Among patients with KTAS 2–3, the proportion of urgent patients was higher in the non-pain group than the pain group (p<0.001). The odds ratios for urgent patients at each KTAS level revealed a more evident discriminatory power of KTAS for urgent patients in the non-pain group. The predictability of KTAS for urgent patients was higher in the non-pain group than the pain group (area under the curve; 0.736 vs. 0.765, p-value <0.001). Considering the degree of pain with KTAS led to overestimation of patient severity and had a negative impact on the predictability of KTAS for urgent patients.

Intermittent manual measurement of vital signs may not rapidly predict sepsis development in febrile patients admitted to the emergency department (ED). We aimed to evaluate the predictive performance of a wireless monitoring device that continuously measures heart rate (HR) and respiratory rate (RR) and a machine learning analysis in febrile but stable patients in the ED. We analysed 468 patients (age, ≥18 years; training set, n = 277; validation set, n = 93; test set, n = 98) having fever (temperature >38 °C) and admitted to the isolation care unit of the ED. The AUROC of the fragmented model with device data was 0.858 (95% confidence interval [CI], 0.809–0.908), and that with manual data was 0.841 (95% CI, 0.789–0.893). The AUROC of the accumulated model with device data was 0.861 (95% CI, 0.811–0.910), and that with manual data was 0.853 (95% CI, 0.803–0.903). Fragmented and accumulated models with device data detected clinical deterioration in febrile patients at risk of septic shock 9 h and 5 h 30 min earlier, respectively, than those with manual data. Continuous vital sign monitoring using a wearable device could accurately predict clinical deterioration and reduce the time to recognise potential clinical deterioration in stable ED patients with fever.