The aim of this study was to develop machine learning (ML) and initial nursing assessment (INA)-based emergency department (ED) triage to predict adverse clinical outcome. The retrospective study included ED visits between January 2016 and December 2017 that resulted in either intensive care unit admission or emergency room death. We trained four classifiers using logistic regression and a deep learning model on INA and low dimensional (LD) INA, logistic regression on the Korea Triage and acuity scale (KTAS) and Sequential Related Organ Failure Assessment (SOFA). We varied the outcome ratio for external validation. Finally, variables of importance were identified using the random forest model's information gain. The four most influential variables were used for LD modeling for efficiency. A total of 86,304 patient visits were included, with an overall outcome rate of 3.5%. The area under the curve (AUC) values for the KTAS model were 76.8 (74.9-78.6) with logistic regression and 74.0 (72.1-75.9) for the SOFA model, while the AUC values of the INA model were 87.2 (85.9-88.6) and 87.6 (86.3-88.9) with logistic regression and deep learning, suggesting that the ML and INA-based triage system result more accurately predicted the outcomes. The AUC values for the LD model were 81.2 (79.4-82.9) and 80.7 (78.9-82.5) for logistic regression and deep learning, respectively. We developed an ML and INA-based triage system for EDs. The novel system was able to predict clinical outcomes more accurately than existing triage systems, KTAS and SOFA.

(1) Background: During a pandemic, patients and processes in the emergency department (ED) change. These circumstances affect the length of stay (LOS) or degree of crowding in the ED. The processes for patients with acute critical illness, such as cerebrovascular disease (CVD), can be also delayed. Using the process mining (PM) method, this study aimed to evaluate LOS, ED processes for CVD, and delayed processes during the coronavirus disease 2019 (COVID-19) pandemic. (2) Methods: Data were collected from the Clinical Data Warehouse of a medical center. Phase 1 included patients who visited the ED before the COVID-19 outbreak. In Phase 2, post-COVID-19 ED patients were divided into the COVID-19 tested group (CTG) and COVID-19 not tested group (CNTG) according to whether polymerase chain reaction test was performed. We analyzed patients’ ED processes before and after COVID-19 using the PM method. We analyzed patients with acute CVD separately to determine whether the process and LOS of patients with acute critical illness were changed or delayed. (3) Results: After the COVID-19 outbreak, the overall LOS was delayed and all processes in CTG patients were delayed. Registration to triage and triage were delayed in both CTG and CNTG patients. The brain imaging process for CTG patients with acute CVD was also delayed. (4) Conclusion: After a pandemic, some processes were changed, new processes were developed, and processes for patients with acute CVD who needed proper time management were not exempted.

Objective Falls are one of the most frequently occurring adverse events among hospitalized patients. The Morse Fall Scale, which has been widely used for fall risk assessment, has the two limitations of low specificity and difficulty in practical implementation. The aim of this study was to develop and validate an interpretable machine learning model for prediction of falls to be integrated in an electronic medical record (EMR) system.Methods This was a retrospective study involving a tertiary teaching hospital in Seoul, Korea. Based on the literature, 83 known predictors were grouped into seven categories. Interpretable fall event prediction models were developed using multiple machine learning models including gradient boosting and Shapley values.Results Overall, 191,778 cases with 272 fall events (0.1%) were included in the analysis. With the validation cohort of 2020, the area under the receiver operating curve (AUROC) of the gradient boosting model was 0.817 (95% confidence interval [CI], 0.720–0.904), better performance than random forest (AUROC, 0.801; 95% CI, 0.708–0.890), logistic regression (AUROC, 0.802; 95% CI, 0.721–0.878), artificial neural net (AUROC, 0.736; 95% CI, 0.650–0.821), and conventional Morse fall score (AUROC, 0.652; 95% CI, 0.570–0.715). The model’s interpretability was enhanced at both the population and patient levels. The algorithm was later integrated into the current EMR system.Conclusion We developed an interpretable machine learning prediction model for inpatient fall events using EMR integration formats.

Background: Alert fatigue is unavoidable when many irrelevant alerts are generated in response to a small number of useful alerts. It is necessary to increase the effectiveness of the clinical decision support system (CDSS) by understanding physicians’ responses. Objective: This study aimed to understand the CDSS and physicians’ behavior by evaluating the clinical appropriateness of alerts and the corresponding physicians’ responses in a medication-related passive alert system. Methods: Data on medication-related orders, alerts, and patients’ electronic medical records were analyzed. The analyzed data were generated between August 2019 and June 2020 while the patient was in the emergency department. We evaluated the appropriateness of alerts and physicians’ responses for a subset of 382 alert cases and classified them. Results: Of the 382 alert cases, only 7.3% (n=28) of the alerts were clinically appropriate. Regarding the appropriateness of the physicians’ responses about the alerts, 92.4% (n=353) were deemed appropriate. In the classification of alerts, only 3.4% (n=13) of alerts were successfully triggered, and 2.1% (n=8) were inappropriate in both alert clinical relevance and physician’s response. In this study, the override rate was 92.9% (n=355). Conclusions: We evaluated the appropriateness of alerts and physicians’ responses through a detailed medical record review of the medication-related passive alert system. An excessive number of unnecessary alerts are generated, because the algorithm operates as a rule base without reflecting the individual condition of the patient. It is important to maximize the value of the CDSS by comprehending physicians’ responses.

Kim M, et al. J Korean Med Sci. 2022 Oct;37(44):e317. https://doi.org/10.3346/jkms.2022.37.e317

Background: Acute kidney injury (AKI) is a marker of clinical deterioration and renal toxicity. While there are many studies offering prediction models for the early detection of AKI, those predicting AKI occurrence using distributed research network (DRN)–based time series data are rare. Objective: In this study, we aimed to detect the early occurrence of AKI by applying an interpretable long short-term memory (LSTM)–based model to hospital electronic health record (EHR)–based time series data in patients who took nephrotoxic drugs using a DRN. Methods: We conducted a multi-institutional retrospective cohort study of data from 6 hospitals using a DRN. For each institution, a patient-based data set was constructed using 5 drugs for AKI, and an interpretable multivariable LSTM (IMV-LSTM) model was used for training. This study used propensity score matching to mitigate differences in demographics and clinical characteristics. Additionally, the temporal attention values of the AKI prediction model’s contribution variables were demonstrated for each institution and drug, with differences in highly important feature distributions between the case and control data confirmed using 1-way ANOVA. Results: This study analyzed 8643 and 31,012 patients with and without AKI, respectively, across 6 hospitals. When analyzing the distribution of AKI onset, vancomycin showed an earlier onset (median 12, IQR 5-25 days), and acyclovir was the slowest compared to the other drugs (median 23, IQR 10-41 days). Our temporal deep learning model for AKI prediction performed well for most drugs. Acyclovir had the highest average area under the receiver operating characteristic curve score per drug (0.94), followed by acetaminophen (0.93), vancomycin (0.92), naproxen (0.90), and celecoxib (0.89). Based on the temporal attention values of the variables in the AKI prediction model, verified lymphocytes and calcvancomycin ium had the highest attention, whereas lymphocytes, albumin, and hemoglobin tended to decrease over time, and urine pH and prothrombin time tended to increase. Conclusions: Early surveillance of AKI outbreaks can be achieved by applying an IMV-LSTM based on time series data through an EHR-based DRN. This approach can help identify risk factors and enable early detection of adverse drug reactions when prescribing drugs that cause renal toxicity before AKI occurs.