Ye Jin Kim, BPT, PT, Joo-Hee Park, PhD, PT, Ji-hyun Kim, BPT, PT, Gyeong Ah Moon, BPT, PT, Hye-Seon Jeon, PhD, PT. Phys. Ther. Korea 2021;28:65-71. https://doi.org/10.12674/ptk.2021.28.1.65

Background/Objectives: Data availability can affect the performance of AI-based early warning scores (EWSs). This study evaluated how the extent of missing data and imputation strategies influence the predictive performance of the VitalCare–Major Adverse Event Score (VC-MAES), an AI-based EWS that uses last observation carried forward and normal-value imputation for missing values, to forecast clinical deterioration events, including unplanned ICU transfers, cardiac arrests, or death, up to 6 h in advance. Methods: We analyzed real-world data from 6039 patient encounters at Keimyung University Dongsan Hospital, Republic of Korea. Performance was evaluated under three scenarios: (1) using only vital signs and age, treating all other variables as missing; (2) reintroducing a full set of real-world clinical variables; and (3) imputing missing values drawn from a distribution within one standard deviation of the observed mean or using Multiple Imputation by Chained Equations (MICE). Results: VC-MAES achieved the area under the receiver operating characteristic curve (AUROC) of 0.896 using only vital signs and age, outperforming traditional EWSs, including the National Early Warning Score (0.797) and the Modified Early Warning Score (0.722). Reintroducing full clinical variables improved the AUROC to 0.918, whereas mean-based imputation or MICE decreased the performance to 0.885 and 0.827, respectively. Conclusions: VC-MAES demonstrates robust predictive performance with limited inputs, outperforming traditional EWSs. Incorporating actual clinical data significantly improved accuracy. In contrast, mean-based or MICE imputation yielded poorer results than the default normal-value imputation, potentially due to disregarding the “informative presence” embedded in missing data patterns. These findings underscore the importance of understanding missingness patterns and employing imputation strategies that consider the decision-making context behind data availability to enhance model reliability.

Objective: The heterogeneous nature of sepsis renders determining its underlying causes difficult, which may delay diagnosis and intervention. VitalCare-SEPsis Score (VC-SEPS) is a deep learning-based algorithm that predicts sepsis and monitors patient conditions based on electronic medical record data. However, few studies have prospectively compared medical artificial intelligence software algorithms and traditional scoring systems to predict sepsis. This prospective observational study attempted to validate the predictive performance and risk stratification of VC-SEPS for early prediction of sepsis. Methods: In this prospective observational study, we collected electronic medical record data from 6,797 patients hospitalised at Keimyung University Dongsan Hospital, Daegu, South Korea. The final version of the analysed set included 6,455 patients, 325 of whom were diagnosed with sepsis. Results: The area under the receiver operating characteristic curve of VC-SEPS was 0.880, indicating its superiority over traditional scoring systems. The algorithm performance showed a consistent trend within 24 hours. On patients’ initial admission, the VC-SEPS was associated with the risk of developing sepsis, and the score accurately predicted sepsis by an average of 68.05 min compared with diagnosis time by an operational definition of sepsis. Discussion: VC-SEPS could assist medical staff with early diagnosis and intervention in clinical practice by providing a sepsis risk score. Prompt recognition assisting recognition can significantly help shorten the time between recognition and intervention in clinical decision-making processes. Conclusion: This study suggests that using a clinical decision support system can help improve hospital workflows as well as the quality of medical care.

Background: Untact cultures have rapidly spread around the world as a result of the prolongation of the COVID-19 pandemic, leading to various types of research and technological developments in the fields of medicine and health care, where digital health care refers to health care services provided in a digital environment. Previous studies relating to digital health care demonstrated its effectiveness in managing chronic diseases such as hypertension and diabetes. While many studies have applied digital health care to various diseases, daily health care is needed for healthy individuals before they are diagnosed with a disease. Accordingly, research on individuals who have not been diagnosed with a disease is also necessary. Objective: This study aimed to identify the effects of using a customized digital health care service (CDHCS) on risk factors for metabolic syndrome (MS) and lifestyle improvement. Methods: The population consisted of 63 adults who underwent a health checkup at the National Health Insurance Service Ilsan (NHIS) Hospital in 2020. Measured variables include basic clinical indicators, MS-related variables, and lifestyle variables. All items were measured at NHIS Ilsan Hospital before the use of the CDHCS and 3 months thereafter. The CDHCS used in this study is a mobile app that analyzes the health condition of the user by identifying their risk factors and provides appropriate health care content. For comparison between before and after CDHCS use (pre-post comparison), paired t test was used for continuous variables, and a chi-square test was used for nominal variables. Results: The study population included 30 (47.6%) male and 33 (52.4%) female participants, and the mean age was 47.61 (SD 13.93) years. The changes in clinical indicators before and after intervention results showed a decrease in weight, waist circumference, triglyceride, and high-density lipoprotein cholesterol and increases in systolic blood pressure and diastolic blood pressure. The distribution of the risk group increased from 32 (50.8%) to 34 (54%) and that of the MS group decreased from 18 (28.6%) to 16 (25.4%). The mean metabolic syndrome age–chronological age before the CDHCS was 2.20 years, which decreased to 1.72 years after CDHCS, showing a decrease of 0.48 years in the mean metabolic syndrome age–chronological age after the intervention. While all lifestyle variables, except alcohol consumption, showed a tendency toward improvement, the differences were not statistically significant. Conclusions: Although there was no statistical significance in the variables under study, this pilot study will provide a foundation for more accurate verification of CDHCS in future research.

Background: Acute deterioration of patients in general wards often leads to major adverse events (MAEs), including unplanned intensive care unit transfers, cardiac arrest, or death. Traditional early warning scores (EWSs) have shown limited predictive accuracy, with frequent false positives. We conducted a prospective observational external validation study of an artificial intelligence (AI)-based EWS, the VitalCare - Major Adverse Event Score (VC-MAES), at a tertiary medical center in the Republic of Korea. Methods: Adult patients from general wards, including internal medicine (IM) and obstetrics and gynecology (OBGYN)—the latter were rarely investigated in prior AI-based EWS studies—were included. The VC-MAES predictions were compared with National Early Warning Score (NEWS) and Modified Early Warning Score (MEWS) predictions using the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and logistic regression for baseline EWS values. False-positives per true positive (FPpTP) were assessed based on the power threshold. Results: Of 6,039 encounters, 217 (3.6%) had MAEs (IM: 9.5%, OBGYN: 0.26%). Six hours prior to MAEs, the VC-MAES achieved an AUROC of 0.918 and an AUPRC of 0.352, including the OBGYN subgroup (AUROC, 0.964; AUPRC, 0.388), outperforming the NEWS (0.797 and 0.124) and MEWS (0.722 and 0.079). The FPpTP was reduced by up to 71%. Baseline VC-MAES was strongly associated with MAEs (P<0.001). Conclusions: The VC-MAES significantly outperformed traditional EWSs in predicting adverse events in general ward patients. The robust performance and lower FPpTP suggest that broader adoption of the VC-MAES may improve clinical efficiency and resource allocation in general wards.

Background: In hospitals, Code Blue is an emergency that refers to a patient requiring immediate resuscitation. Over 85% of patients with cardiopulmonary arrest exhibit abnormal vital sign trends prior to the event. Continuous monitoring and accurate interpretation of clinical data through artificial intelligence (AI) models can contribute to preventing critical events. Objective: This study aims to evaluate changes in clinical outcomes following the use of VitalCare (Major Adverse Event Score and Mortality Score), which is an AI-based early warning system, and to validate the performance of the algorithm. Methods: A retrospective analysis was conducted by extracting electronic health record data, using a total of 30,785 inpatient cases from general wards and intensive care units. A comparative analysis was performed by setting a 3-month period before and after the system implementation. For clinical evaluation, we measured the incidence rates of Code Blue and adverse events, the proportion of prolonged hospitalization, and the frequency of early interventions. The area under the receiver operating characteristic curve (AUROC) was calculated to assess the performance of the algorithm. Results: This study demonstrated that, following the implementation of VitalCare, there was a 24.97% reduction in major events such as Code Blue (P=.004) and the proportion of prolonged hospitalization in general wards (P<.05), along with a significant increase in the rate of early interventions. The model performance exhibited superior outcomes compared with traditional scoring systems, with a Major Adverse Event Score AUROC of 0.865 (95% CI 0.857‐0.873) and Mortality Score AUROC of 0.937 (95% CI 0.931‐0.944). Conclusions: A well-developed AI-based model that provides high predictive power can contribute to the prevention of major in-hospital events by providing early predictive information to clinicians. Additionally, it plays a crucial role in effectively addressing unmet needs and challenges in terms of human resources and practical procedures.