Background: Decision support systems based on reinforcement learning (RL) have been implemented to facilitate the delivery of personalized care. This paper aimed to provide a comprehensive review of RL applications in the critical care setting. Objective: This review aimed to survey the literature on RL applications for clinical decision support in critical care and to provide insight into the challenges of applying various RL models. Methods: We performed an extensive search of the following databases: PubMed, Google Scholar, Institute of Electrical and Electronics Engineers (IEEE), ScienceDirect, Web of Science, Medical Literature Analysis and Retrieval System Online (MEDLINE), and Excerpta Medica Database (EMBASE). Studies published over the past 10 years (2010-2019) that have applied RL for critical care were included. Results: We included 21 papers and found that RL has been used to optimize the choice of medications, drug dosing, and timing of interventions and to target personalized laboratory values. We further compared and contrasted the design of the RL models and the evaluation metrics for each application. Conclusions: RL has great potential for enhancing decision making in critical care. Challenges regarding RL system design, evaluation metrics, and model choice exist. More importantly, further work is required to validate RL in authentic clinical environments.

Despite the overall success of using artificial intelligence (AI) to assist radiologists in performing computer-aided patient diagnosis, it remains challenging to build good models with small datasets at individual sites. Because many medical images do not come with proper labelling for training, this requires radiologists to perform strenuous labelling work and to prepare the dataset for training. Placing such demands on radiologists is unsustainable, given the ever-increasing number of medical images taken each year. We propose an alternative solution using a relatively new learning framework. This framework, called federated learning, allows individual sites to train a global model in a collaborative effort. Federated learning involves aggregating training results from multiple sites to create a global model without directly sharing datasets. This ensures that patient privacy is maintained across sites. Furthermore, the added supervision obtained from the results of partnering sites improves the global model’s overall detection abilities. This alleviates the issue of insufficient supervision when training AI models with small datasets. Lastly, we also address the major challenges of adopting federated learning.

In data-driven medical research, multi-center studies have long been preferred over single-center ones due to a single institute sometimes not having enough data to obtain sufficient statistical power for certain hypothesis testings as well as predictive and subgroup studies. The wide adoption of electronic health records (EHRs) has made multi-institutional collaboration much more feasible. However, concerns over infrastructures, regulations, privacy, and data standardization present a challenge to data sharing across healthcare institutions. Federated Learning (FL), which allows multiple sites to collaboratively train a global model without directly sharing data, has become a promising paradigm to break the data isolation. In this study, we surveyed existing works on FL applications in EHRs and evaluated the performance of current state-of-the-art FL algorithms on two EHR machine learning tasks of significant clinical importance on a real world multi-center EHR dataset.

Introduction: Limited information exists on the etiology, prevalence, and significance of hyperdynamic left ventricular ejection fraction (HDLVEF) in the intensive care unit (ICU). Our aim in the present study was to compare characteristics and outcomes of patients with HDLVEF with those of patients with normal left ventricular ejection fraction in the ICU using a large, public, deidentified critical care database. Methods: We conducted a longitudinal, single-center, retrospective cohort study of adult patients who underwent echocardiography during a medical or surgical ICU admission at the Beth Israel Deaconess Medical Center using the Multiparameter Intelligent Monitoring in Intensive Care II database. The final cohort had 2867 patients, of whom 324 had HDLVEF, defined as an ejection fraction >70 %. Patients with an ejection fraction <55 % were excluded. Results: Compared with critically ill patients with normal left ventricular ejection fraction, the finding of HDLVEF in critically ill patients was associated with female sex, increased age, and the diagnoses of hypertension and cancer. Patients with HDLVEF had increased 28-day mortality compared with those with normal ejection fraction in multivariate logistic regression analysis adjusted for age, sex, Sequential Organ Failure Assessment score, Elixhauser score for comorbidities, vasopressor use, and mechanical ventilation use (odds ratio 1.38, 95 % confidence interval 1.039–1.842, p =0.02). Conclusions: The presence of HDLVEF portended increased 28-day mortality, and may be helpful as a gravity marker for prognosis in patients admitted to the ICU. Further research is warranted to gain a better understanding of how these patients respond to common interventions in the ICU and to determine if pharmacologic modulation of HDLVEF improves outcomes.

Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet