In this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level control algorithms. The motivation for using policy gradient algorithms comes from the fact that adaptively administering insulin is an inherently continuous task. Policy gradient algorithms are known to be superior in continuous high-dimensional control tasks. Previously, most of the approaches for automated blood glucose control using reinforcement learning has used a finite set of actions. We use the Trust-Region Policy Optimization algorithm in this work. It represents the state of the art for deep policy gradient algorithms. The experiments are carried out in-silico using the Hovorka model, and stochastic behavior is modeled through simulated carbohydrate counting errors to illustrate the full potential of the framework. Furthermore, we use a model-free approach where no prior information about the patient is given to the algorithm. Our experiments show that the reinforcement learning agent is able to compete with and sometimes outperform state-of-the-art model predictive control in blood glucose regulation.

Patients with type 1 diabetes must continually decide how much insulin to inject before each meal to maintain blood glucose levels within a healthy range. Recent research has worked on a solution for this burden, showing the potential of reinforcement learning as an emerging approach for the task of controlling blood glucose levels. In this paper, we test and evaluate several deep Q-learning algorithms for automated and personalized blood glucose regulation in an in silico type 1 diabetes patient with the goal of estimating and delivering proper insulin doses. The proposed algorithms are model-free approaches with no prior information about the patient. We used the Hovorka model with meal variation and carbohydrate counting errors to simulate the patient included in this work. Our experiments compare different deep Q-learning extensions showing promising results controlling blood glucose levels, with some of the proposed algorithms outperforming standard baseline treatment.

Surgical site infections are hospital-acquired infections resulting in severe risk for patients and significantly increased costs for healthcare providers. In this work, we show how to leverage irregularly sampled preoperative blood tests to predict, on the day of surgery, a future surgical site infection and its severity. Our dataset is extracted from the electronic health records of patients who underwent gastrointestinal surgery and developed either deep, shallow or no infection. We represent the patients using the concentrations of fourteen common blood components collected over the four weeks preceding the surgery partitioned into six time windows. A gradient boosting based classifier trained on our new set of features reports an AUROC of 0.991 for predicting a postoperative infection and and AUROC of 0.937 for classifying the severity of the infection. Further analyses support the clinical relevance of our approach as the most important features describe the nutritional status and the liver function over the two weeks prior to surgery.

Type 1 Diabetes is characterized by the lack of insulin-producing beta cells in the pancreas. The artificial pancreas promises to alleviate the burdens of self-management. While the physical components of the system - the continuous glucose monitor and insulin pump - have experienced rapid advances, a technological bottleneck remains in the control algorithm, which is responsible for translating data from the former into instructions for the latter. In this work, we propose to bring machine learning techniques to bear upon the challenges of blood glucose control. Specifically, we employ reinforcement learning to learn an optimal insulin policy. Learning is generalized using nonparametric regression with functional features, exploiting information contained in the shape of the glucose curve. Our algorithm is model-free, data-driven and personalized. In-silico simulations with T1D models demonstrate the potential of the proposed algorithm.

The aim of this paper is two-fold. First, we show that the newly developed spectral method known as kernel entropy component analysis (kernel ECA) captures cluster structure, which is very important in semi-supervised learning, and we provide an analysis showing how mixture weights influence kernel ECA in a mixture of cluster components setting. Second, we develop a semi-supervised kernel ECA classifier based on the Lasso framework, and report promising results compared to the state-of-the art.

In this paper we present two contributions to the framework of ensemble clustering. Our work expands the robust k nearest neighborhood clustering ensemble to include random projections for further increasing the stochastic exploration of the data set to be clustered. In addition we propose to constrain the random projection ensemble to only contain distance-preserving projection directions. The latter is obtained by constraining the projection directions to be orthogonal to the so-called Secant manifold of the input data set. Promising results are shown on a series of benchmark data sets.