In this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level control algorithms. The motivation for using policy gradient algorithms comes from the fact that adaptively administering insulin is an inherently continuous task. Policy gradient algorithms are known to be superior in continuous high-dimensional control tasks. Previously, most of the approaches for automated blood glucose control using reinforcement learning has used a finite set of actions. We use the Trust-Region Policy Optimization algorithm in this work. It represents the state of the art for deep policy gradient algorithms. The experiments are carried out in-silico using the Hovorka model, and stochastic behavior is modeled through simulated carbohydrate counting errors to illustrate the full potential of the framework. Furthermore, we use a model-free approach where no prior information about the patient is given to the algorithm. Our experiments show that the reinforcement learning agent is able to compete with and sometimes outperform state-of-the-art model predictive control in blood glucose regulation.

Background. Since physical activity has a high impact on patients with type 1 diabetes and the risk of hypoglycemia (low blood glucose levels) is significantly higher during and after physical activities, an automatic method to provide a personalized recommendation is needed to improve the blood glucose management and harness the benefits of physical activities. This paper aims to reduce the risk of hypoglycemia and hyperglycemia (high blood glucose levels), and empowers type 1 diabetes patients to make decisions regarding food choices connected with physical activities. Methods. Traditional and Bayesian feedforward neural network models are developed to provide accurate predictions of the blood glucose outcome and the risks of hyperglycemia and hypoglycemia with uncertainty information. Using the proposed models, safe actions that minimize the risk of both hypoglycemia and hyperglycemia are provided as food recommendations to the patient. Results. The predicted blood glucose responses to the optimal and safe food recommendations are significantly better and safer than by taking random food. Conclusions. Simulations conducted on the state-of-the-art UVA/Padova simulator combined with Brenton’s physical activity model show that the proposed methodology is safe and effective in managing blood glucose during and after physical activities.