Model Predictive Control (MPC) of an Artificial Pancreas with Data-Driven Learning of Multi-Step-Ahead Blood Glucose Predictors

We present the design and \textit{in-silico} evaluation of a closed-loop insulin delivery algorithm to treat type 1 diabetes (T1D) consisting in a data-driven multi-step-ahead blood glucose (BG) predictor integrated into a Linear Time-Varying (LTV) Model Predictive Control (MPC) framework. Instead of identifying an open-loop model of the glucoregulatory system from available data, we propose to directly fit the entire BG prediction over a predefined prediction horizon to be used in the MPC, as a nonlinear function of past input-ouput data and an affine function of future insulin control inputs. For the nonlinear part, a Long Short-Term Memory (LSTM) network is proposed, while for the affine component a linear regression model is chosen. To assess benefits and drawbacks when compared to a traditional linear MPC based on an auto-regressive with exogenous (ARX) input model identified from data, we evaluated the proposed LSTM-MPC controller in three simulation scenarios: a nominal case with 3 meals per day, a random meal disturbances case where meals were generated with a recently published meal generator, and a case with 25$\%$ decrease in the insulin sensitivity. Further, in all the scenarios, no feedforward meal bolus was administered. For the more challenging random meal generation scenario, the mean $\pm$ standard deviation percent time in the range 70-180 [mg/dL] was 74.99 $\pm$ 7.09 vs. 54.15 $\pm$ 14.89, the mean $\pm$ standard deviation percent time in the tighter range 70-140 [mg/dL] was 47.78$\pm$8.55 vs. 34.62 $\pm$9.04, while the mean $\pm$ standard deviation percent time in sever hypoglycemia, i.e., $<$ 54 [mg/dl] was 1.00$\pm$3.18 vs. 9.45$\pm$11.71, for our proposed LSTM-MPC controller and the traditional ARX-MPC, respectively. Our approach provided accurate predictions of future glucose concentrations and good closed-loop performances of the overall MPC controller.Comment: 10 pages, 5 Figures, 2 Table

In-Silico Evaluation of Glucose Regulation Using Policy Gradient Reinforcement Learning for Patients with Type 1 Diabetes Mellitus

In this paper, we test and evaluate policy gradient reinforcement learning for automated blood glucose control in patients with Type 1 Diabetes Mellitus. Recent research has shown that reinforcement learning is a promising approach to accommodate the need for individualized blood glucose level control algorithms. The motivation for using policy gradient algorithms comes from the fact that adaptively administering insulin is an inherently continuous task. Policy gradient algorithms are known to be superior in continuous high-dimensional control tasks. Previously, most of the approaches for automated blood glucose control using reinforcement learning has used a finite set of actions. We use the Trust-Region Policy Optimization algorithm in this work. It represents the state of the art for deep policy gradient algorithms. The experiments are carried out in-silico using the Hovorka model, and stochastic behavior is modeled through simulated carbohydrate counting errors to illustrate the full potential of the framework. Furthermore, we use a model-free approach where no prior information about the patient is given to the algorithm. Our experiments show that the reinforcement learning agent is able to compete with and sometimes outperform state-of-the-art model predictive control in blood glucose regulation

Therapy-driven Deep Glucose Forecasting

none5noThe automatic regulation of blood glucose for Type 1 diabetes patients is the main goal of the artificial pancreas, a closed-loop system that exploits continue glucose monitoring data to define an optimal insulin therapy. One of the most successful approaches for developing the artificial pancreas is the model predictive control, which exhibits promising results on both virtual and real patients. The performance of such controller is highly dependent on the reliability of the glucose–insulin model used for prediction purpose, which is usually implemented with classic mathematical models. The main limitation of these models consists in the difficulties of modeling the physiological nonlinear dynamics typical of this system. The availability of big amount of in silico and in vivo data moved the attention to new data-driven methods which are able to easily overcome this problem. In this paper we propose Deep Glucose Forecasting, a deep learning approach for forecasting glucose levels, based on a novel, two-headed Long-Short Term Memory implementation. It takes in input the previous values obtained through continue glucose monitoring, the carbohydrate intake, the suggested insulin therapy and forecasts the interstitial glucose level of the patient. The proposed architecture has been trained on 100 virtual adult patients of the UVA/Padova simulator, and tested on both virtual and real patients. The proposed solution is able to generalize to new unseen data, outperforms classical population models and reaches performance comparable to classical personalized models when fine-tuning is exploited on real patients.noneAiello E.M.; Lisanti G.; Magni L.; Musci M.; Toffanin C.Aiello E.M.; Lisanti G.; Magni L.; Musci M.; Toffanin C