On-line policy learning and adaptation for real-time personalization of an artificial pancreas

The dynamic complexity of the glucose-insulin metabolism in diabetic patients is the main obstacle towards widespread use of an artificial pancreas. The significant level of subject-specific glycemic variability requires continuously adapting the control policy to successfully face daily changes in patient´s metabolism and lifestyle. In this paper, an on-line selective reinforcement learning algorithm that enables real-time adaptation of a control policy based on ongoing interactions with the patient so as to tailor the artificial pancreas is proposed. Adaptation includes two online procedures: on-line sparsification and parameter updating of the Gaussian process used to approximate the control policy. With the proposed sparsification method, the support data dictionary for on-line learning is modified by checking if in the arriving data stream there exists novel information to be added to the dictionary in order to personalize the policy. Results obtained in silico experiments demonstrate that on-line policy learning is both safe and efficient for maintaining blood glucose variability within the normoglycemic range.Fil: de Paula, Mariano. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ingeniería Olavarria. Departamento de Electromecánica. Grupo INTELYMEC; Argentina. Universidad Nacional del Centro de la Pcia.de Bs.as.. Centro de Investigaciones En Fisica E Ingenieria del Centro de la Provincia de Buenos Aires. - Consejo Nacional de Investigaciones Cientificas y Tecnicas. Centro Cientifico Tecnologico Conicet - Tandil. Centro de Investigaciones En Fisica E Ingenieria del Centro de la Provincia de Buenos Aires. - Provincia de Buenos Aires. Gobernacion. Comision de Invest.cientificas. Centro de Investigaciones En Fisica E Ingenieria del Centro de la Provincia de Buenos Aires; ArgentinaFil: Acosta, Gerardo Gabriel. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ingenieria Olavarria; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Martinez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentin

Anomaly Detection in the Artificial Pancreas

The integration of subcutaneous sensing and insulin delivery technologies with novel control strategies has brought closer the development of the Artificial Pancreas. Nevertheless, thought recent developments are aimed at preventing chronic complications and less patient discomfort, few works have addressed the critical issue of performance monitoring of the artificial pancreas as well as detection of abnormal functioning in any of its components. This work presents an anomaly detection monitoring tool using the widely known Clarke Error-Grid to identify functional degradation in the artificial pancreas components and guarantee safetycritical control of blood glucose levels. The effect of imperfect calibration of glucose sensors, time lag between blood glucose concentration and interstitial glucose readings, and excessive variability in glucose levels are evaluated against an expected behavior of the glucose regulation loop achieved through an optimal control policy. Results obtained evidence the feasibility of this novel use of the Clarke error grid as a comprehensive monitoring tool for the artificial pancreas.Sociedad Argentina de Informática e Investigación Operativ

Anomaly Detection in the Artificial Pancreas

The integration of subcutaneous sensing and insulin delivery technologies with novel control strategies has brought closer the development of the Artificial Pancreas. Nevertheless, thought recent developments are aimed at preventing chronic complications and less patient discomfort, few works have addressed the critical issue of performance monitoring of the artificial pancreas as well as detection of abnormal functioning in any of its components. This work presents an anomaly detection monitoring tool using the widely known Clarke Error-Grid to identify functional degradation in the artificial pancreas components and guarantee safetycritical control of blood glucose levels. The effect of imperfect calibration of glucose sensors, time lag between blood glucose concentration and interstitial glucose readings, and excessive variability in glucose levels are evaluated against an expected behavior of the glucose regulation loop achieved through an optimal control policy. Results obtained evidence the feasibility of this novel use of the Clarke error grid as a comprehensive monitoring tool for the artificial pancreas.Sociedad Argentina de Informática e Investigación Operativ

Anomaly Detection in the Artificial Pancreas

The integration of subcutaneous sensing and insulin delivery technologies with novel control strategies has brought closer the development of the Artificial Pancreas. Nevertheless, thought recent developments are aimed at preventing chronic complications and less patient discomfort, few works have addressed the critical issue of performance monitoring of the artificial pancreas as well as detection of abnormal functioning in any of its components. This work presents an anomaly detection monitoring tool using the widely known Clarke Error-Grid to identify functional degradation in the artificial pancreas components and guarantee safetycritical control of blood glucose levels. The effect of imperfect calibration of glucose sensors, time lag between blood glucose concentration and interstitial glucose readings, and excessive variability in glucose levels are evaluated against an expected behavior of the glucose regulation loop achieved through an optimal control policy. Results obtained evidence the feasibility of this novel use of the Clarke error grid as a comprehensive monitoring tool for the artificial pancreas.Sociedad Argentina de Informática e Investigación Operativ

Reinforcement learning application in diabetes blood glucose control: A systematic review

Background: Reinforcement learning (RL) is a computational approach to understanding and automating goal-directed learning and decision-making. It is designed for problems which include a learning agent interacting with its environment to achieve a goal. For example, blood glucose (BG) control in diabetes mellitus (DM), where the learning agent and its environment are the controller and the body of the patient respectively. RL algorithms could be used to design a fully closed-loop controller, providing a truly personalized insulin dosage regimen based exclusively on the patient’s own data. Objective: In this review we aim to evaluate state-of-the-art RL approaches to designing BG control algorithms in DM patients, reporting successfully implemented RL algorithms in closed-loop, insulin infusion, decision support and personalized feedback in the context of DM. Methods: An exhaustive literature search was performed using different online databases, analyzing the literature from 1990 to 2019. In a first stage, a set of selection criteria were established in order to select the most relevant papers according to the title, keywords and abstract. Research questions were established and answered in a second stage, using the information extracted from the articles selected during the preliminary selection. Results: The initial search using title, keywords, and abstracts resulted in a total of 404 articles. After removal of duplicates from the record, 347 articles remained. An independent analysis and screening of the records against our inclusion and exclusion criteria defined in Methods section resulted in removal of 296 articles, leaving 51 relevant articles. A full-text assessment was conducted on the remaining relevant articles, which resulted in 29 relevant articles that were critically analyzed. The inter-rater agreement was measured using Cohen Kappa test, and disagreements were resolved through discussion. Conclusions: The advances in health technologies and mobile devices have facilitated the implementation of RL algorithms for optimal glycemic regulation in diabetes. However, there exists few articles in the literature focused on the application of these algorithms to the BG regulation problem. Moreover, such algorithms are designed for control tasks as BG adjustment and their use have increased recently in the diabetes research area, therefore we foresee RL algorithms will be used more frequently for BG control in the coming years. Furthermore, in the literature there is a lack of focus on aspects that influence BG level such as meal intakes and physical activity (PA), which should be included in the control problem. Finally, there exists a need to perform clinical validation of the algorithms

Control of Blood Glucose for Type-1 Diabetes by Using Reinforcement Learning with Feedforward Algorithm

Source at https://doi.org/10.1155/2018/4091497.Background: Type-1 diabetes is a condition caused by the lack of insulin hormone, which leads to an excessive increase in blood glucose level. The glucose kinetics process is difficult to control due to its complex and nonlinear nature and with state variables that are difficult to measure. Methods: This paper proposes a method for automatically calculating the basal and bolus insulin doses for patients with type-1 diabetes using reinforcement learning with feedforward controller. The algorithm is designed to keep the blood glucose stable and directly compensate for the external events such as food intake. Its performance was assessed using simulation on a blood glucose model. The usage of the Kalman filter with the controller was demonstrated to estimate unmeasurable state variables. Results: Comparison simulations between the proposed controller with the optimal reinforcement learning and the proportional-integral-derivative controller show that the proposed methodology has the best performance in regulating the fluctuation of the blood glucose. The proposed controller also improved the blood glucose responses and prevented hypoglycemia condition. Simulation of the control system in different uncertain conditions provided insights on how the inaccuracies of carbohydrate counting and meal-time reporting affect the performance of the control system. Conclusion: The proposed controller is an effective tool for reducing postmeal blood glucose rise and for countering the effects of external known events such as meal intake and maintaining blood glucose at a healthy level under uncertainties

Model Fusion to Enhance the Clinical Acceptability of Long-Term Glucose Predictions

This paper presents the Derivatives Combination Predictor (DCP), a novel model fusion algorithm for making long-term glucose predictions for diabetic people. First, using the history of glucose predictions made by several models, the future glucose variation at a given horizon is predicted. Then, by accumulating the past predicted variations starting from a known glucose value, the fused glucose prediction is computed. A new loss function is introduced to make the DCP model learn to react faster to changes in glucose variations. The algorithm has been tested on 10 \textit{in-silico} type-1 diabetic children from the T1DMS software. Three initial predictors have been used: a Gaussian process regressor, a feed-forward neural network and an extreme learning machine model. The DCP and two other fusion algorithms have been evaluated at a prediction horizon of 120 minutes with the root-mean-squared error of the prediction, the root-mean-squared error of the predicted variation, and the continuous glucose-error grid analysis. By making a successful trade-off between prediction accuracy and predicted-variation accuracy, the DCP, alongside with its specifically designed loss function, improves the clinical acceptability of the predictions, and therefore the safety of the model for diabetic people

Study of Short-Term Personalized Glucose Predictive Models on Type-1 Diabetic Children

Research in diabetes, especially when it comes to building data-driven models to forecast future glucose values, is hindered by the sensitive nature of the data. Because researchers do not share the same data between studies, progress is hard to assess. This paper aims at comparing the most promising algorithms in the field, namely Feedforward Neural Networks (FFNN), Long Short-Term Memory (LSTM) Recurrent Neural Networks, Extreme Learning Machines (ELM), Support Vector Regression (SVR) and Gaussian Processes (GP). They are personalized and trained on a population of 10 virtual children from the Type 1 Diabetes Metabolic Simulator software to predict future glucose values at a prediction horizon of 30 minutes. The performances of the models are evaluated using the Root Mean Squared Error (RMSE) and the Continuous Glucose-Error Grid Analysis (CG-EGA). While most of the models end up having low RMSE, the GP model with a Dot-Product kernel (GP-DP), a novel usage in the context of glucose prediction, has the lowest. Despite having good RMSE values, we show that the models do not necessarily exhibit a good clinical acceptability, measured by the CG-EGA. Only the LSTM, SVR and GP-DP models have overall acceptable results, each of them performing best in one of the glycemia regions