Basal-Bolus Advisor for Type 1 Diabetes (T1D) Patients Using Multi-Agent Reinforcement Learning (RL) Methodology

This paper presents a novel multi-agent reinforcement learning (RL) approach for personalized glucose control in individuals with type 1 diabetes (T1D). The method employs a closed-loop system consisting of a blood glucose (BG) metabolic model and a multi-agent soft actor-critic RL model acting as the basal-bolus advisor. Performance evaluation is conducted in three scenarios, comparing the RL agents to conventional therapy. Evaluation metrics include glucose levels (minimum, maximum, and mean), time spent in different BG ranges, and average daily bolus and basal insulin dosages. Results demonstrate that the RL-based basal-bolus advisor significantly improves glucose control, reducing glycemic variability and increasing time spent within the target range (70-180 mg/dL). Hypoglycemia events are effectively prevented, and severe hyperglycemia events are reduced. The RL approach also leads to a statistically significant reduction in average daily basal insulin dosage compared to conventional therapy. These findings highlight the effectiveness of the multi-agent RL approach in achieving better glucose control and mitigating the risk of severe hyperglycemia in individuals with T1D.Comment: 8 pages, 2 figures, 1 Tabl

Model Predictive Control (MPC) of an Artificial Pancreas with Data-Driven Learning of Multi-Step-Ahead Blood Glucose Predictors

We present the design and \textit{in-silico} evaluation of a closed-loop insulin delivery algorithm to treat type 1 diabetes (T1D) consisting in a data-driven multi-step-ahead blood glucose (BG) predictor integrated into a Linear Time-Varying (LTV) Model Predictive Control (MPC) framework. Instead of identifying an open-loop model of the glucoregulatory system from available data, we propose to directly fit the entire BG prediction over a predefined prediction horizon to be used in the MPC, as a nonlinear function of past input-ouput data and an affine function of future insulin control inputs. For the nonlinear part, a Long Short-Term Memory (LSTM) network is proposed, while for the affine component a linear regression model is chosen. To assess benefits and drawbacks when compared to a traditional linear MPC based on an auto-regressive with exogenous (ARX) input model identified from data, we evaluated the proposed LSTM-MPC controller in three simulation scenarios: a nominal case with 3 meals per day, a random meal disturbances case where meals were generated with a recently published meal generator, and a case with 25$\%$ decrease in the insulin sensitivity. Further, in all the scenarios, no feedforward meal bolus was administered. For the more challenging random meal generation scenario, the mean $\pm$ standard deviation percent time in the range 70-180 [mg/dL] was 74.99 $\pm$ 7.09 vs. 54.15 $\pm$ 14.89, the mean $\pm$ standard deviation percent time in the tighter range 70-140 [mg/dL] was 47.78$\pm$8.55 vs. 34.62 $\pm$9.04, while the mean $\pm$ standard deviation percent time in sever hypoglycemia, i.e., $<$ 54 [mg/dl] was 1.00$\pm$3.18 vs. 9.45$\pm$11.71, for our proposed LSTM-MPC controller and the traditional ARX-MPC, respectively. Our approach provided accurate predictions of future glucose concentrations and good closed-loop performances of the overall MPC controller.Comment: 10 pages, 5 Figures, 2 Table

Incorporating the Effect of Behavioral States in Multi-Step Ahead Deep Learning Based Multivariate Predictors for Blood Glucose Forecasting in Type 1 Diabetes

Behavioral factors can affect the blood glucose (BG) levels in people with type 1 diabetes (T1D), therefore, their effects need to be incorporated in blood glucose management for these individuals. Accordingly, in this work, we study the effect of two behavioral states, physical activity (PA) and stress state (SS), on BG fluctuations in individuals with T1D. We provide two methods for quantifying biomarkers related to PA and SS using raw acceleration (ACC) and electrodermal activity (EDA) data collected with a wearable device. We evaluate the impact of PA and SS on BG fluctuation by adding the derived behavior-related biomarkers in two cutting-edge deep learning-based glucose predictive models, a long short-term memory (LSTM) and a convolutional neural network (CNN)-LSTM network, for prediction horizons (PHs) of 30, 60 and 90 min. Through an ablation study, we demonstrate that incorporating the estimated behavior-related biomarkers improves the BG predictive model’s performance obtaining mean absolute error (MAE) 9.13 ± 0.95, 17.75 ± 1.93 and 31.85 ± 2.88 in [mg/dL], root mean square error (RMSE), 12.35 ± 1.06, 24.71 ± 2.31 and 41.64 ± 4.12 in [mg/dL], and coefficient of determination (R2), 95.34 ± 3.34, 78.87 ± 4.35 and 60.11 ± 4.76 in [%], for the LSTM model; and MAE 9.37 ± 0.88, 17.87 ± 1.67 and 29.47 ± 2.13 in [mg/dL], RMSE 12.51 ± 1.40, 24.37 ± 2.49 and 39.52 ± 3.89 in [mg/dL], and R2 94.65 ± 3.90, 78.37 ± 4.11 and 61.12 ± 4.30 in [%], for the CNN-LSTM model, respectively, across all PHs. Additionally, we illustrate the generalizability of the proposed models by performing both population- and patient-wise

Incorporating the Effect of Behavioral States in Multi-Step Ahead Deep Learning Based Multivariate Predictors for Blood Glucose Forecasting in Type 1 Diabetes

Behavioral factors can affect the blood glucose (BG) levels in people with type 1 diabetes (T1D), therefore, their effects need to be incorporated in blood glucose management for these individuals. Accordingly, in this work, we study the effect of two behavioral states, physical activity (PA) and stress state (SS), on BG fluctuations in individuals with T1D. We provide two methods for quantifying biomarkers related to PA and SS using raw acceleration (ACC) and electrodermal activity (EDA) data collected with a wearable device. We evaluate the impact of PA and SS on BG fluctuation by adding the derived behavior-related biomarkers in two cutting-edge deep learning-based glucose predictive models, a long short-term memory (LSTM) and a convolutional neural network (CNN)-LSTM network, for prediction horizons (PHs) of 30, 60 and 90 min. Through an ablation study, we demonstrate that incorporating the estimated behavior-related biomarkers improves the BG predictive model&rsquo;s performance obtaining mean absolute error (MAE) 9.13 &plusmn; 0.95, 17.75 &plusmn; 1.93 and 31.85 &plusmn; 2.88 in [mg/dL], root mean square error (RMSE), 12.35 &plusmn; 1.06, 24.71 &plusmn; 2.31 and 41.64 &plusmn; 4.12 in [mg/dL], and coefficient of determination (R2), 95.34 &plusmn; 3.34, 78.87 &plusmn; 4.35 and 60.11 &plusmn; 4.76 in [%], for the LSTM model; and MAE 9.37 &plusmn; 0.88, 17.87 &plusmn; 1.67 and 29.47 &plusmn; 2.13 in [mg/dL], RMSE 12.51 &plusmn; 1.40, 24.37 &plusmn; 2.49 and 39.52 &plusmn; 3.89 in [mg/dL], and R2 94.65 &plusmn; 3.90, 78.37 &plusmn; 4.11 and 61.12 &plusmn; 4.30 in [%], for the CNN-LSTM model, respectively, across all PHs. Additionally, we illustrate the generalizability of the proposed models by performing both population- and patient-wise