On-line policy learning and adaptation for real-time personalization of an artificial pancreas

The dynamic complexity of the glucose-insulin metabolism in diabetic patients is the main obstacle towards widespread use of an artificial pancreas. The significant level of subject-specific glycemic variability requires continuously adapting the control policy to successfully face daily changes in patient´s metabolism and lifestyle. In this paper, an on-line selective reinforcement learning algorithm that enables real-time adaptation of a control policy based on ongoing interactions with the patient so as to tailor the artificial pancreas is proposed. Adaptation includes two online procedures: on-line sparsification and parameter updating of the Gaussian process used to approximate the control policy. With the proposed sparsification method, the support data dictionary for on-line learning is modified by checking if in the arriving data stream there exists novel information to be added to the dictionary in order to personalize the policy. Results obtained in silico experiments demonstrate that on-line policy learning is both safe and efficient for maintaining blood glucose variability within the normoglycemic range.Fil: de Paula, Mariano. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ingeniería Olavarria. Departamento de Electromecánica. Grupo INTELYMEC; Argentina. Universidad Nacional del Centro de la Pcia.de Bs.as.. Centro de Investigaciones En Fisica E Ingenieria del Centro de la Provincia de Buenos Aires. - Consejo Nacional de Investigaciones Cientificas y Tecnicas. Centro Cientifico Tecnologico Conicet - Tandil. Centro de Investigaciones En Fisica E Ingenieria del Centro de la Provincia de Buenos Aires. - Provincia de Buenos Aires. Gobernacion. Comision de Invest.cientificas. Centro de Investigaciones En Fisica E Ingenieria del Centro de la Provincia de Buenos Aires; ArgentinaFil: Acosta, Gerardo Gabriel. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ingenieria Olavarria; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Martinez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentin

Receding Horizon Control of Type 1 Diabetes Mellitus by Using Nonlinear Programming

Receding Horizon Controllers are one of the mostly used advanced control solutions in the industry. By utilizing their possibilities we are able to predict the possible future behavior of our system; moreover, we are able to intervene in its operation as well. In this paper we have investigated the possibilities of the design of a Receding Horizon Controller by using Nonlinear Programming. We have applied the developed solution in order to control Type 1 Diabetes Mellitus. The nonlinear optimization task was solved by the Generalized Reduced Gradient method. In order to investigate the performance of our solution two scenarios were examined. In the first scenario, we applied “soft” disturbance—namely, smaller amount of external carbohydrate—in order to be sure that the proposed method operates well and the solution that appeared through optimization is acceptable. In the second scenario, we have used “unfavorable” disturbance signal—a highly oscillating external excitation with cyclic peaks. We have found that the performance of the realized controller was satisfactory and it was able to keep the blood glucose level in the desired healthy range—by considering the restrictions for the usable control action

Control-Oriented Model with Intra-Patient Variations for an Artificial Pancreas

In this work, a low-order model designed for glucose regulation in Type 1 Diabetes Mellitus (T1DM) is obtained from the UVA/Padova metabolic simulator. It captures not only the nonlinear behavior of the glucose-insulin system, but also intrapatient variations related to daily insulin sensitivity (SI) changes. To overcome the large inter-subject variability, the model can also be personalized based on a priori patient information. The structure is amenable for linear parameter varying (LPV) controller design, and represents the dynamics from the subcutaneous insulin input to the subcutaneous glucose output. The efficacy of this model is evaluated in comparison with a previous control-oriented model which in turn is an improvement of previous models. Both models are compared in terms of their open- and closed-loop differences with respect to the UVA/Padova model. The proposed model outperforms previous T1DM controloriented models, which could potentially lead to more robust and reliable controllers for glycemia regulation.Fil: Moscoso Vásquez, Hilda Marcela. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Instituto Tecnológico de Buenos Aires. Departamento de Matemática. Centro de Sistemas y Control; ArgentinaFil: Colmegna, Patricio Hernán. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. University of Virginia; Estados Unidos. Universidad Nacional de Quilmes. Departamento de Ciencia y Tecnología; ArgentinaFil: Rosales, Nicolás. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Instituto de Investigaciones en Electrónica, Control y Procesamiento de Señales. Universidad Nacional de La Plata. Instituto de Investigaciones en Electrónica, Control y Procesamiento de Señales; ArgentinaFil: Garelli, Fabricio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Instituto de Investigaciones en Electrónica, Control y Procesamiento de Señales. Universidad Nacional de La Plata. Instituto de Investigaciones en Electrónica, Control y Procesamiento de Señales; ArgentinaFil: Sanchez Peña, Ricardo Salvador. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Instituto Tecnológico de Buenos Aires. Departamento de Matemática. Centro de Sistemas y Control; Argentin

Artificial Intelligence based multi-agent control system

Le metodologie di Intelligenza Artificiale (AI) si occupano della possibilità di rendere le macchine in grado di compiere azioni intelligenti con lo scopo di aiutare l’essere umano; quindi è possibile affermare che l’Intelligenza Artificiale consente di portare all’interno delle macchine, caratteristiche tipiche considerate come caratteristiche umane. Nello spazio dell’Intelligenza Artificiale ci sono molti compiti che potrebbero essere richiesti alla macchina come la percezione dell’ambiente, la percezione visiva, decisioni complesse. La recente evoluzione in questo campo ha prodotto notevoli scoperte, princi- palmente in sistemi ingegneristici come sistemi multi-agente, sistemi in rete, impianti, sistemi veicolari, sistemi sanitari; infatti una parte dei suddetti sistemi di ingegneria è presente in questa tesi di dottorato. Lo scopo principale di questo lavoro è presentare le mie recenti attività di ricerca nel campo di sistemi complessi che portano le metodologie di intelligenza artifi- ciale ad essere applicati in diversi ambienti, come nelle reti di telecomunicazione, nei sistemi di trasporto e nei sistemi sanitari per la Medicina Personalizzata. Gli approcci progettati e sviluppati nel campo delle reti di telecomunicazione sono presentati nel Capitolo 2, dove un algoritmo di Multi Agent Reinforcement Learning è stato progettato per implementare un approccio model-free al fine di controllare e aumentare il livello di soddisfazione degli utenti; le attività di ricerca nel campo dei sistemi di trasporto sono presentate alla fine del capitolo 2 e nel capitolo 3, in cui i due approcci riguardanti un algoritmo di Reinforcement Learning e un algoritmo di Deep Learning sono stati progettati e sviluppati per far fronte a soluzioni di viaggio personalizzate e all’identificazione automatica dei mezzi trasporto; le ricerche svolte nel campo della Medicina Personalizzata sono state presentate nel Capitolo 4 dove è stato presentato un approccio basato sul controllo Deep Learning e Model Predictive Control per affrontare il problema del controllo dei fattori biologici nei pazienti diabetici.Artificial Intelligence (AI) is a science that deals with the problem of having machines perform intelligent, complex, actions with the aim of helping the human being. It is then possible to assert that Artificial Intelligence permits to bring into machines, typical characteristics and abilities that were once limited to human intervention. In the field of AI there are several tasks that ideally could be delegated to machines, such as environment aware perception, visual perception and complex decisions in the various field. The recent research trends in this field have produced remarkable upgrades mainly on complex engineering systems such as multi-agent systems, networked systems, manufacturing, vehicular and transportation systems, health care; in fact, a portion of the mentioned engineering system is discussed in this PhD thesis, as most of them are typical field of application for traditional control systems. The main purpose if this work is to present my recent research activities in the field of complex systems, bringing artificial intelligent methodologies in different environments such as in telecommunication networks, transportation systems and health care for Personalized Medicine. The designed and developed approaches in the field of telecommunication net- works is presented in Chapter 2, where a multi-agent reinforcement learning algorithm was designed to implement a model-free control approach in order to regulate and improve the level of satisfaction of the users, while the research activities in the field of transportation systems are presented at the end of Chapter 2 and in Chapter 3, where two approaches regarding a Reinforcement Learning algorithm and a Deep Learning algorithm were designed and developed to cope with tailored travels and automatic identification of transportation moralities. Finally, the research activities performed in the field of Personalized Medicine have been presented in Chapter 4 where a Deep Learning and Model Predictive control based approach are presented to address the problem of controlling biological factors in diabetic patients

Diabetes Monitoring System

This is the author accepted manuscript. The final version is available from CRC Press via the DOI in this recordThis chapter reviews the current technologies for monitoring and intervention of diabetes. Especially various blood glucose concentration estimation, online signal monitoring and adaptive control mechanisms are discussed. Recent research has proposed many control engineering approaches for Type 1 diabetes and many algorithms of artificial pancreas have been proposed. This book chapter reviews the current state of the art and industrial standards on diabetes monitoring and control

Insulin Estimation and Prediction A REVIEW OF THE ESTIMATION AND PREDICTION OF SUBCUTANEOUS INSULIN PHARMACOKINETICS IN CLOSED-LOOP GLUCOSE CONTROL

This work was supported by the Spanish Ministry of Economy and Competitiveness (MINECO) through grant DPI2013-46982-C2-1-R and the EU through FEDER funds.Bondía Company, J.; Romero Vivó, S.; Ricarte Benedito, B.; Diez, J. (2018). Insulin Estimation and Prediction A REVIEW OF THE ESTIMATION AND PREDICTION OF SUBCUTANEOUS INSULIN PHARMACOKINETICS IN CLOSED-LOOP GLUCOSE CONTROL. IEEE Control Systems. 38(1):47-66. https://doi.org/10.1109/MCS.2017.2766312S476638

Integrating Meal and Exercise into Personalized Glucoregulation Models: Metabolic Dynamics and Diabetic Athletes

Diabetes affects nearly 26 million Americans, according to the American Diabetes Association, with as many as three million Americans who have Type 1 Diabetes (ADA, 2015). Type 1 Diabetes (T1D) is autoimmune and characterized by little to no insulin production whereas Type 2 Diabetes (T2D) concerns insulin resistance and inability to use produced insulin. Factors contributing to current diabetes management and regulation include exercise type, daily movement activities, and distinct tissue compartment metabolism, each challenging to model in a robust and comprehensive manner. Past models are highly limited in regard to exercise and varying glucose fluctuations dependent on type, intensity, and duration. Modeling could greatly enhance factors that contribute to diabetes management—currently, T1D is managed with a pump and/or injections, informed by constant blood glucose monitoring. This thesis addresses knowledge gaps in the management and etiology of diabetes through development of a novel dynamic mathematical model informing controller design and implementation (artificial pancreas, continuous glucose monitors, and pumps). Diet and meal content on the basis of varying glycemic index and on the effects of activity and exercise, with lifestyle habit implications are a main focus. Emphasis is placed on model personalization with a T1D athlete example. The following model and case study implement specific aims: • 10th order model designed in Matlab with 4 interrelated sub-models to integrate meal diversity, exercise activities, and personalized body composition. o 3-State Glucose Compartmental Model o 2-State Control Mechanisms: Insulin and Glucagon o 2-State Digestion Model o 2-State Exogenous Insulin Control o Skeletal Muscle Model with Mitochondrial State o Nonlinear relations including Hill Functions • A 2 Phase Case Study, IRB approved for a Type 1 athletic 23-year-old female to evaluate and develop the model. Results illustrate effects of meal type (slow vs. fast glycemic index) and exercise/activity based glucose-glycogen consumption on blood plasma glucose predictions and hormonal control action for both non-diabetic and diabetic model versions. Current challenges are addressed with model personalization, providing input flexibility for body mass, muscle ratio, stress, and types of diabetes (T1D, T2D) informing artificial pancreas design and possible sports performance applications

In silico assessment of biomedical products: the conundrum of rare but not so rare events in two case studies

In silico clinical trials, defined as “The use of individualized computer simulation in the development or regulatory evaluation of a medicinal product, medical device, or medical intervention,” have been proposed as a possible strategy to reduce the regulatory costs of innovation and the time to market for biomedical products. We review some of the the literature on this topic, focusing in particular on those applications where the current practice is recognized as inadequate, as for example, the detection of unexpected severe adverse events too rare to be detected in a clinical trial, but still likely enough to be of concern. We then describe with more details two case studies, two successful applications of in silico clinical trial approaches, one relative to the University of Virginia/Padova simulator that the Food and Drug Administration has accepted as possible replacement for animal testing in the preclinical assessment of artificial pancreas technologies, and the second, an investigation of the probability of cardiac lead fracture, where a Bayesian network was used to combine in vivo and in silico observations, suggesting a whole new strategy of in silico-augmented clinical trials, to be used to increase the numerosity where recruitment is impossible, or to explore patients’ phenotypes that are unlikely to appear in the trial cohort, but are still frequent enough to be of concern