Reinforcement learning application in diabetes blood glucose control: A systematic review

Background: Reinforcement learning (RL) is a computational approach to understanding and automating goal-directed learning and decision-making. It is designed for problems which include a learning agent interacting with its environment to achieve a goal. For example, blood glucose (BG) control in diabetes mellitus (DM), where the learning agent and its environment are the controller and the body of the patient respectively. RL algorithms could be used to design a fully closed-loop controller, providing a truly personalized insulin dosage regimen based exclusively on the patient’s own data. Objective: In this review we aim to evaluate state-of-the-art RL approaches to designing BG control algorithms in DM patients, reporting successfully implemented RL algorithms in closed-loop, insulin infusion, decision support and personalized feedback in the context of DM. Methods: An exhaustive literature search was performed using different online databases, analyzing the literature from 1990 to 2019. In a first stage, a set of selection criteria were established in order to select the most relevant papers according to the title, keywords and abstract. Research questions were established and answered in a second stage, using the information extracted from the articles selected during the preliminary selection. Results: The initial search using title, keywords, and abstracts resulted in a total of 404 articles. After removal of duplicates from the record, 347 articles remained. An independent analysis and screening of the records against our inclusion and exclusion criteria defined in Methods section resulted in removal of 296 articles, leaving 51 relevant articles. A full-text assessment was conducted on the remaining relevant articles, which resulted in 29 relevant articles that were critically analyzed. The inter-rater agreement was measured using Cohen Kappa test, and disagreements were resolved through discussion. Conclusions: The advances in health technologies and mobile devices have facilitated the implementation of RL algorithms for optimal glycemic regulation in diabetes. However, there exists few articles in the literature focused on the application of these algorithms to the BG regulation problem. Moreover, such algorithms are designed for control tasks as BG adjustment and their use have increased recently in the diabetes research area, therefore we foresee RL algorithms will be used more frequently for BG control in the coming years. Furthermore, in the literature there is a lack of focus on aspects that influence BG level such as meal intakes and physical activity (PA), which should be included in the control problem. Finally, there exists a need to perform clinical validation of the algorithms

Probability and Quantum Mechanics: A Christian Theistic Interpretation

It has been claimed by some in creationist circles that modern physics is derived from a priori atheistic reductionistic presuppositions. On the contrary, it is claimed in this paper that it is possible to construct a Christian theistic interpretation of the experimental facts which is consistent with the Biblical doctrines of man, creation, revelation, and God. This reconciliation is derived by the correct method which subordinates scientific interpretation to the authority of special revelation. It is also demonstrated that some of the presuppositions and arguments of creationists against theories such as quantum mechanics and relativity actually exhibit anti-theistic philosophies (Thomism and Arminianism) or fundamental misunderstandings of these theories. A Christian theistic view of the phenomena and theories of moderr physics is presented

Dynamic modelling of blood glucose concentration in people with type 1 diabetes

The behaviour of blood glucose concentration (BGC) in free living conditions is not well understood in people with type 1 diabetes; in particular, the effect of different types of activity experienced in everyday life has not been fully investigated. Better understanding of the effect of major disturbances to BGC can improve treatment regimes and delay or prevent complications associated with diabetes. The current research investigates approaches to modelling BGC, based on blood glucose, physical activity, food and insulin data collected from a Diabetes UK study. Exploratory analysis of the study data found that BGC is non-stationary and exhibits strong autocorrelation, which varies among and within individuals. Analysis of BGC in the frequency domain also highlights indistinct low-frequency periodicities. However, BGC measurements alone are not enough to predict BGC over several hours using autoregressive models. Dynamic linear models are used to model BGC empirically using inputs from measured physical activity, and estimates of glucose and insulin absorption after food intake and injections, respectively, derived from physiological models in the literature. Dynamic linear models are used for parameter learning and predicting BGC over several hours: the models show some capability for predicting BGC for up to one hour, in particular highlighting periods of low and high BGC, but parameter estimates do not comply with established physiological knowledge. A new semi-empirical compartmental model is developed to impose a structure that incorporates well established physiology. A set of differential equations are converted into a probabilistic Bayesian framework, suitable for simultaneous, model-wide parameter estimation and prediction. A simulation study is conducted to determine the feasibility of using Markov chain Monte Carlo methods as a means for parameter estimation, and test performance in the predictive space. The methods show an ability to estimate a subset of the parameters simultaneously with good coverage, robustness to parameter misspecification, and insensitivity to specification of prior distributions. The current research represents a new paradigm for analysing mathematical models of BGC, and highlights important practical and theoretical issues not previously addressed in the quest for an artificial pancreas as treatment for type 1 diabetes

Estudo de retalhos venosos arterializados no modelo experimental de rato wistar e no cadáver humano

RESUMO:Introdução: Os retalhos de perfusão não convencionais (UPFs) são opções reconstrutivas caracterizadas por serem perfundidas exclusivamente por veias. Nos UPFs, pelo menos uma das veias aferentes do retalho é anastomosada a um vaso de alimentação. Normalmente, este vaso de alimentação é uma artéria, e o UPF é chamado de retalho venoso arterializado (AVF). Se o vaso de alimentação for uma veia, o UPF é chamado de retalho venoso (VF). O fluxo de sangue é assegurado na maioria dos casos pela continuidade de uma ou mais veias do UPF com veias vizinhas. Embora os UPFs apresentem várias vantagens potenciais em relação aos retalhos de perfusão convencionais (CFs), raramente são mencionados na literatura, devido às altas taxas de necrose relatadas, particularmente na presença de infeção, e devido a uma má compreensão de seus mecanismos fisiológicos subjacentes. Métodos: Realizámos revisões sistemáticas e metanálises sobre o uso clínico e experimental dos UPFs. Seguidamente, estudamos detalhadamente a anatomia vascular do aspecto ventrolateral do abdômen do rato. Com esse conhecimento, melhoramos o modelo de um retalho convencional colhido na região epigástrica. Posteriormente, desenvolvemos um modelo otimizado de AVF no abdómen do rato. Avaliamos o efeito de transfectar este modelo otimizado com genes de beta defensina humana (BD-2 e BD-3) para aumentar a sobrevivência do retalho na presença de infecção por Pseudomonas aeruginosa e de um corpo estranho. Além disso, comparamos a eficácia dos retalhos neurovenosos arterializados (ANVFs) com outros condutos nervosos com o intuito de reconstruir um hiato de 10 mm no nervo mediano do rato num ambiente de isquémia local. Seguidamente, realizamos estudos cadavéricos para avaliar os aspetos pertinentes da anatomia e histologia das regiões anatômicas comumente usadas para colher UPFs. Finalmente, usamos algumas das informações coletadas para tratar um adolescente com um defeito do pedículo vásculo-nervoso ulnar ao nível do antebraço. Resultados: Estimámos uma taxa de sobrevivência global de UPFs de 89,5% no contexto clínico e de 90,8% em condições experimentais. Clinicamente, houve uma correlação positiva entre a taxa de infeção pós-operatória e a necessidade de um novo retalho (coeficiente de Pearson 0,405; p = 0,001). O fornecimento de sangue ao tegumento abdominal do rato era sobretudo dependente dos vasos axiais, contrastando com o que acontece no Homem. Válvulas venosas foram claramente observadas nesta região. O retalho convencional epigástrico livre e o AVF otimizada homólogo apresentaram taxas de sobrevivência de quase 100% e 76,86 ± 13,67%, respectivamente. Transfectando-se o modelo de AVF com BD-2 e BD-3, observou-se aumento da sobrevivência do retalho e diminuição da formação de biofilmes. Os ANVFs produziram uma recuperação mais completa e mais rápida do que os enxertos nervosos, para a maioria dos parâmetros utilizados para avaliar a regeneração nervosa. Estudos anatómicos e histológicos revelaram que as veias subcutâneas de maiores dimensões encontravam-se envolvidas por desdobramentos da fáscia superficial. Além disso, as veias encontravam-se em profundidades diferentes, estando as maiores colocadas profundamente e as mais pequenas localizadas mais superficialmente. Finalmente, observou-se que os nervos cutâneos superficiais, rotineiramente utilizados como enxertos de nervos autólogos, estavam mais próximos das veias superficiais do que das artérias e respetivas veias comitantes com calibre significativo. Os UPFs podem ser adaptados a defeitos específicos, incluindo pele, tecido celular subcutâneo, tendões, nervos, fáscia muscular e / ou osso em combinações variáveis. O uso de um ANVF no referido adolescente permitiu a reconstrução dos defeitos arterial e nervoso com sucesso. Conclusão: Apesar de muitas questões continuarem por responder em relação à fisiologia, otimização e indicações dos UPFs, parece haver evidência suficiente para apoiar seu uso no âmbito da reconstrução tegumentar e nervosa.ABSTRACT: Introduction: Unconventional perfusion flaps (UPFs) are reconstructive options characterized by being perfused exclusively by veins. In UPFs at least one of the afferent veins of the flap is anastomosed to a feeding vessel. Usually, this feeding vessel is an artery, and the UPF is called an arterialized venous flap (AVF). If the feeding vessel is a vein, the UPF is called a venous flap (VF). The efflux of blood is ensured in most cases by the continuity of one or more of the UPF’s veins with neighboring veins. Although UPFs present several potential advantages relatively to conventional perfusion flaps, they have rarely been mentioned in the clinical literature, due to reported high necrosis rates, particularly in the presence of infection, and due to a poor understanding of their underlying physiologic mechanisms. Methods: We performed systematic reviews and meta-analyses on the clinical and experimental used of UPFs. Followingly, we studied in detail the vascular anatomy of the ventrolateral aspect of the rat’s abdomen. Using this knowledge, we improved a model of a conventional flap (CF) harvested from the epigastric region of the fat. Subsequently, we developed an optimized a model of AVF in the abdomen of the rat. We, then, evaluated the effect of transfecting the optimized model with human beta defensin genes (BD-2 and BD-3) to increase flap survival in the presence of Pseudomonas aeruginosa infection and of a foreign body. Moreover, we compared the efficacy of arterialized neurovenous flaps (ANVFs) with other nerve conduits to reconstruct a 10-mm-long median nerve gap in an ischemic environment in a rat model. Followingly, we performed cadaveric studies to assess pertinent aspects of the anatomy and histology of anatomical regions commonly used to harvest UPFs. Finally, we used some of the information gathered to treat a teenager with an ulnar artery and nerve composite defect at the forearm level. Results: We estimated an overall survival rate of UPFs of 89.5% in the clinical context and of 90.8% in the experimetal setting. Clinically, there was a positive correlation between the rate of postoperative infection and the need of a new flap (Pearson coefficient 0.405; p=0.001). Blood supply to the abdominal integument of the rat was more dependent on axial vessels, comparatively to humans. Venous valves were clearly observed in this region. The free epigastric CF and the homologous optimized AVF presented survival rates of nearly 100%, and 76.86 ± 13.67%, respectively. Transfecting the AVF model with BD-2 and BD-3 increased flap survival, and decreased biofilm formation. ANVFs produced more complete and faster recovery than nerve grafts, for most of the parameters used to assess nerve regeneration. Anatomical and histological studies revealed that large subcutaneous veins were surrounded by doublings of the superficial fascia. Moreover, veins were placed at different depths, with the largest ones being deeply placed and the smallest more superficially placed. Finally, it was noted that superficial cutaneous nerves, routinely used as autologous nerve grafts, were closer to sizeable superficial veins than to arteries and respective comitante veins of significant caliber. UPFs could be tailored to specific defects by including either skin, subcutaneous tissue, tendons, nerves, muscle fascia and/or bone in variable combinations. The used of an ANVF in a teenager allowed the successful reconstruction of both the arterial and nerve defects. Conclusion: Although many question remain to be answered relatively to UPFs physiology, optimization, and indications, there seems to be enough evidence to support their use in the realm of integumentary and nerve reconstruction.RÉSUMÉ: Introduction: Les lambeaux de perfusion non conventionnels (UPF) sont des options de reconstructives caractérisées par une perfusion exclusive pour le veines. Dans les UPF, il ya au moins une des veines afférentes du Lambeau qui est anastomosée à un vase d'alimentation. Habituellement, ce vaisseau d'alimentation est une artère, et l'UPF s'appelle un lambeau veineux arterialisé (AVF). Si le vaisseau est une veine, l'UPF s'appelle un lambeau veineux (VF). L'effusion de sang est assurée dans la plupart des cas par la continuité d'une ou plusieurs veines de l'UPF avec des veines voisines. Bien que les UPF présentent plusieurs avantages potentiels par rapport aux lambeaux de perfusion conventionnels, ils ont rarement été mentionnés dans la littérature clinique, en raison des taux élevés de nécrose reportés, en particulier en présence d'une infection, et en raison d'une mauvaise compréhension de leurs mécanismes physiologiques. Méthodes: Nous avons procédé à deux analyses systématiques et méta-analyses sur l'utilisation clinique et expérimentale des UPFs. Nous avons étudié en détail l'anatomie vasculaire de l'aspect ventral de l'abdomen du rat. À l'aide de cette connaissance, nous avons amélioré un modèle de lambeau conventionnel (CF) récolté dans la région épigastrique de l’abdomen. Par la suite, nous avons développé un modèle optimisé d'AVF dans l'abdomen du rat. Nous avons ensuite évalué l'effet de la transfection du modèle optimisé avec les gènes de la beta-defensine humaine (BD-2 et BD-3) pour augmenter la survie des lambeaux en présence d'une infection pour Pseudomonas aeruginosa et d'un corps étranger. Nous avons aussi comparé l'efficacité des lambeaux neurovenous arterialisés (ANVF) avec d'autres conduits nerveux pour reconstruire un hiatus nerveux de le nerf médian de 10 mm de longueur dans un environnement ischémique dans le modèle de le rat Wistar. Par la suite, nous avons effectué des études cadavériques pour évaluer les aspects pertinents de l'anatomie et de l'histologie des régions anatomiques couramment utilisées pour récolter des UPFs. Enfin, nous avons utilisé une partie des informations recueillies pour traiter un adolescent souffrant d'un défaut composite de l’artère ulnaire e de le nerf ulnaire au niveau de l'avant-bras. Résultats: Nous avons estimé un taux global de survie des UPF de 89,5% dans le contexte clinique et de 90,8% dans le contexte expérimental. Cliniquement, il y avait une corrélation positive entre le taux d'infection postopératoire et la besoin d'un nouveau lambeau (coefficient de Pearson 0,405; p = 0,001). L'approvisionnement en sang du tégument abdominal du rat dépendait davantage des vaisseaux axiaux, comparativement aux humains. Les valves veineuses ont été clairement observées dans cette région. Le lambeau épigastrique conventionnel livre et l'AVF homologue optimisée ont présenté des taux de survie de près de 100% et 76,86 ± 13,67%, respectivement. Transféction de le modèle AVF optimisée avec BD-2 et BD-3 augmenté la survie du lambeau et la formation de biofilm a été diminuée. Les ANVF ont produit une récupération plus complète et plus rapide que les greffes nerveuses, pour la plupart des paramètres utilisés pour évaluer la régénération nerveuse. Des études anatomiques et histologiques ont révélé que les grandes veines sous-cutanées étaient entourées de doublures du fascia superficiel. De plus, les veines ont été placées à différentes profondeurs, les plus grandes étant profondément placées et les plus petites placées plus superficiellement. Enfin, il a été noté que les nerfs cutanés superficiels, habituellement utilisés comme greffes autologues, étaient plus proches des veines superficielles importantes que des artères et des veines comitantes respectives de calibre significatif. Les UPF pourraient être adaptés à des défauts spécifiques en incluant la peau, le tissu sous-cutané, les tendons, les nerfs, le fascia musculaire et / ou l'os dans des combinaisons variables. L'utilisation d'un ANVF chez un adolescent a permis la reconstruction réussie des défauts artériels et nerveux. Conclusion: Bien que beaucoup de questions restent à répondre par rapport à la physiologie, à l'optimisation et aux indications de l'UPF, il semble y avoir suffisamment des preuves scientifiques pour favoriser leur utilisation dans le domaine de la reconstruction de l’integument et des les nerfs

Contributions to modelling and control for improved hypoglycaemia and variability mitigation by dual-hormone artificial pancreas systems

[ES] Las personas con diabetes tipo 1 carecen de la capacidad de secretar insulina y, por lo tanto, necesitan regular su glucosa en sangre con la administración de insulina exógena. El páncreas artificial se presenta como la solución tecnológica ideal para alcanzar los objetivos terapéuticos de la normoglucemia, liberando al paciente de la carga actual de autocontrol y manejo. Sin embargo, el riesgo de hipoglucemia y la variabilidad glucémica siguen siendo factores limitantes en los algoritmos de control actuales integrados en el páncreas artificial. El propósito de la presente tesis es profundizar en el conocimiento de la hipoglucemia y avanzar los algoritmos de control del páncreas artificial para minimizar la incidencia de hipoglucemia y reducir la variabilidad glucémica. Después de proporcionar una visión general del estado del arte del control de la glucosa y el páncreas artificial, esta tesis aborda temas relacionados con el modelado y el control, con las siguientes contribuciones: Se presenta una extensión del modelo de Bergman Minimal que tiene en cuenta la respuesta contrarreguladora a la hipoglucemia. Este modelo explica la relación entre los diversos cambios fisiológicos producidos durante la hipoglucemia, con la adrenalina y los ácidos grasos libres como actores principales. Como resultado, se obtiene una mejor comprensión de la hipoglucemia, lo que permite explicar una auto-potenciación paradójica de la hipoglucemia como se modela a través de enfoques funcionales en el ampliamente utilizado simulador de diabetes tipo 1 UVA-Padova, que se utilizará en esta tesis para la validación in silico de los controladores desarrollados. Se realiza una evaluación de las métricas de variabilidad de la glucosa y los índices de calidad de control. La evaluación de la variabilidad glucémica en el desempeño de los controladores es necesaria; pero todavía no hay un conjunto de métricas de variabilidad glucémica que sea considerado como el "gold estándar". Por tanto, se lleva a cabo un análisis de las métricas de variabilidad disponibles en la literatura para definir un conjunto de indicadores recomendables. Debido a las limitaciones de los sistemas de páncreas artificiales unihormonales para mitigar la hipoglucemia en escenarios difíciles como el ejercicio, esta tesis se centra en el desarrollo de nuevos algoritmos de control bihormonales, con infusión simultanea de insulina y glucagón. Se propone un controlador coordinado bihormonal con estructuras de control paralelas como un algoritmo de control factible para la mitigación de la hipoglucemia y la reducción de la variabilidad glucémica, demostrando un rendimiento superior al de las estructuras de control utilizadas actualmente con lazos de control independientes de insulina y glucagón. Los controladores están diseñados y evaluados in silico en escenarios desafiantes y su rendimiento se evalúa principalmente con el conjunto de métricas definidas previamente como las recomendables.[CA] Les persones amb diabetis tipus 1 no tenen la capacitat de secretar insulina secreta i per tant, necessiten regular la seva glucosa en sang amb l'administració d'insulina exògena. El Pàncrees Artificial es presenta com la solució tecnològica ideal per assolir els objectius terapèutics de la normoglucèmia, alliberant al pacient de la càrrega actual d'autocontrol. No obstant, el risc d'hipoglucèmia i l'alta variabilitat glucèmica continuen sent un factor limitant en els algoritmes de control actuals integrats en el Pàncrees Artificials. El propòsit de la present tesi és aprofundir en el coneixement de la hipoglucèmia i millorar els algoritmes de control per corregir amb antelació la dosi excessiva d'insulina, minimitzant la incidència d'hipoglucèmia i reduint la variabilitat glucèmica. Després de donar una visió general de l'estat de l'art del control de la glucosa i el pàncrees artificial, aquesta tesi aborda aspectes de modelització i control, amb les següents contribucions: Es presenta una extensió del model Minimal de Bergman amb la contrarregulació. Aquest model explica la relació entre els diversos canvis siològics produïts durant la hipoglucèmia. Així, permet comprendre millor la hipoglucèmia i comparar els resultats amb els proporcionats per l'enfocament funcional del simulador de diabetis tipus 1 més utilitzat a la comunitat científica. Es realitza una avaluació de les mètriques de variabilitat glucèmica i dels índexs de qualitat de control. Es necessària l'avaluació de la variabilitat glucèmica en el rendiment dels controladors; però encara no hi ha un conjunt de mètriques considerades com les "gold standard". Per tant, es realitza una anàlisi de les mètriques de variabilitat disponibles a la literatura per definir un conjunt d'indicadors recomanables. Es proposa un controlador bi-hormonal coordinat amb estructures de control paral.leles com un algoritme de control viable per a la mitigació d'hipoglucèmia i la reducció de la variabilitat glucèmica. Els controladors estan dissenyats i avaluats in-silico en escenaris desafiadors i el seu rendiment es valora principalment amb el conjunt de mètriques definides prèviament com les mètriques recomanables.[EN] People with Type 1 Diabetes lack the ability to secrete insulin and therefore need to regulate their blood glucose with exogenous insulin delivery. The Artificial Pancreas is presented as the ideal technological solution to reach the therapeutic goals of normoglycaemia, freeing the patient from the current burden of self-control and management. Nevertheless, the risk of hypoglycaemia and the high glycaemic variability are still a limiting factors in the current control algorithms integrated in the Artificial Pancreas. The purpose of the present thesis is to delve into knowledge of hypoglycaemia and to advance in the artificial pancreas control algorithms in order to minimise hypoglycaemia incidence and reduce glycaemic variability. After providing an overview of the state of the art in the eld of glucose control and articial pancreas, this thesis addresses issues on modelling and control, with the following contributions: An extension of the Bergman Minimal model accounting for counterregulatory response to hypoglycaemia is presented. This model explains the relationship between the several physiological changes produced during hypoglycaemia, with adrenaline and free fatty acids as main players. As a result, a better understanding of hypoglycaemia is gained, allowing to explain a paradoxical auto-potentiation of hypoglycaemia as modeled through functional approaches in the widespread used UVA-Padova Type 1 Diabetes simulator, which will be used in this thesis for in silico validation of the developed controllers. An assessment of glucose variability metrics and control quality indices is carried out. The evaluation of the glycaemic variability on the controllers performance is necessary; but there is not a gold standard variability metrics yet. Therefore, an analysis of the variability metrics available in literature is conducted in order to define a recommendable set of indicators. Due to the limitations of single-hormone artificial pancreas systems in mitigating hypoglycaemia in challenging scenarios such as exercise, this thesis focuses on the developement of new dual-hormone control algorithms, with concomitant infusion of insulin and glucagon. A coordinated dual-hormone controller with parallel control structures is proposed as a feasible control algorithm for hypoglycaemia mitigation and glycaemic variability reduction, demonstrating superior performance as currently used control structures with independent insulin and glucagon control loops. The controllers are designed and evaluated in-silico under challenging scenarios and their performance are assessed mainly with the set of metrics defined previously as the recommendable ones.Moscardó García, V. (2019). Contributions to modelling and control for improved hypoglycaemia and variability mitigation by dual-hormone artificial pancreas systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/120456TESI

Study of NASA Aeronautics Safety Research Programs 1980-1989

Educational Administratio