Reinforcement learning application in diabetes blood glucose control: A systematic review

Background: Reinforcement learning (RL) is a computational approach to understanding and automating goal-directed learning and decision-making. It is designed for problems which include a learning agent interacting with its environment to achieve a goal. For example, blood glucose (BG) control in diabetes mellitus (DM), where the learning agent and its environment are the controller and the body of the patient respectively. RL algorithms could be used to design a fully closed-loop controller, providing a truly personalized insulin dosage regimen based exclusively on the patient’s own data. Objective: In this review we aim to evaluate state-of-the-art RL approaches to designing BG control algorithms in DM patients, reporting successfully implemented RL algorithms in closed-loop, insulin infusion, decision support and personalized feedback in the context of DM. Methods: An exhaustive literature search was performed using different online databases, analyzing the literature from 1990 to 2019. In a first stage, a set of selection criteria were established in order to select the most relevant papers according to the title, keywords and abstract. Research questions were established and answered in a second stage, using the information extracted from the articles selected during the preliminary selection. Results: The initial search using title, keywords, and abstracts resulted in a total of 404 articles. After removal of duplicates from the record, 347 articles remained. An independent analysis and screening of the records against our inclusion and exclusion criteria defined in Methods section resulted in removal of 296 articles, leaving 51 relevant articles. A full-text assessment was conducted on the remaining relevant articles, which resulted in 29 relevant articles that were critically analyzed. The inter-rater agreement was measured using Cohen Kappa test, and disagreements were resolved through discussion. Conclusions: The advances in health technologies and mobile devices have facilitated the implementation of RL algorithms for optimal glycemic regulation in diabetes. However, there exists few articles in the literature focused on the application of these algorithms to the BG regulation problem. Moreover, such algorithms are designed for control tasks as BG adjustment and their use have increased recently in the diabetes research area, therefore we foresee RL algorithms will be used more frequently for BG control in the coming years. Furthermore, in the literature there is a lack of focus on aspects that influence BG level such as meal intakes and physical activity (PA), which should be included in the control problem. Finally, there exists a need to perform clinical validation of the algorithms

A CASE-BASED REASONING SYSTEM FOR THE DIAGNOSIS OF INDIVIDUAL SENSITIVITY TO STRESS IN PSYCHOPHYSIOLOGY

Abstract Stress is an increasing problem in our present world. Especially negative stress could cause serious health problems if it remains undiagnosed/misdiagnosed and untreated. In stress medicine, clinicians' measure blood pressure, ECG, finger temperature and breathing rate during a number of exercises to diagnose stressrelated disorders. One of the physiological parameters for quantifying stress levels is the finger temperature measurement which helps the clinicians in diagnosis and treatment of stress. However, in practice, it is difficult and tedious for a clinician to understand, interpret and analyze complex, lengthy sequential sensor signals. There are only few experts who are able to diagnose and predict stress-related problems. A system that can help the clinician in diagnosing stress is important, but the large individual variations make it difficult to build such a system. This research work has investigated several artificial Intelligence techniques for the purpose of developing an intelligent, integrated sensor system for establishing diagnosis and treatment plan in the psychophysiological domain. To diagnose individual sensitivity to stress, case-based reasoning is applied as a core technique to facilitate experience reuse by retrieving previous similar cases. Furthermore, fuzzy techniques are also employed and incorporated into the case-based reasoning system to handle vagueness, uncertainty inherently existing in clinicians reasoning process. The validation of the approach is based on close collaboration with experts and measurements from twenty four persons used as reference. 39 time series from these 24 persons have been used to evaluate the approach (in terms of the matching algorithms) and an expert has ranked and estimated the similarity. The result shows that the system reaches a level of performance close to an expert. The proposed system could be used as an expert for a less experienced clinician or as a second option for an experienced clinician to their decision making process in stress diagnosis. Sammanfattning Den ökande stressnivån i vårt samhälle med allt högre krav och högt tempo har ett högt pris. Stressrelaterade problem och sjukdom är en stor samhällskostnad och speciellt om negativ stress förblir oupptäckt, eller ej korrekt identifierad/diagnostiserad och obehandlad under en längre tid kan den få alvarliga hälsoeffekter för individen vilket kan leda till långvarig sjukskrivning. Inom stressmedicinen mäter kliniker blodtryck, EKG, fingertemperatur och andning under olika situationer för att diagnostisera stress. Stressdiagnos baserat fingertemperaturen (FT) är något som en skicklig klinker kan utföra vilket stämmer med forskningen inom klinisk psykofysiologi. Emellertid i praktiken är det mycket svårt, och mödosamt för att en kliniker att i detalj följa och analysera långa serier av mätvärden och det finns endast mycket få experter som är kompetent att diagnostisera och/eller förutsäga stressproblem. Därför är ett system, som kan hjälpa kliniker i diagnostisering av stress, viktig. Men de stora individvariationerna och bristen av precisa diagnosregler gör det svårt att använda ett datorbaserat system. Detta forskningsarbete har tittat på flera tekniker och metoder inom artificiell intelligens för att hitta en väg fram till ett intelligent sensorbaserat system för diagnos och utformning av behandlingsplaner inom stressområdet. För att diagnostisera individuell stress har fallbaserat resonerande visat sig framgångsrikt, en teknik som gör det möjligt att återanvända erfarenhet, förklara beslut, genom att hämta tidigare liknande fingertemperaturprofilerar. Vidare används "fuzzy logic", luddig logik så att systemet kan hantera de inneboende vagheter i domänen. Metoder och algoritmer har utvecklats för detta. Valideringen av ansatsen baseras på nära samarbete med experter och mätningar från tjugofyra användare. Trettionio tidserier från dessa 24 personer har varit basen för utvärderingen av ansatsen, och en erfaren kliniker har klassificerat alla fall och systemet har visat sig producera resultat nära en expert. Det föreslagna systemet kan användas som ett referens för en mindre erfaren kliniker eller som ett "second opinion" för en erfaren kliniker i deras beslutsprocess. Dessutom har finger temperatur visat sig passa bra för användning i hemmet vid träning eller kontroll vilket blir möjligt med ett datorbaserat stressklassificeringssystem på exempelvis en PC med en USB fingertemperaturmätare. vii Acknowledgemen

Perspectives and Best Practices for Artificial Intelligence and Continuously Learning Systems in Healthcare

Goals of this paper Healthcare is often a late adopter when it comes to new techniques and technologies; this works to our advantage in the development of this paper as we relied on lessons learned from CLS in other industries to help guide the content of this paper. Appendix V includes a number of example use cases of AI in Healthcare and other industries. This paper focuses on identifying unique attributes, constraints and potential best practices towards what might represent “good” development for Continuously Learning Systems (CLS) AI systems with applications ranging from pharmaceutical applications for new drug development and research to AI enabled smart medical devices. It should be noted that although the emphasis of this paper is on CLS, some of these issues are common to all AI products in healthcare. Additionally, there are certain topics that should be considered when developing CLS for healthcare, but they are outside of the scope of this paper. These topics will be briefly touched upon, but will not be explored in depth. Some examples include: Human Factors – this is a concern in the development of any product – what are the unique usability challenges that arise when collecting data and presenting the results? Previous efforts at generating automated alerts have often created problems (e.g. alert fatigue.) CyberSecurity and Privacy – holding a massive amount of patient data is an attractive target for hackers, what steps should be taken to protect data from misuse? How does the European Union’s General Data Protection Regulation (GDPR) impact the use of patient data? Legal liability – if a CLS system recommends action that is then reviewed and approved by a doctor, where does the liability lie if the patient is negatively affected? Regulatory considerations – medical devices are subject to regulatory oversight around the world; in fact, if a product is considered a medical device depends on what country you are in. AI provides an interesting challenge to traditional regulatory models. Additionally, some organizations like the FTC regulate non-medical devices. This paper is not intended to be a standard, nor is this paper trying to advocate for one and only one method of developing, verifying, and validating CLS systems – this paper highlights best practices from other industries and suggests adaptation of those processes for healthcare. This paper is also not intended to evaluate existing or developing regulatory, legal, ethical, or social consequences of CLS systems. This is a rapidly evolving subject with many companies, and now some countries, establishing their own AI Principles or Code of Conduct which emphasize legal and ethical considerations including goals and principles of fairness, reliability and safety, transparency around how the results of these learning systems are explained to the people using those systems5 . The intended audience of this paper are Developers, Researchers, Quality Assurance and Validation personnel, Business Managers and Regulators across both Medical Device and Pharmaceutical industries that would like to learn more about CLS best practices, and CLS practitioners wanting to learn more about medical device software development

【研究分野別】シーズ集 [英語版]

[英語版