A nonovershooting controller with integral action for multi-input multi-output drug dosing control

In this paper, a nonovershooting tracking controller is proposed for the continuous infusion of multiple drugs that have interactive effects. The proposed controller design method exploits the freedom of eigenstructure assignment pertinent to the design of feedback controllers for multi-input, multi-output (MIMO) systems. For drug dosing, a nonovershooting tracking controller restricts the undesirable side effects of drug overdosing. The proposed tracking controller is based on an estimate of the full state using a hybrid extended Kalman filter (EKF) that is used to reconstruct the system states from the measurable system outputs. An integral control action is included in the controller design to achieve robust tracking in the presence of patient parameter uncertainty. Simulation results and performance analysis of the proposed control strategy are also presented using 20 simulated patients. 2018Qatar National Research FundScopu

Q-Learning Based Closed-Loop Control of Anesthesia Administration by Accounting for Hemodynamic Parameter Variations

Critically ill patients in the intensive care units (ICUs) are often in acutely disturbed state of mind characterized by restlessness, illusions, and nervousness. Such patients, for instance, those who are mechanically ventilated may incur difficulties during treatment procedures such as endotracheal tube intubation/extubation. Apart from critical illness, treatment induced delirium may cause them to dislodge themselves from life-saving equipment and thus hinder cooperative and safe treatment in the ICU. Hence, it is often recommended to moderately sedate such patients for several days to reduce patient anxiety, facilitate sleep, aid treatment and thus endure patient safety. However, most anesthetics affect cardiac and respiratory functions. Hence, it is important to monitor and control the infusion of anesthetics to meet sedation requirements while keeping patient vital parameters within safe limits. The critical task of anesthesia administration also necessitates that drug dosing be optimal, patient specific, and robust. Towards this end, we propose to use a reinforcement learning based approach to develop a closed-loop anesthesia controller that accounts for hemodynamic parameter variations. Main advantage of the proposed approach is that it does not require a model, it involves optimization, and is robust to interpatient variabilities. We formulate the problem of deriving control laws that track a desired trajectory as a sequential decision making problem represented by a finite Markov decision process (MDP) and then use reinforcement learning-based approach to solve the MDPs for goal oriented decision making. Specifically, we use reinforcement learning approaches, such as Q-learning, to develop a closed-loop anesthesia controller using the bispectral index (BIS) as a control variable while concurrently accounting for the mean arterial pressure (MAP). Moreover, the proposed method monitors and controls the infusion of anesthetics by minimizing a weighted combination of the error of the BIS and MAP signals. Account for two variables by considering the error reduces the computational complexity of the reinforcement learning algorithm and consequently the controller processing time. We present simulation results and statistical results using the 30 simulated patients. For our simulations, the pharmacokinetic and the pharmacodynamic values of the simulated patients are chosen randomly from a predefined range. To quantify the performance of the trained agent in the closed-loop anesthesia control, we use the median performance error (MDPE), median absolute performance error (MDAPE), root mean square error (RMSE), and interquartile range (IQ). In order to further investigate the effect of simultaneous regulation of the BIS and MAP parameters on the sedation level (BIS) of a patient, we also conducted three different in silico case studies. In the first case study, a hemodynamic disturbance is considered in which the MAP is altered by d units. This case study considers the effect of other factors such as hemorrhage on MAP as an exogenous disturbance. In the second case study, the MAP is set to a constant value irrespective of propofol infusion, which corresponds to patients that remain intubated in the ICU with post-aortic aneurysm repair or septic patients with respiratory failure. In the third case study, a disturbance due to administration of a synergetic drug such as remifentanil is considered during the administration of propofol. This case study considers the effect of drug interaction on the closed-loop control of hypnotic agent administration.qscienc

Crosstalk between HER2 and PD-1/PD-L1 in Breast Cancer: From Clinical Applications to Mathematical Models.

Breast cancer is one of the major causes of mortality in women worldwide. The most aggressive breast cancer subtypes are human epidermal growth factor receptor-positive (HER2) and triple-negative breast cancers. Therapies targeting HER2 receptors have significantly improved HER2 breast cancer patient outcomes. However, several recent studies have pointed out the deficiency of existing treatment protocols in combatting disease relapse and improving response rates to treatment. Overriding the inherent actions of the immune system to detect and annihilate cancer via the immune checkpoint pathways is one of the important hallmarks of cancer. Thus, restoration of these pathways by various means of immunomodulation has shown beneficial effects in the management of various types of cancers, including breast. We herein review the recent progress in the management of HER2 breast cancer via HER2-targeted therapies, and its association with the programmed death receptor-1 (PD-1)/programmed death ligand-1 (PD-L1) axis. In order to link research in the areas of medicine and mathematics and point out specific opportunities for providing efficient theoretical analysis related to HER2 breast cancer management, we also review mathematical models pertaining to the dynamics of HER2 breast cancer and immune checkpoint inhibitors

Reinforcement learning-based decision support system for COVID-19

Globally, informed decision on the most effective set of restrictions for the containment of COVID-19 has been the subject of intense debates. There is a significant need for a structured dynamic framework to model and evaluate different intervention scenarios and how they perform under different national characteristics and constraints. This work proposes a novel optimal decision support framework capable of incorporating different interventions to minimize the impact of widely spread respiratory infectious pandemics, including the recent COVID-19, by taking into account the pandemic's characteristics, the healthcare system parameters, and the socio-economic aspects of the community. The theoretical framework underpinning this work involves the use of a reinforcement learning-based agent to derive constrained optimal policies for tuning a closed-loop control model of the disease transmission dynamics

Quantification of the growth suppression of HER2+ breast cancer colonies under the effect of trastuzumab and PD-1/PD-L1 inhibitor.

Immune checkpoint blockade (ICB)-based therapy is revolutionizing cancer treatment by fostering successful immune surveillance and effector cell responses against various types of cancers. However, patients with HER2+ cancers are yet to benefit from this therapeutic strategy. Precisely, several questions regarding the right combination of drugs, drug modality, and effective dose recommendations pertaining to the use of ICB-based therapy for HER2+ patients remain unanswered. In this study, we use a mathematical modeling-based approach to quantify the growth inhibition of HER2+ breast cancer (BC) cell colonies (ZR75) when treated with anti-HER2; trastuzumab (TZ) and anti-PD-1/PD-L1 (BMS-202) agents. Our data show that a combination therapy of TZ and BMS-202 can significantly reduce the viability of ZR75 cells and trigger several morphological changes. The combination decreased the cell's invasiveness along with altering several key pathways, such as Akt/mTor and ErbB2 compared to monotherapy. In addition, BMS-202 causes dose-dependent growth inhibition of HER2+ BC cell colonies alone, while this effect is significantly improved when used in combination with TZ. Based on the in-vitro monoculture experiments conducted, we argue that BMS-202 can cause tumor growth suppression not only by mediating immune response but also by interfering with the growth signaling pathways of HER2+BC. Nevertheless, further studies are imperative to substantiate this argument and to uncover the potential crosstalk between PD-1/PD-L1 inhibitors and HER2 growth signaling pathways in breast cancer.This research was funded by grants from Qatar University: QUCG-CENG-19/20-3, QUHI-CMED-19/20-1, and QUCG-CMED-20/21-2

Closed-loop control of anesthesia and mean arterial pressure using reinforcement learning

General anesthesia is required for patients undergoing surgery as well as for some patients in the intensive care units with acute respiratory distress syndrome. How-ever, most anesthetics affect cardiac and respiratory functions. Hence, it is important to monitor and control the infusion of anesthetics to meet sedation requirements while keeping patient vital parameters within safe limits. The critical task of anesthesia administration also necessitates that drug dosing be optimal, patient specific, and robust. In this paper, the concept of reinforcement learning (RL) is used to develop a closed-loop anesthesia controller using the bispectral index (BIS) as a control variable while concurrently accounting for mean arterial pressure (MAP). In particular, the proposed framework uses these two parameters to control propofol infusion rates to regulate the BIS and MAP within a desired range. Specifically, a weighted combination of the error of the BIS and MAP signals is considered in the proposed RL algorithm. This reduces the computational complexity of the RL algorithm and consequently the controller processing time.NPRP grant No. 4-187-2-060 from Qatar National Research Fund (a member of Qatar Foundation).Scopu

Reinforcement learning-based control of drug dosing for cancer chemotherapy treatment

The increasing threat of cancer to human life and the improvement in survival rate of this disease due to effective treatment has promoted research in various related fields. This research has shaped clinical trials and emphasized the necessity to properly schedule cancer chemotherapy to ensure effective and safe treatment. Most of the control methodologies proposed for cancer chemotherapy scheduling treatment are model-based. In this paper, a reinforcement learning (RL)-based, model-free method is proposed for the closed-loop control of cancer chemotherapy drug dosing. Specifically, the Q-learning algorithm is used to develop an optimal controller for cancer chemotherapy drug dosing. Numerical examples are presented using simulated patients to illustrate the performance of the proposed RL-based controller. 1 2017 Elsevier Inc.This publication was made possible by the GSRA grant No. GSRA1-1-1128-13016 from the Qatar National Research Fund (a member of Qatar Foundation). The findings achieved herein are solely the responsibility of the authors.Scopu

Reinforcement learning-based control for combined infusion of sedatives and analgesics

The focus of several clinical trials and research in the area of clinical pharmacology is to fine tune the drug dosing in the phase of additive, antagonistic, and synergistic drug interactive effects. It is important to consider the interactive effects of the drugs to restrict the drug usage to the optimal level required to achieve certain therapeutic effects. Such optimal drug dosing methods will minimize the adverse drug effects and cost associated with the treatment. In this paper, we discuss the use of a reinforcement learning (RL)-based controller to fine tune the drug titration while different drugs with interactive effects are administrated simultaneously. We demonstrate the efficacy of the method by using 25 simulated patients for the simultaneous infusion of a sedative and analgesic drug which has synergistic interactive effect. 1 2017 IEEE.This publication was made possible by the GSRA grant No. GSRA1-1-1128-13016 from the Qatar National Research Fund (a member of Qatar Foundation). The findings achieved herein are solely the responsibility of the authors.Scopu

LEARNING-BASED CONTROL OF CANCER CHEMOTHERAPY TREATMENT

The increasing threat of cancer to human life and the improvement in survival rate of this disease due to effective treatment has promoted research in various related fields. This research has shaped clinical trials and emphasized the necessity to properly schedule cancer chemotherapy to ensure effective and safe treatment. Most of the control methodologies proposed for cancer chemotherapy scheduling treatment are model-based. In this paper, a reinforcement learning (RL)-based, model-free method is proposed for the closed-loop control of cancer chemotherapy drug dosing. Specifically, the Q-learning algorithm is used to develop an optimal controller for cancer chemotherapy drug dosing. Numerical examples are presented using simulated patients to illustrate the performance of the proposed RL-based controller.Scopu

A nonovershooting tracking controller for simultaneous infusion of anesthetics and analgesics

In this paper, a nonovershooting tracking controller is proposed for the continuous infusion of multiple drugs that have interactive effects. The proposed controller design method exploits the freedom of eigenstructure assignment pertinent to the design of feedback controllers for multi-input, multi-output (MIMO) systems. For drug dosing, a nonovershooting tracking controller restricts the undesirable side effects of drug overdosing. The proposed tracking controller is based on an estimate of the full state using a hybrid extended Kalman filter (EKF) that is used to reconstruct the system states from the measurable system outputs. To illustrate the proposed method, we use one of the common anesthetic and analgesic drug combinations (i.e., propofol and remifentanil) which exhibit nonlinear and synergistic drug interaction. An integral control action is included in the controller design to achieve robust tracking in the presence of patient parameter uncertainty. Simulation results and performance analysis of the proposed control strategy are also presented using 20 simulated patients. (C) 2018 Elsevier Ltd. All rights reserved