Robust Control of Burst Suppression Amid Physical and Neurological Uncertainty

Burst suppression is a clinical term describing a phenomenon in which the electroencephalogram (EEG) of a sedated patient produces behavior that switches between higher frequency and amplitude bursting to lower frequency and lower amplitude suppression. This phenomenon can be observed during general anesthesia, hypothermia, or in an otherwise induced coma state. In a clinical setting, this phenomenon is typically induced by sedation from a drug such as propofol (2,6-diisopropylphenol). The level of sedation can be quantified by something called the burst suppression ratio (BSR), which is defined as the amount of time that a patient’s EEG is in a suppressed state over the amount of time measured. One can vary this ratio by either increasing or decreasing the propofol infusion rate that the patient is given to bring them to a deeper or lighter state of sedation. By measuring the EEG data, one can form a closed loop feedback system where the EEG data is monitored for signs of burst suppression and the propofol is increased or decreased accordingly. Therefore, it becomes desirable to create models of this closed loop system to simulate the kind of behavior that would be expected from a clinical setting such as the one described. Many methods and experimental paradigms have been developed to address this problem including development of pharmacokinetic (PK) models that describe the dynamics of drug infusion in the body as well as signal processing methods for computing the burst suppression estimation such as the burst suppression probability. Some of these paradigms have been tested in rodent experiments, though human studies remain elusive. In this regard, simulations and detailed physiological modeling and control design can play a key role. This thesis seeks to add on the rich body of work that has been done thus far by incorporating a Schnider PK model with the Wilson-Cowan neural mass model to form a closed loop model which we can use as a basis for more detailed analysis which includes real-time burst suppression estimation as well as uncertainty modeling in both the patient’s physical characteristics (such as weight, height, age and gender) in addition to neurological phenomena such as the recovery and consumption rates of neurons during burst suppression behavior. By creating a conversion from the physiological parameters that describe the PK models to the dimensionless and more abstract parameters which guide the Wilson-Cowan equations, and implementing an actuator and burst suppression ratio estimation algorithm, we have effectively modeled the clinical setting with which the BSR is sought to be controlled. Thus, in this study we wished to show that with PID control, one could control this model at a nominal condition (i.e., the patient and neurological parameters which the gains were designed for) as well as at various uncertainty conditions that include both physical and neurological uncertainty, as described above. Using the Zeigler-Nichols tuning method, we were able to design gains to sufficiently control this system at set points of 0.8, 0.5 and 0.2 BSR over a simulation time of roughly 18 hours in both nominal, patient varied with noise added and with reduced performance when including patient variation and noise as well as neurological uncertainty. This time duration was chosen because it was convenient for the model’s time constants but also because it is representative of the time a patient may be sedated. The BSR ranges were chosen so as to show the closed loop system’s ability to maintain control at multiple levels of sedation. The reduced performance due to neurological uncertainty was due to the BSR estimation algorithm estimating a lower bound that was too high for the system to be controlled at a BSR of 0.2. The minimum BSR the system with added neurological uncertainty could be controlled to was 0.38, which is where the system held at during the portion of the trajectory that a BSR of 0.2 was commanded. During the achievable parts of the envelope, however, the control scheme worked with similar performance to that of the nominal case. This would suggest that an adaptive estimation algorithm needs to be developed to estimate the neurological deviations from the nominal case. Further, this suggests that if variations in the BSR of a patient due to neurological uncertainty is expected, then accurate estimation of these parameters are vital to reaching a robust solution in a real-time system

Enhancing Nervous System Recovery through Neurobiologics, Neural Interface Training, and Neurorehabilitation.

After an initial period of recovery, human neurological injury has long been thought to be static. In order to improve quality of life for those suffering from stroke, spinal cord injury, or traumatic brain injury, researchers have been working to restore the nervous system and reduce neurological deficits through a number of mechanisms. For example, neurobiologists have been identifying and manipulating components of the intra- and extracellular milieu to alter the regenerative potential of neurons, neuro-engineers have been producing brain-machine and neural interfaces that circumvent lesions to restore functionality, and neurorehabilitation experts have been developing new ways to revitalize the nervous system even in chronic disease. While each of these areas holds promise, their individual paths to clinical relevance remain difficult. Nonetheless, these methods are now able to synergistically enhance recovery of native motor function to levels which were previously believed to be impossible. Furthermore, such recovery can even persist after training, and for the first time there is evidence of functional axonal regrowth and rewiring in the central nervous system of animal models. To attain this type of regeneration, rehabilitation paradigms that pair cortically-based intent with activation of affected circuits and positive neurofeedback appear to be required-a phenomenon which raises new and far reaching questions about the underlying relationship between conscious action and neural repair. For this reason, we argue that multi-modal therapy will be necessary to facilitate a truly robust recovery, and that the success of investigational microscopic techniques may depend on their integration into macroscopic frameworks that include task-based neurorehabilitation. We further identify critical components of future neural repair strategies and explore the most updated knowledge, progress, and challenges in the fields of cellular neuronal repair, neural interfacing, and neurorehabilitation, all with the goal of better understanding neurological injury and how to improve recovery

Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learning

Reinforcement Learning (RL) can be used to fit a mapping from patient state to a medication regimen. Prior studies have used deterministic and value-based tabular learning to learn a propofol dose from an observed anesthetic state. Deep RL replaces the table with a deep neural network and has been used to learn medication regimens from registry databases. Here we perform the first application of deep RL to closed-loop control of anesthetic dosing in a simulated environment. We use the cross-entropy method to train a deep neural network to map an observed anesthetic state to a probability of infusing a fixed propofol dosage. During testing, we implement a deterministic policy that transforms the probability of infusion to a continuous infusion rate. The model is trained and tested on simulated pharmacokinetic/pharmacodynamic models with randomized parameters to ensure robustness to patient variability. The deep RL agent significantly outperformed a proportional-integral-derivative controller (median absolute performance error 1.7% +/- 0.6 and 3.4% +/- 1.2). Modeling continuous input variables instead of a table affords more robust pattern recognition and utilizes our prior domain knowledge. Deep RL learned a smooth policy with a natural interpretation to data scientists and anesthesia care providers alike.Comment: International Conference on Artificial Intelligence in Medicine 202