Approximate dynamic programming for anemia management.

The focus of this dissertation work is the formulation and improvement of anemia management process involving trial-and-error. A two-stage method is adopted toward this objective. Given a medical treatment process, a discrete Markov representation is first derived as a formal translation of the treatment process to a control problem under uncertainty. A simulative numerical solution of the control problem is then obtained on-the-fly in the form of a control law maximizing the long-term benefit at each decision stage. Approximate dynamic programming methods are employed in the proposed solution. The motivation underlying this choice is that, in reality, some patient characteristics, which are critical for the sake of treatment, cannot be determined through diagnosis and remain unknown until early stages of treatment, when the patient demonstrates them upon actions by the decision maker. A review of these simulative control tools, which are studied extensively in reinforcement learning theory, is presented. Two approximate dynamic programming tools, namely SARSA and Q -learning, are introduced. Their performance in discovering the optimal individualized drug dosing policy is illustrated on hypothetical patients made up as fuzzy models for simulations. As an addition to these generic reinforcement learning methods, a state abstraction scheme for the considered application domain is also proposed. The control methods of this study, capturing the essentials of a drug delivery problem, constitutes a novel computational framework for model-free medical treatment. Experimental evaluation of the dosing strategies produced by the proposed methods against the standard policy, which is being followed actually by human experts in Kidney Diseases Program, University of Louisville, shows the advantages for use of reinforcement learning in the drug dosing problem in particular and in medical decision making in general

Dynamical Principles of Emotion-Cognition Interaction: Mathematical Images of Mental Disorders

The key contribution of this work is to introduce a mathematical framework to understand self-organized dynamics in the brain that can explain certain aspects of itinerant behavior. Specifically, we introduce a model based upon the coupling of generalized Lotka-Volterra systems. This coupling is based upon competition for common resources. The system can be regarded as a normal or canonical form for any distributed system that shows self-organized dynamics that entail winnerless competition. Crucially, we will show that some of the fundamental instabilities that arise in these coupled systems are remarkably similar to endogenous activity seen in the brain (using EEG and fMRI). Furthermore, by changing a small subset of the system's parameters we can produce bifurcations and metastable sequential dynamics changing, which bear a remarkable similarity to pathological brain states seen in psychiatry. In what follows, we will consider the coupling of two macroscopic modes of brain activity, which, in a purely descriptive fashion, we will label as cognitive and emotional modes. Our aim is to examine the dynamical structures that emerge when coupling these two modes and relate them tentatively to brain activity in normal and non-normal states

Bio-mimetic classification on modern parallel hardware: realizations in NVidia CUDA and OpenMP

Both the brain and modern digital architectures rely on massive parallelismfor efficient solutions to demanding computational tasks, such as pattern recognition. Inthis paper, we implement a parallel classi cation scheme inspired by the insect brain intwo popular parallel computing frameworks, namely as an NVidiarCUDATMimplemen-tation on a TeslaTMdevice and a brute force OpenMPTMparallel implementation on aquad-core CPU. When evaluating the systems on the MNIST data-set of handwrittendigits, we can report that, compared with a standard serial implementation on a singleCPU core, CUDATMimplementations of the bio-inspired classi cation provide a 7-to-11fold speed-up, whereas the OpenMPTMimplementation is 2-to-4 times faster. Our re-sults are a proof of concept that suggests that modern parallel computing architectures andbio-mimetic algorithms are compatible and that the CUDATMsolution on an NVidiarTeslaTMC870 device at the time of writing has a small edge over an OpenMP solutionon a recent quad core processor (3 GHz AMDrPhenomTMII X4 940

2005 Special Issue Individualization of pharmacological anemia management using reinforcement learning *

Effective management of anemia due to renal failure poses many challenges to physicians. Individual response to treatment varies across patient populations and, due to the prolonged character of the therapy, changes over time. In this work, a Reinforcement Learning-based approach is proposed as an alternative method for individualization of drug administration in the treatment of renal anemia. Q-learning, an offpolicy approximate dynamic programming method, is applied to determine the proper dosing strategy in real time. Simulations compare the proposed methodology with the currently used dosing protocol. Presented results illustrate the ability of the proposed method to achieve the therapeutic goal for individuals with different response characteristics and its potential to become an alternative to currently used techniques