A new fuzzy reinforcement learning method for effective chemotherapy

A key challenge for drug dosing schedules is the ability to learn an optimal control policy even when there is a paucity of accurate information about the systems. Artificial intelligence has great potential for shaping a smart control policy for the dosage of drugs for any treatment. Motivated by this issue, in the present research paper a Caputo–Fabrizio fractional-order model of cancer chemotherapy treatment was elaborated and analyzed. A fix-point theorem and an iterative method were implemented to prove the existence and uniqueness of the solutions of the proposed model. Afterward, in order to control cancer through chemotherapy treatment, a fuzzy-reinforcement learning-based control method that uses the State-Action-Reward-State-Action (SARSA) algorithm was proposed. Finally, so as to assess the performance of the proposed control method, the simulations were conducted for young and elderly patients and for ten simulated patients with different parameters. Then, the results of the proposed control method were compared with Watkins’s Q-learning control method for cancer chemotherapy drug dosing. The results of the simulations demonstrate the superiority of the proposed control method in terms of mean squared error, mean variance of the error, and the mean squared of the control action—in other words, in terms of the eradication of tumor cells, keeping normal cells, and the amount of usage of the drug during chemotherapy treatment

Anemia management in end stage renal disease patients undergoing dialysis: a comprehensive approach through machine learning techniques and mathematical modeling

Kidney impairment has global consequences in the organism homeostasis and a disorder like Chronic Kidney Disease (CKD) might eventually exacerbates into End Stage Renal Disease (ESRD) where a complete renal replacement therapy like dialysis is necessary. Dialysis partially reintegrates the blood ltration process; however, even when it is associated to a pharmacological therapy, this is not su fficient to completely replace the renal endocrine role and causes the development of common complications, like CKD secondary anemia (CKD-anemia) The availability of exogenous Erythropoiesis Stimulating Agents (ESA, synthetic molecules with similar structure and same mechanism of action as human erythropoietin) improved the treatment of CKD-anemia although the clinical outcomes are still not completely successful. In particular, for ERSD dialysis patients main di culties in the selection of an optimal therapy dosing derive from the high intra- and inter-individual response variability and the temporal discrepancy between the short ESA permanence in the blood (hours) and the long Red Blood Cells lifespan (months). The aim of this thesis has been to describe the development of the Anemia Control Model (ACM), a tool designed to support physicians in managing anemia for ESRD patines undergoing dialysis. Five main pillars constitute the foundation of this work: - Understanding the medical problem; - Availability of the data needed to derive the models; - Mathematical and Machine Learning modeling; - Development of a product usable at the point of care; - Medical device certi cation and clinical evaluation of the developed product. The understanding of the medical problem is fundamental for two reasons: firstly because the medical problem must be the driver of the product scope and consequently of its design; secondly because a good understanding of the medical problem is of fundamental importance to develop optimized models. In the case of anemia management the drug dosing is an important task where predictive models could support physicians to improve the treatment quality. In particular, considering that hemoglobin is the typical parameter used to measure anemia, our model were tailored to predict hemoglobin response to the two main drugs normally used to correct anemia, that is ESA and Iron. In a mathematical model based on di erential equations, like the one presented in this thesis, the knowledge of the main physiological processes related to anemia is the base to properly design the equations. A machine learning approach in principle can be built with no hypotesis, because it relays in learning from data, nevertheless knowledge of the domain helps to make better use of the available data. The medical problem has been discussed in Chapter 1. The availability of a huge database of very well structured data was basic for the development of models. Quality of the data is another important aspect. Chapter 2 gives the reader an overview of the available data.. The core of the ACM is the capability to predict for each patient the future hemoglobin concentrations as a function of past patient's clinical history and future drug prescription. By means of well performing and personalized predictive model it is possible to simulate how, for each specific c patient, di erent doses would a ffect hemoglobin trends. Mathematical and machine learning models present both advantages and limitations. Chapter 3 describes the mathematical model and analyzes its performances, while Chapter 4 is dedicated to the machine learning models. In our case the machine learning approach resulted more suitable for our scope, because its was well performing on the entire population, more stable and, once trained, very quick in elaborating the prediction. Once the predictive model was obtained, the next step was to wrap it into a service that could be consumed by a third party system (for example an app or a clinical system) where physicians could benefi t from the model prediction capability. To achieve that, firstly an algorithm for the dose selection was developed; secondly, a data structure for the communication with the third party system was defi ned; fi nally, the whole package was wrapped in a web service. These arguments have been discussed in the rst part of Chapter 5. Mistakes in ESA or Iron dosing might have serious consequences on patients' health, for this reason ACM intended use was limited to provide dose suggestions only; physicians must evaluate them and decide whether to accept or reject them. Nevertheless, such a tool could be considered as Medical Device under European Medical Device Directive (MDD); for this reason, to be on the safe side, it was decided to certify the ACM as medical device. A novel approach was developed to perform the risk assessment, the main idea being that ACM might generate risks when a dose suggestion is produced based on a wrong prediction. To assess this risk the model error distribution over the test set was utilized as estimation of the error distribution of the live system. Finally, a clinical evaluation of the ACM in three pilot clinics has been performed before deciding to roll-out the tool in more clinics. These arguments have been discussed in the second part of Chapter 5

Reinforcement learning in large, structured action spaces: A simulation study of decision support for spinal cord injury rehabilitation

Reinforcement learning (RL) has helped improve decision-making in several applications. However, applying traditional RL is challenging in some applications, such as rehabilitation of people with a spinal cord injury (SCI). Among other factors, using RL in this domain is difficult because there are many possible treatments (i.e., large action space) and few patients (i.e., limited training data). Treatments for SCIs have natural groupings, so we propose two approaches to grouping treatments so that an RL agent can learn effectively from limited data. One relies on domain knowledge of SCI rehabilitation and the other learns similarities among treatments using an embedding technique. We then use Fitted Q Iteration to train an agent that learns optimal treatments. Through a simulation study designed to reflect the properties of SCI rehabilitation, we find that both methods can help improve the treatment decisions of physiotherapists, but the approach based on domain knowledge offers better performance

Data-Driven Decision-Making for Medications Management Modalities

abstract: One of the critical issues in the U.S. healthcare sector is attributed to medications management. Mismanagement of medications can not only bring more unfavorable medical outcomes for patients, but also imposes avoidable medical expenditures, which can be partially accounted for the enormous $750 billion that the American healthcare system wastes annually. The lack of efficiency in medical outcomes can be due to several reasons. One of them is the problem of drug intensification: a problem associated with more aggressive management of medications and its negative consequences for patients. To address this and many other challenges in regard to medications mismanagement, I take advantage of data-driven methodologies where a decision-making framework for identifying optimal medications management strategies will be established based on real-world data. This data-driven approach has the advantage of supporting decision-making processes by data analytics, and hence, the decision made can be validated by verifiable data. Thus, compared to merely theoretical methods, my methodology will be more applicable to patients as the ultimate beneficiaries of the healthcare system. Based on this premise, in this dissertation I attempt to analyze and advance three streams of research that are influenced by issues involving the management of medications/treatments for different medical contexts. In particular, I will discuss (1) management of medications/treatment modalities for new-onset of diabetes after solid organ transplantations and (2) epidemic of opioid prescription and abuse.Dissertation/ThesisDoctoral Dissertation Industrial Engineering 201

Artificial intelligence as a tool for research and development in European patent law

Artificial intelligence (“AI”) is increasingly fundamental for research and development (“R&D”). Thanks to its powerful analytical and generative capabilities, AI is arguably changing how we invent. According to several scholars, this finding calls into question the core principles of European patent law—the field of law devoted to protecting inventions. In particular, the AI revolution might have an impact on the notions of “invention”, “inventor”, “inventive step”, and “skilled person”. The present dissertation examines how AI might affect each of those fundamental concepts. It concludes that European patent law is a flexible legal system capable of adapting to technological change, including the advent of AI. First, this work finds that “invention” is a purely objective notion. Inventions consist of technical subject-matter. Whether artificial intelligence had a role in developing the invention is therefore irrelevant as such. Nevertheless, de lege lata, the inventor is necessarily a natural person. There is no room for attributing inventorship to an AI system. In turn, the notion of “inventor” comprises whoever makes an intellectual contribution to the inventive concept. And patent law has always embraced “serendipitous” inventions—those that one stumbles upon by accident. Therefore, at a minimum, the natural person who recognizes an invention developed through AI would qualify as its inventor. Instead, lacking a human inventor, the right to the patent would not arise at all. Besides, the consensus among scholars is that, de facto, AI cannot invent “autonomously” at the current state of technology. The likelihood of an “invention without an inventor” is thus remote. AI is rather a tool for R&D, albeit a potentially sophisticated one. Coming to the “skilled person”, they are the average expert in the field that can rely on the standard tools for routine research and experimentation. Hence, this work finds that if and when AI becomes a “standard” research tool, it should be framed as part of the skilled person. Since AI is an umbrella term for a myriad of different technologies, the assessment of what is truly “standard” for the skilled person – and what would be considered inventive against that figure – demands a precise case-by-case analysis, which takes into account the different AI techniques that exist, the degree of human involvement and skill for using them, and the crucial relevance of data for many AI tools. However, while AI might cause increased complexities and require adaptations – especially to the inventive step assessment – the fundamental principles of European patent law stand the test of time

Faculty Publications and Creative Works 2002

Introduction One of the ways in which we recognize our faculty at the University of New Mexico is through Faculty Publications & Creative Works. An annual publication, it highlights our faculty\u27s scholarly and creative activities and achievements and serves as a compendium of UNM faculty efforts during the 2001 calendar year. Faculty Publications & Creative Works strives to illustrate the depth and breadth of research activities performed throughout our University\u27s laboratories, studios and classrooms. We believe that the communication of individual research is a significant method of sharing concepts and thoughts and ultimately inspiring the birth of new ideas. In support of this, UNM faculty during 2002 produced over 2,278 works, including 1,735 scholarly papers and articles, 64 books, 195 book chapters, 174 reviews, 84 creative works and 26 patented works. We are proud of the accomplishments of our faculty which are in part reflected in this book, which illustrates the diversity of intellectual pursuits in support of research and education at the University of New Mexico. Terry Yates Vice Provost for Researc

Clinical Studies, Big Data, and Artificial Intelligence in Nephrology and Transplantation

In recent years, artificial intelligence has increasingly been playing an essential role in diverse areas in medicine, assisting clinicians in patient management. In nephrology and transplantation, artificial intelligence can be utilized to enhance clinical care, such as through hemodialysis prescriptions and the follow-up of kidney transplant patients. Furthermore, there are rapidly expanding applications and validations of comprehensive, computerized medical records and related databases, including national registries, health insurance, and drug prescriptions. For this Special Issue, we made a call to action to stimulate researchers and clinicians to submit their invaluable works and present, here, a collection of articles covering original clinical research (single- or multi-center), database studies from registries, meta-analyses, and artificial intelligence research in nephrology including acute kidney injury, electrolytes and acid–base, chronic kidney disease, glomerular disease, dialysis, and transplantation that will provide additional knowledge and skills in the field of nephrology and transplantation toward improving patient outcomes