Aprendizaje por refuerzo en espacios continuos: algoritmos y aplicación al tratamiento de la anemia renal

El aprendizaje por refuerzo es un paradigma de aprendizaje automático orientado a la resolución de problemas de decisión secuenciales. Este tipo de problemas aparece en aplicaciones pertenecientes a campos tan diversos como control automático, medicina, investigación operativa o economía. Los algoritmos clásicos de aprendizaje por refuerzo están fundamentados en la teoría matemática de la programación dinámica, donde se asume que el espacio de estados es discreto y se compone de un número manejable de estados. Desafortunadamente, en la mayoría de aplicaciones de interés práctico el espacio de estados es continuo, por lo que los algoritmos clásicos dejan de ser útiles. Para poder aplicar el aprendizaje por refuerzo en espacios continuos se requiere, por una parte, generalizar el comportamiento aprendido a partir de un conjunto limitado de experiencias a casos que no se hayan experimentado previamente y, por otra parte, representar las políticas de forma compacta. Ambos requisitos han sido ampliamente estudiados en el campo del aprendizaje supervisado, donde a menudo se necesita aproximar una función continua a partir de un conjunto de puntos discretos. La combinación de algoritmos de aprendizaje por refuerzo con técnicas de aproximación de funciones es actualmente un área de investigación activa. A pesar de los avances logrados en los últimos años, todavía hay aspectos que limitan la capacidad del aprendizaje por refuerzo en problemas complejos. Entre ellos destacan la escasa capacidad de escalabilidad a espacios definidos por un número elevado de dimensiones y la elevada cantidad de datos necesarios para aprender políticas útiles. En esta tesis doctoral se proponen algoritmos de aprendizaje por refuerzo enfocados a mejorar estos dos aspectos. Los resultados obtenidos en diversos experimentos demuestran que los algoritmos propuestos suponen un avance hacia métodos de aprendizaje por refuerzo más prácticos y efectivos en problemas complejos. Además de las aportaciones teóricas se ha desarrollado un sistema basado en aprendizaje por refuerzo para la optimización del tratamiento de la anemia asociada a la enfermedad renal crónica.Reinforcement learning is a machine learning paradigm aimed at solving sequential decision making problems. This kind of problems is commonly encountered in areas such as automatic control, medicine, operative research or economy. Classical reinforcement learning algorithms rely on the mathematical theory of dynamic programming, where it is assumed that the state space is discrete and it is composed by a reduced number of states. Unfortunately, in most of the practical applications, the classical algorithms are not useful because the state space is continuous. In order to apply reinforcement learning in continuous spaces is necessary, on the one hand, to generalize the behaviour learned from a limited set of experiences to previously unseen cases and, on the other hand, to represent the policies in a compact way. Both requirements have been widely studied in the supervised learning field, where it is common to approximate a continuous function from a set of discrete points. The combination of reinforcement learning algorithms with function approximation is currently an active field of research. In spite of significant advances over the last years, there are still many issues that limit the ability of reinforcement learning methods in complex domains. Prominent among them are the poor scalability and the high amount of data required to learn useful policies. This thesis proposes several reinforcement learning algorithms intended for improving those two issues. The results obtained in the experiments show that the proposed algorithms represent an important step forward toward more practical and effective methods in complex domains. In addition to the theoretical contributions, this thesis also shows a system based on reinforcement learning aimed to optimize the treatment of patients with secondary anemia to chronic kidney disease

Supervised Quantum Learning without Measurements

We propose a quantum machine learning algorithm for efficiently solving a class of problems encoded in quantum controlled unitary operations. The central physical mechanism of the protocol is the iteration of a quantum time-delayed equation that introduces feedback in the dynamics and eliminates the necessity of intermediate measurements. The performance of the quantum algorithm is analyzed by comparing the results obtained in numerical simulations with the outcome of classical machine learning methods for the same problem. The use of time-delayed equations enhances the toolbox of the field of quantum machine learning, which may enable unprecedented applications in quantum technologies

Sparse Manifold Clustering and Embedding to discriminategene expression profiles of glioblastoma and meningioma tumors

Sparse Manifold Clustering and Embedding (SMCE) algorithm has been recently proposed for simultaneous clustering and dimensionality reduction of data on nonlinear manifolds using sparse representation techniques. In this work, SMCE algorithm is applied to the differential discrimination of Glioblastoma and Meningioma Tumors by means of their Gene Expression Profiles. Our purpose was to evaluate the robustness of this nonlinear manifold to classify gene expression profiles, characterized by the high-dimensionality of their representations and the low discrimination power of most of the genes. For this objective, we used SMCE to reduce the dimensionality of a preprocessed dataset of 35 single-labeling cDNA microarrays with 11500 original clones. Afterwards, supervised and unsupervised methodologies were applied to obtain the classification model: the former was based on linear discriminant analysis, the later on clustering using the SMCE embedding data. The results obtained using both approaches showed that all (100%) the samples could be correctly classified and the results of all repetitions but one formed a compatible cluster of predictive labels. Finally, the embedding dimensionality of the dataset extracted by SMCE revealed large discrimination margins between both classes. (c) 2013 Elsevier Ltd. All rights reserved.This work was supported by the University of Valencia through project UV-INV-AE11-41271.García Gómez, JM.; Gómez-Sanchis, J.; Escandell-Montero, P.; Fuster García, E.; Soria-Olivas, E. (2013). Sparse Manifold Clustering and Embedding to discriminategene expression profiles of glioblastoma and meningioma tumors. Computers in Biology and Medicine. 43(11):1863-1869. doi:10.1016/j.compbiomed.2013.08.025S18631869431