Competitive function approximation for reinforcement learning

The application of reinforcement learning to problems with continuous domains requires representing the value function by means of function approximation. We identify two aspects of reinforcement learning that make the function approximation process hard: non-stationarity of the target function and biased sampling. Non-stationarity is the result of the bootstrapping nature of dynamic programming where the value function is estimated using its current approximation. Biased sampling occurs when some regions of the state space are visited too often, causing a reiterated updating with similar values which fade out the occasional updates of infrequently sampled regions. We propose a competitive approach for function approximation where many different local approximators are available at a given input and the one with expectedly best approximation is selected by means of a relevance function. The local nature of the approximators allows their fast adaptation to non-stationary changes and mitigates the biased sampling problem. The coexistence of multiple approximators updated and tried in parallel permits obtaining a good estimation much faster than would be possible with a single approximator. Experiments in different benchmark problems show that the competitive strategy provides a faster and more stable learning than non-competitive approaches.Preprin

A competitive strategy for function approximation in Q-learning

In this work we propose an approach for generalization in continuous domain Reinforcement Learning that, instead of using a single function approximator, tries many different function approximators in parallel, each one defined in a different region of the domain. Associated with each approximator is a relevance function that locally quantifies the quality of its approximation, so that, at each input point, the approximator with highest relevance can be selected. The relevance function is defined using parametric estimations of the variance of the q-values and the density of samples in the input space, which are used to quantify the accuracy and the confidence in the approximation, respectively. These parametric estimations are obtained from a probability density distribution represented as a Gaussian Mixture Model embedded in the input-output space of each approximator. In our experiments, the proposed approach required a lesser number of experiences for learning and produced more stable convergence profiles than when using a single function approximator.Peer ReviewedPreprin

Stochastic approximations of average values using proportions of samples

IRI Technical ReportIn this work we explain how the stochastic approximation of the average of a random variable is carried out when the observations used in the updates consist in proportion of samples rather than complete samples.Preprin

Antología poética: Gabriel Celaya

Antología poética de Gabriel Celaya

On-line learning of macro planning operators using probabilistic estimations of cause-effects

In this work we propose an on-line learning method for learning action rules for planning. The system uses a probabilistic approach of a constructive induction method that combines a beam search with an example-based search over candidate rules to find those that more concisely describe the world dynamics. The approach permits a rapid integration of the knowledge acquired from experience. Exploration of the world dynamics is guided by the planner, and – if the planner fails because of incomplete knowledge – by a teacher through action instructions

The Spanish society of Parenteral and Enteral Nutrition (SENPE) and its relation with healthcare authorities

Está muy bien documentado en la literatura médica que la desnutrición es un problema común en todos los niveles de atención sanitaria, desde atención primaria a especializada y en centros de atención geriátrica. Este problema no se limita a países con pocos recursos económicos o con limitado desarrollo social y económico. También es un problema universal en Europa. La desnutrición aumenta las cifras de morbilidad, mortalidad, ingresos hospitalarios y duración de la estancia. Estas cifras más elevadas suponen lógicamente un aumento del uso de recursos sanitarios. A pesar de esto, el problema de la desnutrición a menudo puede pasar desapercibido y el paciente no recibir el tratamiento necesario. Este problema requiere la cooperación de múltiples agentes tales como los Gobiernos de los Estados, los profesionales de la salud y los mismos ciudadanos. El VIII Foro de Debate concluye con la necesidad de establecer un claro plan de actuación (a semejanza de la European Alliance for Health Nutrition) y la creación de una plataforma (coalición) que reúna las voces de asociaciones de profesionales sanitarios, instituciones, colegios profesionales, asociaciones de pacientes, industria y entidades aseguradoras. Los fines de esta plataforma consistirán en informar de la extensión del problema, identificar y potenciar líderes que transmitan los fines de esta iniciativa ante las autoridades autonómicas y nacionales, propuesta de soluciones y colaboración en su puesta en marcha y finalmente, evaluación/ control de las acciones desarrolladasIt has been well documented in medical literature that hyponutrition is a common issue at all healthcare levels, from primary to specialized health care, as well as geria - tric healthcare facilities. This problem is not limited to countries with scarce economic resources or limited social development; it is also a universal issue in Europe. Hyponutrition increases the rates of morbidity, mortality, hospital admissions, and hospital stay. These higher figures also represent a higher use of healthcare resources. In spite of this, hyponutrition may often go undetected and the patient may not receive the necessary treatment. This problem requires the cooperation of multiple agents such as the Governments, the healthcare professionals, and the citizens themselves. The VIII Discussion Forum concludes on the need to establish a clear-cut plant for action (similar to the European Alliance for Health Nutrition) and the creation of a platform (coalition) encompassing the voices of healthcare professionals associations, institutions, professional colleges, patients associations, the pharmaceutical companies, and insurance companies. The goals of this platform will be to inform about the extent of this issue, to identity and promote leaders that will convey the aims of this initiative to regional and national healthcare authorities, to present solutions and to collaborate in their implementation, and finally to assess/control the actions take

Revista de occidente.

Monográfico: Chillid

Dedicatòria de Gabriel Celaya a José Agustín Goytisolo i Asunción Carandell

A la Ton y al José Agustín, muy corto ayer, y nunca igual, con un abrazo. Gabriel Celaya