Feature discovery in reinforcement learning using genetic programming

International audienceThe goal of reinforcement learning is to find a policy, directly or indirectly through a value function, that maximizes the expected re- ward accumulated by an agent over time based on its interactions with the environment; a function of the state has to be learned. In some prob- lems, it may not be feasible, or even possible, to use the state variables as they are. Instead, a set of features are computed and used as in- put. However, finding a "good" set of features is generally a tedious task which requires a good domain knowledge. In this paper, we propose a ge- netic programming based approach for feature discovery in reinforcement learning. A population of individuals each representing possibly different number of candidate features is evolved, and feature sets are evaluated by their average performance on short learning trials. The results of ex- periments conducted on several benchmark problems demonstrate that the resulting features allow the agent to learn better policies

Generating Interpretable Fuzzy Controllers using Particle Swarm Optimization and Genetic Programming

Autonomously training interpretable control strategies, called policies, using pre-existing plant trajectory data is of great interest in industrial applications. Fuzzy controllers have been used in industry for decades as interpretable and efficient system controllers. In this study, we introduce a fuzzy genetic programming (GP) approach called fuzzy GP reinforcement learning (FGPRL) that can select the relevant state features, determine the size of the required fuzzy rule set, and automatically adjust all the controller parameters simultaneously. Each GP individual's fitness is computed using model-based batch reinforcement learning (RL), which first trains a model using available system samples and subsequently performs Monte Carlo rollouts to predict each policy candidate's performance. We compare FGPRL to an extended version of a related method called fuzzy particle swarm reinforcement learning (FPSRL), which uses swarm intelligence to tune the fuzzy policy parameters. Experiments using an industrial benchmark show that FGPRL is able to autonomously learn interpretable fuzzy policies with high control performance.Comment: Accepted at Genetic and Evolutionary Computation Conference 2018 (GECCO '18

Multi-agent evolutionary systems for the generation of complex virtual worlds

Modern films, games and virtual reality applications are dependent on convincing computer graphics. Highly complex models are a requirement for the successful delivery of many scenes and environments. While workflows such as rendering, compositing and animation have been streamlined to accommodate increasing demands, modelling complex models is still a laborious task. This paper introduces the computational benefits of an Interactive Genetic Algorithm (IGA) to computer graphics modelling while compensating the effects of user fatigue, a common issue with Interactive Evolutionary Computation. An intelligent agent is used in conjunction with an IGA that offers the potential to reduce the effects of user fatigue by learning from the choices made by the human designer and directing the search accordingly. This workflow accelerates the layout and distribution of basic elements to form complex models. It captures the designer's intent through interaction, and encourages playful discovery