Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

Application of multiobjective genetic programming to the design of robot failure recognition systems

We present an evolutionary approach using multiobjective genetic programming (MOGP) to derive optimal feature extraction preprocessing stages for robot failure detection. This data-driven machine learning method is compared both with conventional (nonevolutionary) classifiers and a set of domain-dependent feature extraction methods. We conclude MOGP is an effective and practical design method for failure recognition systems with enhanced recognition accuracy over conventional classifiers, independent of domain knowledge

Modeling and Optimization of M-cresol Isopropylation for Obtaining N-thymol: Combining a Hybrid Artificial Neural Network with a Genetic Algorithm

The application of a hybrid framework based on the combination, artificial neural network-genetic algorithm (ANN-GA), for n-thymol synthesis modeling and optimization has been developed. The effects of molar ratio propylene/cresol (X1), catalyst mass (X2) and temperature (X3) on n-thymol selectivity Y1 and m-cresol conversion Y2 were studied. A 3-8-2 ANN model was found to be very suitable for reaction modeling. The multiobjective optimization, led to optimal operating conditions (0.55 ≤X1≤0.77; 1.773 g ≤ X2 ≤1.86 g; 289.74 °C ≤ X3 ≤291.33 °C) representing good solutions for obtaining high n-thymol selectivity and high m-cresol conversion. This optimal zone corresponded to n-thymol selectivity and m-cresol conversion ranging respectively in the interval [79.3; 79.5]% and [13.4 %; 23.7]%. These results were better than those obtained with a sequential method based on experimental design for which, optimum conditions led to n-thymol selectivity and m-cresol conversion values respectively equal to 67%and 11%. The hybrid method ANN-GA showed its ability to solve complex problems with a good fitting

Genetic algorithm design of neural network and fuzzy logic controllers

Genetic algorithm design of neural network and fuzzy logic controller

Gradient-free activation maximization for identifying effective stimuli

A fundamental question for understanding brain function is what types of stimuli drive neurons to fire. In visual neuroscience, this question has also been posted as characterizing the receptive field of a neuron. The search for effective stimuli has traditionally been based on a combination of insights from previous studies, intuition, and luck. Recently, the same question has emerged in the study of units in convolutional neural networks (ConvNets), and together with this question a family of solutions were developed that are generally referred to as "feature visualization by activation maximization." We sought to bring in tools and techniques developed for studying ConvNets to the study of biological neural networks. However, one key difference that impedes direct translation of tools is that gradients can be obtained from ConvNets using backpropagation, but such gradients are not available from the brain. To circumvent this problem, we developed a method for gradient-free activation maximization by combining a generative neural network with a genetic algorithm. We termed this method XDream (EXtending DeepDream with real-time evolution for activation maximization), and we have shown that this method can reliably create strong stimuli for neurons in the macaque visual cortex (Ponce et al., 2019). In this paper, we describe extensive experiments characterizing the XDream method by using ConvNet units as in silico models of neurons. We show that XDream is applicable across network layers, architectures, and training sets; examine design choices in the algorithm; and provide practical guides for choosing hyperparameters in the algorithm. XDream is an efficient algorithm for uncovering neuronal tuning preferences in black-box networks using a vast and diverse stimulus space.Comment: 16 pages, 8 figures, 3 table