A Gentle Introduction to Reinforcement Learning and its Application in Different Fields

Due to the recent progress in Deep Neural Networks, Reinforcement Learning (RL) has become one of the most important and useful technology. It is a learning method where a software agent interacts with an unknown environment, selects actions, and progressively discovers the environment dynamics. RL has been effectively applied in many important areas of real life. This article intends to provide an in-depth introduction of the Markov Decision Process, RL and its algorithms. Moreover, we present a literature review of the application of RL to a variety of fields, including robotics and autonomous control, communication and networking, natural language processing, games and self-organized system, scheduling management and configuration of resources, and computer vision

Quality-diversity optimization: a novel branch of stochastic optimization

Traditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning

Quality-diversity optimization: a novel branch of stochastic optimization

Traditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning

On Partially Controlled Multi-Agent Systems

Motivated by the control theoretic distinction between controllable and uncontrollable events, we distinguish between two types of agents within a multi-agent system: controllable agents, which are directly controlled by the system's designer, and uncontrollable agents, which are not under the designer's direct control. We refer to such systems as partially controlled multi-agent systems, and we investigate how one might influence the behavior of the uncontrolled agents through appropriate design of the controlled agents. In particular, we wish to understand which problems are naturally described in these terms, what methods can be applied to influence the uncontrollable agents, the effectiveness of such methods, and whether similar methods work across different domains. Using a game-theoretic framework, this paper studies the design of partially controlled multi-agent systems in two contexts: in one context, the uncontrollable agents are expected utility maximizers, while in the other they are reinforcement learners. We suggest different techniques for controlling agents' behavior in each domain, assess their success, and examine their relationship.Comment: See http://www.jair.org/ for any accompanying file

Fear in horses

Fear is generally considered to be an undesirable emotional state that may reduce welfare, growth and reproductive performance in animals. Fear in horses is additionally problematic, because fear reactions can cause serious injury to both horse and human. Horses are primarily used for sports and leisure for a large number of children and young women. Unfortunately, horse riding ranks as one of the most dangerous sports in terms of the number and seriousness of accidents, and the ability of a horse to habituate to a range of otherwise frightening stimuli greatly increases safety in the horse-human relationship. However, there is a lack of research on fear reactions and no published research on basic habituation processes in horses. This licentiate project aimed to investigate the types of fear responses horses show towards novel stimuli acting on different senses, and to study how horses learn to be confident with an otherwise frightening stimulus using classical learning theory techniques. The experiments were conducted on two different groups of naïve stallions (n=24 and n=27). The first experiment showed that horses responded differently towards an olfactory stimulus compared to auditory and visual stimuli. The heart rate responses correlated between tests and probably reflected a non-differentiated activation of the sympathetic nervous system, while the behavioural responses were linked to the type of stimulus. The second experiment showed that gradual habituation was the most effective training method for horses to learn to react calmly to an otherwise frightening stimulus, compared to classic habituation and associative learning. Heart rate data revealed that horses may show physiological responses even when their behavioural response towards the stimulus has ceased. Choice of training method is likely to be especially important for the most fearful horses