Reuse of Neural Modules for General Video Game Playing

A general approach to knowledge transfer is introduced in which an agent controlled by a neural network adapts how it reuses existing networks as it learns in a new domain. Networks trained for a new domain can improve their performance by routing activation selectively through previously learned neural structure, regardless of how or for what it was learned. A neuroevolution implementation of this approach is presented with application to high-dimensional sequential decision-making domains. This approach is more general than previous approaches to neural transfer for reinforcement learning. It is domain-agnostic and requires no prior assumptions about the nature of task relatedness or mappings. The method is analyzed in a stochastic version of the Arcade Learning Environment, demonstrating that it improves performance in some of the more complex Atari 2600 games, and that the success of transfer can be predicted based on a high-level characterization of game dynamics.Comment: Accepted at AAAI 1

Inference in classifier systems

Classifier systems (Css) provide a rich framework for learning and induction, and they have beenı successfully applied in the artificial intelligence literature for some time. In this paper, both theı architecture and the inferential mechanisms in general CSs are reviewed, and a number of limitations and extensions of the basic approach are summarized. A system based on the CS approach that is capable of quantitative data analysis is outlined and some of its peculiarities discussed

Evolution of associative learning in chemical networks

Organisms that can learn about their environment and modify their behaviour appropriately during their lifetime are more likely to survive and reproduce than organisms that do not. While associative learning – the ability to detect correlated features of the environment – has been studied extensively in nervous systems, where the underlying mechanisms are reasonably well understood, mechanisms within single cells that could allow associative learning have received little attention. Here, using in silico evolution of chemical networks, we show that there exists a diversity of remarkably simple and plausible chemical solutions to the associative learning problem, the simplest of which uses only one core chemical reaction. We then asked to what extent a linear combination of chemical concentrations in the network could approximate the ideal Bayesian posterior of an environment given the stimulus history so far? This Bayesian analysis revealed the ’memory traces’ of the chemical network. The implication of this paper is that there is little reason to believe that a lack of suitable phenotypic variation would prevent associative learning from evolving in cell signalling, metabolic, gene regulatory, or a mixture of these networks in cells

Learning Reservoir Dynamics with Temporal Self-Modulation

Reservoir computing (RC) can efficiently process time-series data by transferring the input signal to randomly connected recurrent neural networks (RNNs), which are referred to as a reservoir. The high-dimensional representation of time-series data in the reservoir significantly simplifies subsequent learning tasks. Although this simple architecture allows fast learning and facile physical implementation, the learning performance is inferior to that of other state-of-the-art RNN models. In this paper, to improve the learning ability of RC, we propose self-modulated RC (SM-RC), which extends RC by adding a self-modulation mechanism. The self-modulation mechanism is realized with two gating variables: an input gate and a reservoir gate. The input gate modulates the input signal, and the reservoir gate modulates the dynamical properties of the reservoir. We demonstrated that SM-RC can perform attention tasks where input information is retained or discarded depending on the input signal. We also found that a chaotic state emerged as a result of learning in SM-RC. This indicates that self-modulation mechanisms provide RC with qualitatively different information-processing capabilities. Furthermore, SM-RC outperformed RC in NARMA and Lorentz model tasks. In particular, SM-RC achieved a higher prediction accuracy than RC with a reservoir 10 times larger in the Lorentz model tasks. Because the SM-RC architecture only requires two additional gates, it is physically implementable as RC, providing a new direction for realizing edge AI

Action Generalization in Humanoid Robots Through Artificial Intelligence With Learning From Demonstration

Mención Internacional en el título de doctorAction Generalization is the ability to adapt an action to different contexts and environments. In humans, this ability is taken for granted. Robots are yet far from achieving the human level of Action Generalization. Current robotic frameworks are limited frameworks that are only able to work in the small range of contexts and environments for which they were programmed. One of the reasons why we do not have a robot in our house yet is because every house is different. In this thesis, two different approaches to improve the Action Generalization capabilities of robots are proposed. First, a study of different methods to improve the performance of the Continuous Goal-Directed Actions framework within highly dynamic real world environments is presented. Continuous Goal-Directed Actions is a Learning from Demonstration framework based on the idea of encoding actions as the effects these actions produce on the environment. No robot kinematic information is required for the encoding of actions. This improves the generalization capabilities of robots by solving the correspondence problem. This problem is related to the execution of the same action with different kinematics. The second approach is the proposition of the Neural Policy Style Transfer framework. The goal of this framework is to achieve Action Generalization by providing the robot the ability to introduce Styles within robotic actions. This allows the robot to adapt one action to different contexts with the introduction of different Styles. Neural Style Transfer was originally proposed as a way to perform Style Transfer between images. Neural Policy Style Transfer proposes the introduction of Neural Style Transfer within robotic actions. The structure of this document was designed with the goal of depicting the continuous research work that this thesis has been. Every time a new approach is proposed, the reasons why this was considered the best new step based on the experimental results obtained are provided. Each approach can be studied separately and, at the same time, they are presented as part of the larger research project from which they are part. Solving the problem of Action Generalization is currently a too ambitious goal for any single research project. The goal of this thesis is to make finding this solution one step closer.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Saffiotti Alessandro.- Secretario: Santiago Martínez de la Casa Díaz.- Vocal: Fernando Torres Medin

Continual Lifelong Learning with Neural Networks: A Review

Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. This ability, referred to as lifelong learning, is mediated by a rich set of neurocognitive mechanisms that together contribute to the development and specialization of our sensorimotor skills as well as to long-term memory consolidation and retrieval. Consequently, lifelong learning capabilities are crucial for autonomous agents interacting in the real world and processing continuous streams of information. However, lifelong learning remains a long-standing challenge for machine learning and neural network models since the continual acquisition of incrementally available information from non-stationary data distributions generally leads to catastrophic forgetting or interference. This limitation represents a major drawback for state-of-the-art deep neural network models that typically learn representations from stationary batches of training data, thus without accounting for situations in which information becomes incrementally available over time. In this review, we critically summarize the main challenges linked to lifelong learning for artificial learning systems and compare existing neural network approaches that alleviate, to different extents, catastrophic forgetting. We discuss well-established and emerging research motivated by lifelong learning factors in biological systems such as structural plasticity, memory replay, curriculum and transfer learning, intrinsic motivation, and multisensory integration