Reinforcement Learning Embedded in Brains and Robots

In many ways and in various tasks, computers are able to outperform humans. They can store and retrieve much larger amounts of data or even beat humans at chess. However, when looking at robots they are still far behind even a small child in terms of their performance capabilities. Even a sophisticated robot, such as ASIMO, is limited to mostl

Neural Control of Actions Involving Different Coordinate Systems

di Parm

Clyde: A deep reinforcement learning DOOM playing agent

In this paper we present the use of deep reinforcement learn-ing techniques in the context of playing partially observablemulti-agent 3D games. These techniques have traditionallybeen applied to fully observable 2D environments, or navigation tasks in 3D environments. We show the performanceof Clyde in comparison to other competitors within the con-text of the ViZDOOM competition that saw 9 bots competeagainst each other in DOOM death matches. Clyde managedto achieve 3rd place in the ViZDOOM competition held at theIEEE Conference on Computational Intelligence and Games2016. Clyde performed very well considering its relative sim-plicity and the fact that we deliberately avoided a high levelof customisation to keep the algorithm generic

Visual Attention in Dynamic Environments and its Application to Playing Online Games

Abstract In this thesis we present a prototype of Cognitive Programs (CPs) - an executive controller built on top of Selective Tuning (ST) model of attention. CPs enable top-down control of visual system and interaction between the low-level vision and higher-level task demands. Abstract We implement a subset of CPs for playing online video games in real time using only visual input. Two commercial closed-source games - Canabalt and Robot Unicorn Attack - are used for evaluation. Their simple gameplay and minimal controls put the emphasis on reaction speed and attention over planning. Abstract Our implementation of Cognitive Programs plays both games at human expert level, which experimentally proves the validity of the concept. Additionally we resolved multiple theoretical and engineering issues, e.g. extending the CPs to dynamic environments, finding suitable data structures for describing the task and information flow within the network and determining the correct timing for each process

Grounding the Meanings in Sensorimotor Behavior using Reinforcement Learning

The recent outburst of interest in cognitive developmental robotics is fueled by the ambition to propose ecologically plausible mechanisms of how, among other things, a learning agent/robot could ground linguistic meanings in its sensorimotor behavior. Along this stream, we propose a model that allows the simulated iCub robot to learn the meanings of actions (point, touch, and push) oriented toward objects in robot’s peripersonal space. In our experiments, the iCub learns to execute motor actions and comment on them. Architecturally, the model is composed of three neural-network-based modules that are trained in different ways. The first module, a two-layer perceptron, is trained by back-propagation to attend to the target position in the visual scene, given the low-level visual information and the feature-based target information. The second module, having the form of an actor-critic architecture, is the most distinguishing part of our model, and is trained by a continuous version of reinforcement learning to execute actions as sequences, based on a linguistic command. The third module, an echo-state network, is trained to provide the linguistic description of the executed actions. The trained model generalizes well in case of novel action-target combinations with randomized initial arm positions. It can also promptly adapt its behavior if the action/target suddenly changes during motor execution

A Posture Sequence Learning System for an Anthropomorphic Robotic Hand

The paper presents a cognitive architecture for posture learning of an anthropomorphic robotic hand. Our approach is aimed to allow the robotic system to perform complex perceptual operations, to interact with a human user and to integrate the perceptions by a cognitive representation of the scene and the observed actions. The anthropomorphic robotic hand imitates the gestures acquired by the vision system in order to learn meaningful movements, to build its knowledge by different conceptual spaces and to perform complex interaction with the human operator

Video-Game Agents with Human-Like Behavior Using DQN and Biological Constraints

Video game agents that surpass humans always select the optimal behavior, which may make them look mechanical and uninteresting to human players and audience. Since score-oriented game agents have been almost achieved, a next goal should be to entertain human players and audience by realizing agents that reproduce human-like behavior. A previous method implemented such game agents by introducing biological constraints into Q-learning and A^* search. In this paper, we propose video game agents with more entertaining and more practical human-like behavior by applying biological constraints into the deep Q-network (DQN). Especially, to reduce the problem of the conspicuous mechanical behavior found in the previous method, we improve the method of the biological constraint “tired”, and propose additional biological constraints “confusion” and “carelessness”. We implemented our method in the video game “Infinite Mario Bros.” and conducted two types of experiments that were subjective evaluation. One is to evaluate the behavior of game agents that improved “tired”. The other is to evaluate the behavior of game agents that introduced “confusion” and “carelessness” The results of both experiments indicated that the agents implemented with our method were rated more human-like than those implemented with the previous method

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field