Deep learning based approaches for imitation learning.

Imitation learning refers to an agent's ability to mimic a desired behaviour by learning from observations. The field is rapidly gaining attention due to recent advances in computational and communication capabilities as well as rising demand for intelligent applications. The goal of imitation learning is to describe the desired behaviour by providing demonstrations rather than instructions. This enables agents to learn complex behaviours with general learning methods that require minimal task specific information. However, imitation learning faces many challenges. The objective of this thesis is to advance the state of the art in imitation learning by adopting deep learning methods to address two major challenges of learning from demonstrations. Firstly, representing the demonstrations in a manner that is adequate for learning. We propose novel Convolutional Neural Networks (CNN) based methods to automatically extract feature representations from raw visual demonstrations and learn to replicate the demonstrated behaviour. This alleviates the need for task specific feature extraction and provides a general learning process that is adequate for multiple problems. The second challenge is generalizing a policy over unseen situations in the training demonstrations. This is a common problem because demonstrations typically show the best way to perform a task and don't offer any information about recovering from suboptimal actions. Several methods are investigated to improve the agent's generalization ability based on its initial performance. Our contributions in this area are three fold. Firstly, we propose an active data aggregation method that queries the demonstrator in situations of low confidence. Secondly, we investigate combining learning from demonstrations and reinforcement learning. A deep reward shaping method is proposed that learns a potential reward function from demonstrations. Finally, memory architectures in deep neural networks are investigated to provide context to the agent when taking actions. Using recurrent neural networks addresses the dependency between the state-action sequences taken by the agent. The experiments are conducted in simulated environments on 2D and 3D navigation tasks that are learned from raw visual data, as well as a 2D soccer simulator. The proposed methods are compared to state of the art deep reinforcement learning methods. The results show that deep learning architectures can learn suitable representations from raw visual data and effectively map them to atomic actions. The proposed methods for addressing generalization show improvements over using supervised learning and reinforcement learning alone. The results are thoroughly analysed to identify the benefits of each approach and situations in which it is most suitable

Predicting opponent actions in the RoboSoccer

Proceeding of: IEEE International Conference on Systems, Man, and Cybernetics (SMC-2002), 6-9 Oct. 2002, Hammamet, TunezA very important issue in multi-agent systems is that of adaptability to other agents, be it to cooperate or to compete. In competitive domains, the knowledge about the opponent can give any player a clear advantage. In previous work, we acquired models of another agent (the opponent) based only on the observation of its inputs and outputs (its behavior) by formulating the problem as a classification task. In this paper we extend this previous work to the RoboCup domain. However, we have found that models based on a single classifier have bad accuracy, To solve this problem, In this paper we propose to decompose the learning task into two tasks: learning the action name (i.e. kick or dash) and learning the parameter of that action. By using this hierarchical learning approach accuracy results improve, and at worst, the agent can know what action the opponent will carry out, even if there is no high accuracy on the action parameter.Publicad

A Hybrid Multi-Robot Control Architecture

Multi-robot systems provide system redundancy and enhanced capability versus single robot systems. Implementations of these systems are varied, each with specific design approaches geared towards an application domain. Some traditional single robot control architectures have been expanded for multi-robot systems, but these expansions predominantly focus on the addition of communication capabilities. Both design approaches are application specific and limit the generalizability of the system. This work presents a redesign of a common single robot architecture in order to provide a more sophisticated multi-robot system. The single robot architecture chosen for application is the Three Layer Architecture (TLA). The primary strength of TLA is in the ability to perform both reactive and deliberative decision making, enabling the robot to be both sophisticated and perform well in stochastic environments. The redesign of this architecture includes incorporation of the Unified Behavior Framework (UBF) into the controller layer and an addition of a sequencer-like layer (called a Coordinator) to accommodate the multi-robot system. These combine to provide a robust, independent, and taskable individual architecture along with improved cooperation and collaboration capabilities, in turn reducing communication overhead versus many traditional approaches. This multi-robot systems architecture is demonstrated on the RoboCup Soccer Simulator showing its ability to perform well in a dynamic environment where communication constraints are high