A Developmental Organization for Robot Behavior

This paper focuses on exploring how learning and development can be structured in synthetic (robot) systems. We present a developmental assembler for constructing reusable and temporally extended actions in a sequence. The discussion adopts the traditions of dynamic pattern theory in which behavior is an artifact of coupled dynamical systems with a number of controllable degrees of freedom. In our model, the events that delineate control decisions are derived from the pattern of (dis)equilibria on a working subset of sensorimotor policies. We show how this architecture can be used to accomplish sequential knowledge gathering and representation tasks and provide examples of the kind of developmental milestones that this approach has already produced in our lab

Deep Reinforcement Learning for Swarm Systems

Recently, deep reinforcement learning (RL) methods have been applied successfully to multi-agent scenarios. Typically, these methods rely on a concatenation of agent states to represent the information content required for decentralized decision making. However, concatenation scales poorly to swarm systems with a large number of homogeneous agents as it does not exploit the fundamental properties inherent to these systems: (i) the agents in the swarm are interchangeable and (ii) the exact number of agents in the swarm is irrelevant. Therefore, we propose a new state representation for deep multi-agent RL based on mean embeddings of distributions. We treat the agents as samples of a distribution and use the empirical mean embedding as input for a decentralized policy. We define different feature spaces of the mean embedding using histograms, radial basis functions and a neural network learned end-to-end. We evaluate the representation on two well known problems from the swarm literature (rendezvous and pursuit evasion), in a globally and locally observable setup. For the local setup we furthermore introduce simple communication protocols. Of all approaches, the mean embedding representation using neural network features enables the richest information exchange between neighboring agents facilitating the development of more complex collective strategies.Comment: 31 pages, 12 figures, version 3 (published in JMLR Volume 20

Goals, usefulness and abstraction in value-based choice

Colombian drug lord Pablo Escobar, while on the run, purportedly burned two million dollars in banknotes to keep his daughter warm. A stark reminder that, in life, circumstances and goals can quickly change, forcing us to reassess and modify our values on-the-fly. Studies in decision-making and neuroeconomics have often implicitly equated value to reward, emphasising the hedonic and automatic aspect of the value computation, while overlooking its functional (concept-like) nature. Here we outline the computational and biological principles that enable the brain to compute the usefulness of an option or action by creating abstractions that flexibly adapt to changing goals. We present different algorithmic architectures, comparing ideas from artificial intelligence (AI) and cognitive neuroscience with psychological theories and, when possible, drawing parallels

Emerging Artificial Societies Through Learning

The NewTies project is implementing a simulation in which societies of agents are expected to de-velop autonomously as a result of individual, population and social learning. These societies are expected to be able to solve environmental challenges by acting collectively. The challenges are in-tended to be analogous to those faced by early, simple, small-scale human societies. This report on work in progress outlines the major features of the system as it is currently conceived within the project, including the design of the agents, the environment, the mechanism for the evolution of language and the peer-to-peer infrastructure on which the simulation runs.Artificial Societies, Evolution of Language, Decision Trees, Peer-To-Peer Networks, Social Learning

Cognition-Based Networks: A New Perspective on Network Optimization Using Learning and Distributed Intelligence

IEEE Access Volume 3, 2015, Article number 7217798, Pages 1512-1530 Open Access Cognition-based networks: A new perspective on network optimization using learning and distributed intelligence (Article) Zorzi, M.a , Zanella, A.a, Testolin, A.b, De Filippo De Grazia, M.b, Zorzi, M.bc a Department of Information Engineering, University of Padua, Padua, Italy b Department of General Psychology, University of Padua, Padua, Italy c IRCCS San Camillo Foundation, Venice-Lido, Italy View additional affiliations View references (107) Abstract In response to the new challenges in the design and operation of communication networks, and taking inspiration from how living beings deal with complexity and scalability, in this paper we introduce an innovative system concept called COgnition-BAsed NETworkS (COBANETS). The proposed approach develops around the systematic application of advanced machine learning techniques and, in particular, unsupervised deep learning and probabilistic generative models for system-wide learning, modeling, optimization, and data representation. Moreover, in COBANETS, we propose to combine this learning architecture with the emerging network virtualization paradigms, which make it possible to actuate automatic optimization and reconfiguration strategies at the system level, thus fully unleashing the potential of the learning approach. Compared with the past and current research efforts in this area, the technical approach outlined in this paper is deeply interdisciplinary and more comprehensive, calling for the synergic combination of expertise of computer scientists, communications and networking engineers, and cognitive scientists, with the ultimate aim of breaking new ground through a profound rethinking of how the modern understanding of cognition can be used in the management and optimization of telecommunication network

Modelling learning behaviour of intelligent agents using UML 2.0

This thesis aims to explore and demonstrate the ability of the new standard of structural and behavioural components in Unified Modelling Language (UML 2.0 / 2004) to model the learning behaviour of Intelligent Agents. The thesis adopts the research direction that views agent-oriented systems as an extension to object-oriented systems. In view of the fact that UML has been the de facto standard for modelling object-oriented systems, this thesis concentrates on exploring such modelling potential with Intelligent Agent-oriented systems. Intelligent Agents are Agents that have the capability to learn and reach agreement with other Agents or users. The research focuses on modelling the learning behaviour of a single Intelligent Agent, as it is the core of multi-agent systems. During the writing of the thesis, the only work done to use UML 2.0 to model structural components of Agents was from the Foundation for Intelligent Physical Agent (FIPA). The research builds upon, explores, and utilises this work and provides further development to model the structural components of learning behaviour of Intelligent Agents. The research also shows the ability of UML version 2.0 behaviour diagrams, namely activity diagrams and sequence diagrams, to model the learning behaviour of Intelligent Agents that use learning from observation and discovery as well as learning from examples of strategies. The research also evaluates if UML 2.0 state machine diagrams can model specific reinforcement learning algorithms, namely dynamic programming, Monte Carlo, and temporal difference algorithms. The thesis includes user guides of UML 2.0 activity, sequence, and state machine diagrams to allow researchers in agent-oriented systems to use the UML 2.0 diagrams in modelling the learning components of Intelligent Agents. The capacity for learning is a crucial feature of Intelligent Agents. The research identifies different learning components required to model the learning behaviour of Intelligent Agents such as learning goals, learning strategies, and learning feedback methods. In recent years, the Agent-oriented research has been geared towards the agency dimension of Intelligent Agents. Thus, there is a need to conduct more research on the intelligence dimension of Intelligent Agents, such as negotiation and argumentation skills. The research shows that behavioural components of UML 2.0 are capable of modelling the learning behaviour of Intelligent Agents while structural components of UML 2.0 need extension to cover structural requirements of Agents and Intelligent Agents. UML 2.0 has an extension mechanism to fulfil Agents and Intelligent Agents for such requirements. This thesis will lead to increasing interest in the intelligence dimension rather than the agency dimension of Intelligent Agents, and pave the way for objectoriented methodologies to shift more easily to paradigms of Intelligent Agent-oriented systems.The British Council, the University of Plymouth and the Arab-British Chamber Charitable Foundation

Learning to play using low-complexity rule-based policies: Illustrations through Ms. Pac-Man

In this article we propose a method that can deal with certain combinatorial reinforcement learning tasks. We demonstrate the approach in the popular Ms. Pac-Man game. We define a set of high-level observation and action modules, from which rule-based policies are constructed automatically. In these policies, actions are temporally extended, and may work concurrently. The policy of the agent is encoded by a compact decision list. The components of the list are selected from a large pool of rules, which can be either hand-crafted or generated automatically. A suitable selection of rules is learnt by the cross-entropy method, a recent global optimization algorithm that fits our framework smoothly. Cross-entropy-optimized policies perform better than our hand-crafted policy, and reach the score of average human players. We argue that learning is successful mainly because (i) policies may apply concurrent actions and thus the policy space is sufficiently rich, (ii) the search is biased towards low-complexity policies and therefore, solutions with a compact description can be found quickly if they exist

Deep learning for video game playing

In this article, we review recent Deep Learning advances in the context of how they have been applied to play different types of video games such as first-person shooters, arcade games, and real-time strategy games. We analyze the unique requirements that different game genres pose to a deep learning system and highlight important open challenges in the context of applying these machine learning methods to video games, such as general game playing, dealing with extremely large decision spaces and sparse rewards