A Developmental Organization for Robot Behavior

This paper focuses on exploring how learning and development can be structured in synthetic (robot) systems. We present a developmental assembler for constructing reusable and temporally extended actions in a sequence. The discussion adopts the traditions of dynamic pattern theory in which behavior is an artifact of coupled dynamical systems with a number of controllable degrees of freedom. In our model, the events that delineate control decisions are derived from the pattern of (dis)equilibria on a working subset of sensorimotor policies. We show how this architecture can be used to accomplish sequential knowledge gathering and representation tasks and provide examples of the kind of developmental milestones that this approach has already produced in our lab

Self-Organization in Decentralized Networks: A Trial and Error Learning Approach

International audienceIn this paper, the problem of channel selection and power control is jointly analyzed in the context of multiple-channel clustered ad-hoc networks, i.e., decentralized networks in which radio devices are arranged into groups (clusters) and each cluster is managed by a central controller (CC). This problem is modeled by game in normal form in which the corresponding utility functions are designed for making some of the Nash equilibria (NE) to coincide with the solutions to a global network optimization problem. In order to ensure that the network operates in the equilibria that are globally optimal, a learning algorithm based on the paradigm of trial and error learning is proposed. These results are presented in the most general form and therefore, they can also be seen as a framework for designing both games and learning algorithms with which decentralized networks can operate at global optimal points using only their available local knowledge. The pertinence of the game design and the learning algorithm are highlighted using specific scenarios in decentralized clustered ad hoc networks. Numerical results confirm the relevance of using appropriate utility functions and trial and error learning for enhancing the performance of decentralized networks

Democracy under uncertainty: The ‘wisdom of crowds’ and the free-rider problem in group decision making

We introduce a game theory model of individual decisions to cooperate by contributing personal resources to group decisions versus by free-riding on the contributions of other members. In contrast to most public-goods games that assume group returns are linear in individual contributions, the present model assumes decreasing marginal group production as a function of aggregate individual contributions. This diminishing marginal returns assumption is more realistic and generates starkly different predictions compared to the linear model. One important implication is that, under most conditions, there exist equilibria where some, but not all members of a group contribute, even with completely self-interested motives. An agent-based simulation confirms the individual and group advantages of the equilibria in which behavioral asymmetry emerges from a game structure that is a priori perfectly symmetric for all agents (all agents have the same payoff function and action space, but take different actions in equilibria). And a behavioral experiment demonstrates that cooperators and free-riders coexist in a stable manner in groups performing with the non-linear production function. A collateral result demonstrates that, compared to a ―dictatorial‖ decision scheme guided by the best member in a group, the majority-plurality decision rules can pool information effectively and produce greater individual net welfare at equilibrium, even if free-riding is not sanctioned. This is an original proof that cooperation in ad hoc decision-making groups can be understood in terms of self-interested motivations and that, despite the free-rider problem, majority-plurality decision rules can function robustly as simple, efficient social decision heuristics.group decision making under uncertainty, free-rider problem, majority-plurality rules, marginally-diminishing group returns, evolutionary games, behavioral experiment

CAutoCSD-evolutionary search and optimisation enabled computer automated control system design

This paper attempts to set a unified scene for various linear time-invariant (LTI) control system design schemes, by transforming the existing concept of 'Computer-Aided Control System Design' (CACSD) to the novel 'Computer-Automated Control System Design' (CAutoCSD). The first step towards this goal is to accommodate, under practical constraints, various design objectives that are desirable in both time and frequency-domains. Such performance-prioritised unification is aimed to relieve practising engineers from having to select a particular control scheme and from sacrificing certain performance goals resulting from pre-committing to the adopted scheme. With the recent progress in evolutionary computing based extra-numeric, multi-criterion search and optimisation techniques, such unification of LTI control schemes becomes feasible, analytically and practically, and the resultant designs can be creative. The techniques developed are applied to, and illustrated by, three design problems. The unified approach automatically provides an integrator for zero-steady state error in velocity control of a DC motor, meets multiple objectives in designing an LTI controller for a non-minimum phase plant and offers a high-performing LTI controller network for a nonlinear chemical process

Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems.

International audienceIn the framework of fully cooperative multi-agent systems, independent (non-communicative) agents that learn by reinforcement must overcome several difficulties to manage to coordinate. This paper identifies several challenges responsible for the non-coordination of independent agents: Pareto-selection, nonstationarity, stochasticity, alter-exploration and shadowed equilibria. A selection of multi-agent domains is classified according to those challenges: matrix games, Boutilier's coordination game, predators pursuit domains and a special multi-state game. Moreover the performance of a range of algorithms for independent reinforcement learners is evaluated empirically. Those algorithms are Q-learning variants: decentralized Q-learning, distributed Q-learning, hysteretic Q-learning, recursive FMQ and WoLF PHC. An overview of the learning algorithms' strengths and weaknesses against each challenge concludes the paper and can serve as a basis for choosing the appropriate algorithm for a new domain. Furthermore, the distilled challenges may assist in the design of new learning algorithms that overcome these problems and achieve higher performance in multi-agent applications