Diversity and Specialization in Collaborative Swarm Systems

This paper addresses qualitative and quantitative diversity and specialization issues in the frame- work of self-organizing, distributed, artificial systems. Both diversity and specialization are obtained via distributed learning from initially homogeneous swarms. While measuring diversity essentially quantifies differences among the individuals, assessing the degree of specialization implies to correlate the swarm’s heterogeneity with its overall performance. Starting from a stick-pulling experiment in collective robotics, a task that requires the collaboration of two robots, we abstract and generalize in simulation the task constraints to k robots collaborating sequentially or in parallel. We investi- gate quantitatively the influence of task constraints and type of reinforcement signals on diversity and specialization in these collaborative experiments. Results show that, though diversity is not explicitly rewarded in our learning algorithm and there is no explicit communication among agents, the swarm becomes specialized after learning. The degree of specialization is affected strongly by environmental conditions and task constraints, and reveals characteristics related to performance and learning in a more consistent and clearer way than diversity does

Discrete Message via Online Clustering Labels in Decentralized POMDP

Communication is crucial for solving cooperative Multi-Agent Reinforcement Learning tasks in Partially-Observable Markov Decision Processes. Existing works often rely on black-box methods to encode local information/features into messages shared with other agents. However, such black-box approaches are unable to provide any quantitative guarantees on the expected return and often lead to the generation of continuous messages with high communication overhead and poor interpretability. In this paper, we establish an upper bound on the return gap between an ideal policy with full observability and an optimal partially-observable policy with discrete communication. This result enables us to recast multi-agent communication into a novel online clustering problem over the local observations at each agent, with messages as cluster labels and the upper bound on the return gap as clustering loss. By minimizing the upper bound, we propose a surprisingly simple design of message generation functions in multi-agent communication and integrate it with reinforcement learning using a Regularized Information Maximization loss function. Evaluations show that the proposed discrete communication significantly outperforms state-of-the-art multi-agent communication baselines and can achieve nearly-optimal returns with few-bit messages that are naturally interpretable

Learning and Measuring Specialization in Collaborative Swarm Systems

This paper addresses qualitative and quantitative diversity and specialization issues in the framework of selforganizing, distributed, artificial systems. Both diversity and specialization are obtained via distributed learning from initially homogeneous swarms. While measuring diversity essentially quantifies differences among the individuals, assessing the degree of specialization implies correlation between the swarm’s heterogeneity with its overall performance. Starting from the stick-pulling experiment in collective robotics, a task that requires the collaboration of two robots, we abstract and generalize in simulation the task constraints to k robots collaborating sequentially or in parallel. We investigate quantitatively the influence of task constraints and types of reinforcement signals on performance, diversity, and specialization in these collaborative experiments. Results show that, though diversity is not explicitly rewarded in our learning algorithm, even in scenarios without explicit communication among agents the swarm becomes specialized after learning. The degrees of both diversity and specialization are affected strongly by environmental conditions and task constraints. While the specialization measure reveals characteristics related to performance and learning in a clearer way than diversity does, the latter measure appears to be less sensitive to different noise conditions and learning parameters

Modeling of Swarm Robotic Systems: A Case Study in Collaborative Distributed Manipulation

In this paper, we present a time-discrete, incremental methodology for modeling, at the microscopic and macroscopic level, the dynamics of distributed manipulation experiments using swarms of autonomous robots endowed with reactive controllers. The methodology is well-suited for nonspatial metrics since it does not take into account robots’ trajectories or the spatial distribution of objects in the environment. The strength of the methodology lies in the fact that it has been generated by considering incremental abstraction steps, from real robots to macroscopic models, each with well-defined mappings between successive implementation levels. Precise heuristic criteria based on geometrical considerations and systematic tests with one or two real robots prevent the introduction of free parameters in the calibration procedure of models. As a consequence, we are able to generate highly abstracted macroscopic models that can capture the dynamics of a swarm of robots at the behavioral level while still being closely anchored to the characteristics of the physical set-up. Although this methodology has been and can be applied to other experiments in distributed manipulation (e.g., object aggregation and segregation, foraging), in this paper we focus on a strictly collaborative case study concerned with pulling sticks out of the ground, an action that requires the collaboration of two robots to be successful. Experiments were carried out with teams consisting of two to 600 individuals at different levels of implementation (real robots, embodied simulations, microscopic and macroscopic models). Results show that models can deliver both qualitatively and quantitatively correct predictions in time lapses that are at least four orders of magnitude smaller than those required by embodied simulations and that they represent a useful tool for generalizing the dynamics of these highly stochastic, asynchronous, nonlinear systems, often outperforming intuitive reasoning. Finally, in addition to discussing subtle numerical effects, small prediction discrepancies, and difficulties in generating the mapping between different abstractions levels, we conclude the paper by reviewing the intrinsic limitations of the current modeling methodology and by proposing a few suggestions for future work

Development of cooperative behavioural model for autonomous multi-robots system deployed to underground mines

The number of disasters that occur in underground mine environments monthly all over the world cannot be ignored. Some of these disasters for instance are roof-falls; explosions, toxic gas inhalation, in-mine vehicle accidents, etc. can cause fatalities and/or disabilities. However, when such accidents happen during mining operations, rescuers find it difficult to respond to it immediately. This creates the necessity to bridge the gap between the lives of miners and the product acquired from the underground mines by using multi-robot systems. This thesis proposes an autonomous multi-robot cooperative behavioural model that can help to guide multi-robots in pre-entry safety inspection of underground mines. A hybrid swarm intelligent model termed, QLACS, that is based on Q-Learning (QL) and the Ant Colony System (ACS) is proposed to achieve cooperative behaviour in a MRS. The intelligent model was developed by harnessing the strengths of both QL and ACS algorithms. The ACS is used to optimize the routes used for each robot while the QL algorithm is used to enhance cooperation among the autonomous robots. The communication within the QLACS model for cooperative behavioural purposes is varied. The performance of the algorithms in terms of communication was evaluated by using a simulation approach. An investigation is conducted on the evaluation/scalability of the model using the different numbers of robots. Simulation results show that the methods proposed in this thesis achieved cooperative behaviour among the robots better than state-of-the-art or other common approaches. Using time and memory consumption as performance metrics, the results reveal that the proposed model can guide two, three and up to four robots to achieve efficient cooperative inspection behaviour in underground terrains

Neuro-Evolution for Emergent Specialization in Collective Behavior Systems

Eiben, A.E. [Promotor]Schut, M.C. [Copromotor

Comunidades Inteligentes para la Construcción y Gestión de Arquitecturas Optimizadas de de Sistemas Multiagente

[ES] El desarrollo de sistemas informáticos es una labor más o menos costosa en función de su complejidad. El hecho de poder reutilizar, parcial o totalmente, trozos de un sistema para otros desarrollos, implica una reducción en el tiempo empleado, una mayor facilidad de implementación y evita la redundancia de funcionalidades. Este planteamiento llevado a los sistemas multiagente ha de tener en cuenta las características propias de los agentes, para lo cual se requiere que la reutilización pueda llevarse a cabo a partir de pequeños subsistemas de agentes especializados con una organización establecida. Además, para explotar la capacidad de estos pequeños subsistemas de agentes es necesaria una arquitectura que tenga como finalidad la coordinación de los mismos, y que de forma modular y escalada, pueda desarrollarse para lograr objetivos de mayor complejidad. A lo largo de este trabajo se llevará a cabo un estudio de las características de los agentes y sistemas multiagente, asi como de las organizaciones humanas y su implementación a partir de las organizaciones virtuales, destacando su importancia y efectividad en el desarrollo actual de sistemas multiagente. Llegado este punto se realizará el diseño de SCODA (Distributed and Specialized Agent COmmunities), una nueva arquitectura modular para el desarrollo de sistemas multiagente. Mediante SCODA se permite el desarrollo de sistemas multiagente bajo una filosofía modular especializada, a través de la cual las funcionalidades del sistema puedan ir ampliándose, de forma escalada, en función de las necesidades. SCODA se compone de pequeños subsistemas de agentes, denominados Comunidades Inteligentes Especializadas (CIE), los cuales proveen las funcionalidades necesarias para resolver las necesidades requeridas a través de servicios distribuidos. Mediante estas CIE se permite una escalabilidad de los sistemas de forma que puedan ser reutilizadas en diferentes desarrollos, independientemente de su finalidad. La validación de esta arquitectura se realizará a partir de un caso de estudio centrado en tareas principalmente logísticas, debido a la variedad de situaciones que pueden darse en este tipo de ambientes. A partir de este caso de estudio se analizará y evaluará el comportamiento de la arquitectura y podrá llevarse a cabo su validaciónComputers systems development is more or less difficult task according to its complexity. The fact of being able to re-use, partially or completely, pieces of a system for other developments, involves a time reduction, a major implementation facility and avoids the functionalities redundancy. This aim applied to multiagent systems has to bear in mind the own characteristics of the agents, for which it is needed that the re-using could be carried out from small subsystems of specialized agents with an established organization. Also, to improve the capacity of these small subsystems of agents, is necessary an architecture, that has the objective to take the coordination of the same ones, and in a modular and scalable way, could develop to achieve aims with a major complexity. Throughout this work will be carried out a study of the characteristics of the agents and multiagent systems, as well as of human organizations and its deployment on virtual organizations, highlighting its importance and effectiveness in the current development of multiagent systems. From here it will be developed the design of SCODA (Distributed and Specialized Agent Communities), a new modular architecture for the development of multiagent systems. By means of SCODA, is allowed that multiagent systems could be developed from a specialized modular philosophy, across it the functionalities of the system can be extended in scaled form according to the objectives. SCODA is composed by small subsystems of agents named, Specialized Intelligent Communities (SCI), which provide the necessary functionalities to solve the objectives needed across distributed services. By means of these CIE, scalability of the systems is allowed, so that they could be re-used in different developments, independently of his purpose. SCODA is integrated by smaller subsystems of agents, called Intelligent Communities Specialized (SCI), which provide the functionality necessary to resolve the aims, using distributed services. These SCI allow a scalability of the systems so that can be reused in different developments, regardless of its purpose. The validation of this architecture will be realized through a case of study, focused on logistical tasks mainly due to the variety of situations that may arise in this kind of environments. From this case of study, we will analyze and assess the behaviour of the architecture and will carry out its validation

Emergent Specialization in Swarm Systems

Distributed learning is the learning process of multiple autonomous agents in a varying environment, where each agent has only partial information about the global task. In this paper, we investigate the influence of different reinforcement signals (local and global) and team diversity (homogeneous and heterogeneous agents) on the learned solutions. We compare the learned solutions with those obtained by systematic search in a simple case study in which pairs of agents have to collaborate in order to solve the task without any explicit communication. The results show that policies which allow teammates to specialize find an adequate diversity of the team and, in general, achieve similar or better performances than policies which force homogeneity. However, in this specific case study, the achieved team performances appear to be independent of the locality or globality of the reinforcement signal