A Multi-Agent Approach for Designing Next Generation of Air Traffic Systems

This work was funded by Spanish Ministry of Economy and Competitiveness under grant TEC2011-28626 C01-C02, and by the Government of Madrid under grant S2009/TIC-1485 (CONTEXTS)

Strategic Structural Reorganization in Multi-agent Systems Inspired by Social Organization Theory

Autonomic systems, capable of adaptive behavior, are envisioned as a solution for maintaining large, complex, real-time computing systems that are situated in dynamic and open environments. These systems are subject to uncertainties in their perceptual, computational, and communication loads. As a result, the individual system components find the need to cooperate with each other to acquire more information and accomplish complex tasks. Critical to the effective performance of these systems, is the effectiveness of communication and coordination methods. In many practical applications of distributed and multi-agent systems, the problem of communication and coordination becomes even more complicated because of the geographic disparity of tasks and/or agents that are performing the tasks. Experience with even small systems has shown that lack of an effective communication and coordination strategy leads the system to no-answer, or sub-optimal answer situations. To address this problem, many large-scale systems employ an additional layer of structuring, known as organizational structure, which governs assignment of roles to individual agents, existence of relations between the agents , and any authority structures in between. Applying different organizational structures to the same problem will lead to different performance characteristics. As the system and environment conditions change, it becomes important to reorganize to a more effective organization. Due to the costs associated with reorganization, finding a balance in how often or when a reorganization is performed becomes necessary. In multi-agent systems community, not a lot of attention has been paid to reorganizing a system to a different organizational structure. Most systems reorganize within the same structure, for example reorganizing in a hierarchy by changing the width or depth of the hierarchy. To approach this problem, we looked into adaptation of concepts and theories from social organization theory. In particular, we got insights from Schwaninger's model of Intelligent Human Organizations. We introduced a strategic reorganization model which enables the system to reorganize to a different type of organizational structure at run time. The proposed model employs different levels of organizational control for making organizational change decisions. We study the performance trade-offs and the efficacy of the proposed approach by running experiments using two instances of cooperative distributed problem solving applications. The results indicate that the proposed reorganization model results in performance improvements when task complexity increases

A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity

The key challenge in multiagent learning is learning a best response to the behaviour of other agents, which may be non-stationary: if the other agents adapt their strategy as well, the learning target moves. Disparate streams of research have approached non-stationarity from several angles, which make a variety of implicit assumptions that make it hard to keep an overview of the state of the art and to validate the innovation and significance of new works. This survey presents a coherent overview of work that addresses opponent-induced non-stationarity with tools from game theory, reinforcement learning and multi-armed bandits. Further, we reflect on the principle approaches how algorithms model and cope with this non-stationarity, arriving at a new framework and five categories (in increasing order of sophistication): ignore, forget, respond to target models, learn models, and theory of mind. A wide range of state-of-the-art algorithms is classified into a taxonomy, using these categories and key characteristics of the environment (e.g., observability) and adaptation behaviour of the opponents (e.g., smooth, abrupt). To clarify even further we present illustrative variations of one domain, contrasting the strengths and limitations of each category. Finally, we discuss in which environments the different approaches yield most merit, and point to promising avenues of future research

A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity

The key challenge in multiagent learning is learning a best response to the behaviour of other agents, which may be non-stationary: if the other agents adapt their strategy as well, the learning target moves. Disparate streams of research have approached non-stationarity from several angles, which make a variety of implicit assumptions that make it hard to keep an overview of the state of the art and to validate the innovation and significance of new works. This survey presents a coherent overview of work that addresses opponent-induced non-stationarity with tools from game theory, reinforcement learning and multi-armed bandits. Further, we reflect on the principle approaches how algorithms model and cope with this non-stationarity, arriving at a new framework and five categories (in increasing order of sophistication): ignore, forget, respond to target models, learn models, and theory of mind. A wide range of state-of-the-art algorithms is classified into a taxonomy, using these categories and key characteristics of the environment (e.g., observability) and adaptation behaviour of the opponents (e.g., smooth, abrupt). To clarify even further we present illustrative variations of one domain, contrasting the strengths and limitations of each category. Finally, we discuss in which environments the different approaches yield most merit, and point to promising avenues of future research

Distributed decision-making in electric power system transmission maintenance scheduling using Multi-Agent Systems (MAS)

In this work, motivated by the need to coordinate transmission maintenance scheduling among a multiplicity of self-interested entities in restructured power industry, a distributed decision support framework based on multiagent negotiation systems (MANS) is developed. An innovative risk-based transmission maintenance optimization procedure is introduced. Several models for linking condition monitoring information to the equipment\u27s instantaneous failure probability are presented, which enable quantitative evaluation of the effectiveness of maintenance activities in terms of system cumulative risk reduction. Methodologies of statistical processing, equipment deterioration evaluation and time-dependent failure probability calculation are also described. A novel framework capable of facilitating distributed decision-making through multiagent negotiation is developed. A multiagent negotiation model is developed and illustrated that accounts for uncertainty and enables social rationality. Some issues of multiagent negotiation convergence and scalability are discussed. The relationships between agent-based negotiation and auction systems are also identified. A four-step MAS design methodology for constructing multiagent systems for power system applications is presented. A generic multiagent negotiation system, capable of inter-agent communication and distributed decision support through inter-agent negotiations, is implemented. A multiagent system framework for facilitating the automated integration of condition monitoring information and maintenance scheduling for power transformers is developed. Simulations of multiagent negotiation-based maintenance scheduling among several independent utilities are provided. It is shown to be a viable alternative solution paradigm to the traditional centralized optimization approach in today\u27s deregulated environment. This multiagent system framework not only facilitates the decision-making among competing power system entities, but also provides a tool to use in studying competitive industry relative to monopolistic industry