Coordinating decentralized learning and conflict resolution across agent boundaries

It is crucial for embedded systems to adapt to the dynamics of open environments. This adaptation process becomes especially challenging in the context of multiagent systems because of scalability, partial information accessibility and complex interaction of agents. It is a challenge for agents to learn good policies, when they need to plan and coordinate in uncertain, dynamic environments, especially when they have large state spaces. It is also critical for agents operating in a multiagent system (MAS) to resolve conflicts among the learned policies of different agents, since such conflicts may have detrimental influence on the overall performance. The focus of this research is to use a reinforcement learning based local optimization algorithm within each agent to learn multiagent policies in a decentralized fashion. These policies will allow each agent to adapt to changes in environmental conditions while reorganizing the underlying multiagent network when needed. The research takes an adaptive approach to resolving conflicts that can arise between locally optimal agent policies. First an algorithm that uses heuristic rules to locally resolve simple conflicts is presented. When the environment is more dynamic and uncertain, a mediator-based mechanism to resolve more complicated conflicts and selectively expand the agents' state space during the learning process is harnessed. For scenarios where mediator-based mechanisms with partially global views are ineffective, a more rigorous approach for global conflict resolution that synthesizes multiagent reinforcement learning (MARL) and distributed constraint optimization (DCOP) is developed. These mechanisms are evaluated in the context of a multiagent tornado tracking application called NetRads. Empirical results show that these mechanisms significantly improve the performance of the tornado tracking network for a variety of weather scenarios. The major contributions of this work are: a state of the art decentralized learning approach that supports agent interactions and reorganizes the underlying network when needed; the use of abstract classes of scenarios/states/actions that efficiently manages the exploration of the search space; novel conflict resolution algorithms of increasing complexity that use heuristic rules, sophisticated automated negotiation mechanisms and distributed constraint optimization methods respectively; and finally, a rigorous study of the interplay between two popular theories used to solve multiagent problems, namely decentralized Markov decision processes and distributed constraint optimization

Using mobility and exception handling to achieve mobile agents that survive server crash failures

Mobile agent technology, when designed and used effectively, can minimize bandwidth consumption and autonomously provide a snapshot of the current context of a distributed system. Protecting mobile agents from server crashes is a challenging issue, since developers normally have no control over remote servers. Server crash failures can leave replicas, instable storage, unavailable for an unknown time period. Furthermore, few systems have considered the need for using a fault tolerant protocol among a group of collaborating mobile agents. This thesis uses exception handling to protect mobile agents from server crash failures. An exception model is proposed for mobile agents and two exception handler designs are investigated. The first exists at the server that created the mobile agent and uses a timeout mechanism. The second, the mobile shadow scheme, migrates with the mobile agent and operates at the previous server visited by the mobile agent. A case study application has been developed to compare the performance of the two exception handler designs. Performance results demonstrate that although the second design is slower it offers the smaller trip time when handling a server crash. Furthermore, no modification of the server environment is necessary. This thesis shows that the mobile shadow exception handling scheme reduces complexity for a group of mobile agents to survive server crashes. The scheme deploys a replica that monitors the server occupied by the master, at each stage of the itinerary. The replica exists at the previous server visited in the itinerary. Consequently, each group member is a single fault tolerant entity with respect to server crash failures. Other schemes introduce greater complexity and performance overheads since, for each stage of the itinerary, a group of replicas is sent to servers that offer an equivalent service. In addition, future research is established for fault tolerance in groups of collaborating mobile agents

MAGDA: A Mobile Agent based Grid Architecture

Mobile agents mean both a technology and a programming paradigm. They allow for a flexible approach which can alleviate a number of issues present in distributed and Grid-based systems, by means of features such as migration, cloning, messaging and other provided mechanisms. In this paper we describe an architecture (MAGDA – Mobile Agent based Grid Architecture) we have designed and we are currently developing to support programming and execution of mobile agent based application upon Grid systems

Blockchain Solutions for Multi-Agent Robotic Systems: Related Work and Open Questions

The possibilities of decentralization and immutability make blockchain probably one of the most breakthrough and promising technological innovations in recent years. This paper presents an overview, analysis, and classification of possible blockchain solutions for practical tasks facing multi-agent robotic systems. The paper discusses blockchain-based applications that demonstrate how distributed ledger can be used to extend the existing number of research platforms and libraries for multi-agent robotic systems.Comment: 5 pages, FRUCT-2019 conference pape

Operational design co-ordination : an agent based approach

Operational design co-ordination has been identified as the basis for an approach to engineering design management that is more comprehensive than those that currently exist. As such, an integrated and holistic approach to operational design co-ordination has been developed that enables design to be managed in a coherent, appropriate and timely manner. Furthermore, the approach has been implemented within an agent-based software system, called the Design Co-ordination System, which has been applied to an industrial case study involving the computational design analysis of turbine blades. This application demonstrates that managing and adjusting in real-time in an operationally co-ordinated manner enables reductions in the time taken to complete the turbine blade design process to be achieved

EDI and intelligent agents integration to manage food chains

Electronic Data Interchange (EDI) is a type of inter-organizational information system, which permits the automatic and structured communication of data between organizations. Although EDI is used for internal communication, its main application is in facilitating closer collaboration between organizational entities, e.g. suppliers, credit institutions, and transportation carriers. This study illustrates how agent technology can be used to solve real food supply chain inefficiencies and optimise the logistics network. For instance, we explain how agribusiness companies can use agent technology in association with EDI to collect data from retailers, group them into meaningful categories, and then perform different functions. As a result, the distribution chain can be managed more efficiently. Intelligent agents also make available timely data to inventory management resulting in reducing stocks and tied capital. Intelligent agents are adoptive to changes so they are valuable in a dynamic environment where new products or partners have entered into the supply chain. This flexibility gives agent technology a relative advantage which, for pioneer companies, can be a competitive advantage. The study concludes with recommendations and directions for further research