Applications of DEC-MDPs in multi-robot systems

International audienceOptimizing the operation of cooperative multi-robot systems that can cooperatively act in large and complex environments has become an important focal area of research. This issue is motivated by many applications involving a set of cooperative robots that have to decide in a decentralized way how to execute a large set of tasks in partially observable and uncertain environments. Such decision problems are encountered while developing exploration rovers, teams of patrolling robots, rescue-robot colonies, mine-clearance robots, et cetera.In this chapter, we introduce problematics related to the decentralized control of multi-robot systems. We rst describe some applicative domains and review the main characteristics of the decision problems the robots must deal with. Then, we review some existing approaches to solve problems of multiagent decen- tralized control in stochastic environments. We present the Decentralized Markov Decision Processes and discuss their applicability to real-world multi-robot applications. Then, we introduce OC-DEC-MDPs and 2V-DEC-MDPs which have been developed to increase the applicability of DEC-MDPs

Reasoning about Goal-Plan Trees in Autonomous Agents: Development of Petri net and Constraint-Based Approaches with Resulting Performance Comparisons

Multi-agent systems and autonomous agents are becoming increasingly important in current computing technology. In many applications, the agents are often asked to achieve multiple goals individually or within teams where the distribution of these goals may be negotiated among the agents. It is expected that agents should be capable of working towards achieving all its currently adopted goals concurrently. However, in doing so, the goals can interact both constructively and destructively with each other, so a rational agent must be able to reason about these interactions and any other constraints that may be imposed on them, such as the limited availability of resources that could affect their ability to achieve all adopted goals when pursuing them concurrently. Currently, agent development languages require the developer to manually identify and handle these circumstances. In this thesis, we develop two approaches for reasoning about the interactions between the goals of an individual agent. The first of these employs Petri nets to represent and reason about the goals, while the second uses constraint satisfaction techniques to find efficient ways of achieving the goals. Three types of reasoning are incorporated into these models: reasoning about consumable resources where the availability of the resources is limited; the constructive interaction of goals whereby a single plan can be used to achieve multiple goals; and the interleaving of steps for achieving different goals that could cause one or more goals to fail. Experimental evaluation of the two approaches under various different circumstances highlights the benefits of the reasoning developed here whilst also identifying areas where one approach provides better results than the other. This can then be applied to suggest the underlying technique used to implement the reasoning that the agent may want to employ based on the goals it has been assigned

Distributed Method Selection and Dispatching of Contingent, Temporally Flexible Plans

SM thesisMany applications of autonomous agents require groups to work in tight coordination. To be dependable, these groups must plan, carry out and adapt their activities in a way that is robust to failure and to uncertainty. Previous work developed contingent, temporally flexible plans. These plans provide robustness to uncertain activity durations, through flexible timing constraints, and robustness to plan failure, through alternate approaches to achieving a task. Robust execution of contingent, temporally flexible plans consists of two phases. First, in the plan extraction phase, the executive chooses between the functionally redundant methods in the plan to select an execution sequence that satisfies the temporal bounds in the plan. Second, in the plan execution phase, the executive dispatches the plan, using the temporal flexibility to schedule activities dynamically.Previous contingent plan execution systems use a centralized architecture in which a single agent conducts planning for the entire group. This can result in a communication bottleneck at the time when plan activities are passed to the other agents for execution, and state information is returned. Likewise, a computation bottleneck may also occur because a single agent conducts all processing.This thesis introduces a robust, distributed executive for temporally flexible plans, called Distributed-Kirk, or D-Kirk. To execute a plan, D-Kirk first distributes the plan between the participating agents, by creating a hierarchical ad-hoc network and by mapping the plan onto this hierarchy. Second, the plan is reformulated using a distributed, parallel algorithm into a form amenable to fast dispatching. Finally, the plan is dispatched in a distributed fashion.We then extend the D-Kirk distributed executive to handle contingent plans. Contingent plans are encoded as Temporal Plan Networks (TPNs), which use a non-deterministic choice operator to compose temporally flexible plan fragments into a nested hierarchy of contingencies. A temporally consistent plan is extracted from the TPN using a distributed, parallel algorithm that exploits the structure of the TPN.At all stages of D-Kirk, the communication load is spread over all agents, thus eliminating the communication bottleneck. In particular, D-Kirk reduces the peak communication complexity of the plan execution phase by a factor of O(A/e'), where e' is the number of edges per node in the dispatchable plan, determined by the branching factor of the input plan, and A is the number of agents involved in executing the plan.In addition, the distributed algorithms employed by D-Kirk reduce the computational load on each agent and provide opportunities for parallel processing, thus increasing efficiency. In particular, D-Kirk reduces the average computational complexity of plan dispatching from O(eN^3) in the centralized case, to typical values of O(eN^2) per node and O(eN^3/A) per agent in the distributed case, where N is the number of nodes in the plan and e is the number of edges per node in the input plan.Both of the above results were confirmed empirically using a C++ implementation of D-Kirk on a set of parameterized input plans. The D-Kirk implementation was also tested in a realistic application where it was used to control a pair of robotic manipulators involved in a cooperative assembly task

The exploration of unknown environments by affective agents

Tese de doutoramento em Engenharia Informática apresentada à Fac. de Ciências e Tecnologia de CoimbraIn this thesis, we study the problem of the exploration of unknown environments populated with entities by affective autonomous agents. The goal of these agents is twofold: (i) the acquisition of maps of the environment – metric maps – to be stored in memory, where the cells occupied by the entities that populate that environment are represented; (ii) the construction of models of those entities. We examine this problem through simulations because of the various advantages this approach offers, mainly efficiency, more control, and easy focus of the research. Furthermore, the simulation approach can be used because the simplifications that we made do not influence the value of the results. With this end, we have developed a framework to build multi-agent systems comprising affective agents and then, based on this platform, we developed an application for the exploration of unknown environments. This application is a simulated multi-agent environment in which, in addition to inanimate agents (objects), there are agents interacting in a simple way, whose goal is to explore the environment. By relying on an affective component plus ideas from the Belief-Desire-Intention model, our approach to building artificial agents is that of assigning agents mentalistic qualities such as feelings, basic desires, memory/beliefs, desires/goals, and intentions. The inclusion of affect in the agent architecture is supported by the psychological and neuroscience research over the past decades which suggests that emotions and, in general, motivations play a critical role in decision-making, action, and reasoning, by influencing a variety of cognitive processes (e.g., attention, perception, planning, etc.). Reflecting the primacy of those mentalistic qualities, the architecture of an agent includes the following modules: sensors, memory/beliefs (for entities - which comprises both analogical and propositional knowledge representations -, plans, and maps of the environment), desires/goals, intentions, basic desires (basic motivations/motives), feelings, and reasoning. The key components that determine the exhibition of the exploratory behaviour in an agent are the kind of basic desires, feelings, goals and plans with which the agent is equipped. Based on solid, psychological experimental evidence, an agent is equipped in advance with the basic desires for minimal hunger, maximal information gain (maximal reduction of curiosity), and maximal surprise, as well as with the correspondent feelings of hunger, curiosity and surprise. Each one of those basic desires drives the agent to reduce or to maximize a particular feeling. The desire for minimal hunger, maximal information gain and maximal surprise directs the agent, respectively, to reduce the feeling of hunger, to reduce the feeling of curiosity (by maximizing information gain) and to maximize the feeling of surprise. The desire to reduce curiosity does not mean that the agent dislike curiosity. Instead, it means the agent desires selecting actions whose execution maximizes the reduction of curiosity, i.e., actions that are preceded by maximal levels of curiosity and followed by minimal levels of curiosity, which corresponds to maximize information gain. The intensity of these feelings is, therefore, important to compute the degree of satisfaction of the basic desires. For the basic desires of minimal hunger and maximal surprise it is given by the expected intensities of the feelings of hunger and surprise, respectively, after performing an action, while for the desire of maximal information gain it is given by the intensity of the feeling of curiosity before performing the action (this is the expected information gain). The memory of an agent is setup with goals and decision-theoretic, hierarchical task-network plans for visiting entities that populate the environment, regions of the environment, and for going to places where the agent can recharge its battery. New goals are generated for each unvisited entity of the environment, for each place in the frontier of the explored area, and for recharging battery, by adapting past goals and plans to the current world state computed based on sensorial information and on the generation of expectations and assumptions for the gaps in the environment information provided by the sensors. These new goals and respective plans are then ranked according to their Expected Utility which reflects the positive and negative relevance for the basic desires of their accomplishment. The first one, i.e., the one with highest Expected Utility is taken as an intention. Besides evaluating the computational model of surprise, we experimentally investigated through simulations the following issues: the role of the exploration strategy (role of surprise, curiosity, and hunger), environment complexity, and amplitude of the visual field on the performance of the exploration of environments populated with entities; the role of the size or, to some extent, of the diversity of the memory of entities, and environment complexity on map-building by exploitation. The main results show that: the computational model of surprise is a satisfactory model of human surprise; the exploration of unknown environments populated with entities can be robustly and efficiently performed by affective agents (the strategies that rely on hunger combined or not with curiosity or surprise outperform significantly the others, being strong contenders to the classical strategy based on entropy and cost)

An Integrated System for Multi-Rover Scientific Exploration

This paper describes an integrated system for coordinating multiple rover behavior with the overall goal of collecting planetary surface data. The MultiRover Integrated Science Understanding System combines concepts from machine learning with planning and scheduling to perform autonomous scientic exploration by cooperating rovers. The integrated system utilizes a novel machine learning clustering component to analyze science data and direct new science activities. A planning and scheduling system is employed to generate rover plans for achieving science goals and to coordinate activities among rovers. We describe each of these components and discuss some of the key integration issues that arose during development and in- uenced both system design and performance