Agent teamwork and reorganisation: exploring self-awareness in dynamic situations

We propose attributes that are needed in sophisticated agent teams capable of working to manage an evolving disaster. Such agent teams need to be dynamically formed and ca- pable of adaptive reorganization as the demands and com- plexity of the situation evolve. The agents need to have self- awareness of their own roles, responsibilities and capabilities and be aware of their relationships with others in the team. Each agent is not only empowered to act autonomously to- ward realizing their goals, agents are also able to negotiate to change roles as a situation changes, if reorganization is required or perceived to be in the team interest. The hierar- chical 'position' of an agent and the 'relationships' between agents govern the authority and obligations that an agent adopts. Such sophisticated agents might work in a collabora- tive team with people to self-organize and manage a critical incident such as a bush-¯re. We are planning to implement a team of agents to interface with a bush-¯re simulation, working with people in real time, to test our architecture.E

Optimal and Approximate Q-value Functions for Decentralized POMDPs

Decision-theoretic planning is a popular approach to sequential decision making problems, because it treats uncertainty in sensing and acting in a principled way. In single-agent frameworks like MDPs and POMDPs, planning can be carried out by resorting to Q-value functions: an optimal Q-value function Q* is computed in a recursive manner by dynamic programming, and then an optimal policy is extracted from Q*. In this paper we study whether similar Q-value functions can be defined for decentralized POMDP models (Dec-POMDPs), and how policies can be extracted from such value functions. We define two forms of the optimal Q-value function for Dec-POMDPs: one that gives a normative description as the Q-value function of an optimal pure joint policy and another one that is sequentially rational and thus gives a recipe for computation. This computation, however, is infeasible for all but the smallest problems. Therefore, we analyze various approximate Q-value functions that allow for efficient computation. We describe how they relate, and we prove that they all provide an upper bound to the optimal Q-value function Q*. Finally, unifying some previous approaches for solving Dec-POMDPs, we describe a family of algorithms for extracting policies from such Q-value functions, and perform an experimental evaluation on existing test problems, including a new firefighting benchmark problem

Team Formation for Reformation in Multiagent Domains like RoboCupRescue

Team formation, i.e., allocating agents to roles within a team or subteams of a team, and the reorganization of a team upon team member failure or arrival of new tasks are critical aspects of teamwork. They are very important issues in RoboCupRescue where many tasks need to be done jointly. While empirical comparisons (e.g., in a competition setting as in RoboCup) are useful, we need a quantitative analysis beyond the competition -- to understand the strengths and limitations of different approaches, and their tradeos as we scale up the domain or change domain properties. To this end, we need to provide complexity/optimality tradeoffs, which have been lacking not only in RoboCup but in the multiagent field in general

Collaboration with agents in VR environments

Virtual reality is gaining on importance in many fields – scientific simulation, training, therapy and also more and more in entertainment. All these applications require the human user to interact with virtual worlds inhabited by intelligent characters and to solve simulated or real problems. This thesis will present an integrated approach to simulated problem solving in virtual reality environments, with the emphasis on teamwork and the ability to control the simulations. A simulation framework satisfying these goals will be presented. A unified approach to the representation of semantic information in virtual environments based on predicate calculus will be introduced, including the representation of the world state, action semantics and basic axioms holding in the simulated world. Afterwards, the focus will be on the collaboration model based on task delegation and facilitator-centric architecture. A simple but efficient facilitator design will be presented. The issues of the collaborative problem solving will be examined. A new technique using propositional (STRIPS-like) planning with delegated actions and object-specific planning will be described. A control technique for virtual characters/objects will be detailed, enabling run-time exchange of control and control sharing over a virtual entity between multiple autonomous agents and/or human users. Finally, a set of case studies will be shown, illustrating the possible applications of the techniques developed and described in this dissertation