Interdependent Scheduling Games

We propose a model of interdependent scheduling games in which each player controls a set of services that they schedule independently. A player is free to schedule his own services at any time; however, each of these services only begins to accrue reward for the player when all predecessor services, which may or may not be controlled by the same player, have been activated. This model, where players have interdependent services, is motivated by the problems faced in planning and coordinating large-scale infrastructures, e.g., restoring electricity and gas to residents after a natural disaster or providing medical care in a crisis when different agencies are responsible for the delivery of staff, equipment, and medicine. We undertake a game-theoretic analysis of this setting and in particular consider the issues of welfare maximization, computing best responses, Nash dynamics, and existence and computation of Nash equilibria.Comment: Accepted to IJCAI 201

Adaptive Load Balancing: A Study in Multi-Agent Learning

We study the process of multi-agent reinforcement learning in the context of load balancing in a distributed system, without use of either central coordination or explicit communication. We first define a precise framework in which to study adaptive load balancing, important features of which are its stochastic nature and the purely local information available to individual agents. Given this framework, we show illuminating results on the interplay between basic adaptive behavior parameters and their effect on system efficiency. We then investigate the properties of adaptive load balancing in heterogeneous populations, and address the issue of exploration vs. exploitation in that context. Finally, we show that naive use of communication may not improve, and might even harm system efficiency.Comment: See http://www.jair.org/ for any accompanying file

On Partially Controlled Multi-Agent Systems

Motivated by the control theoretic distinction between controllable and uncontrollable events, we distinguish between two types of agents within a multi-agent system: controllable agents, which are directly controlled by the system's designer, and uncontrollable agents, which are not under the designer's direct control. We refer to such systems as partially controlled multi-agent systems, and we investigate how one might influence the behavior of the uncontrolled agents through appropriate design of the controlled agents. In particular, we wish to understand which problems are naturally described in these terms, what methods can be applied to influence the uncontrollable agents, the effectiveness of such methods, and whether similar methods work across different domains. Using a game-theoretic framework, this paper studies the design of partially controlled multi-agent systems in two contexts: in one context, the uncontrollable agents are expected utility maximizers, while in the other they are reinforcement learners. We suggest different techniques for controlling agents' behavior in each domain, assess their success, and examine their relationship.Comment: See http://www.jair.org/ for any accompanying file

Compromise in negotiation: exploiting worth functions over states

AbstractPrevious work by G. Zlotkin and J.S. Rosenschein (1989, 1990, 1991, 1992) discussed interagent negotiation protocols. One of the main assumptions there was that the agents' goals remain fixed—the agents cannot relax their initial goals, which can be achieved only as a whole and cannot be partially achieved. A goal there was considered a formula that is either satisfied or not satisfied by a given state.We here present a more general approach to the negotiation problem in non-cooperative domains where agents' goals are not expressed as formulas, but rather as worth functions. An agent associates a particular value with each possible final state; this value reflects the degree of satisfaction the agent derives from being in that state.With this new definition of goal as worth function, an agreement may lead to a situation in which one or both goals are only partially achieved (i.e., agents may not reach their most desired state). We present a negotiation protocol that can be used in a general non-cooperative domain when worth functions are available. This multi-plan deal type allows agents to compromise over their degree of satisfaction, and (in parallel) to negotiate over the joint plan that will be implemented to reach the compromise final state. The ability to compromise often results in a better deal, enabling agents to increase their overall utility.Finally, we present more detailed examples of specific worth functions in various domains, and show how they are used in the negotiation process

Cooperative transportation scheduling : an application domain for DAI

A multiagent approach to designing the transportation domain is presented. The MARS system is described which models cooperative order scheduling within a society of shipping companies. We argue why Distributed Artificial Intelligence (DAI) offers suitable tools to deal with the hard problems in this domain. We present three important instances for DAI techniques that proved useful in the transportation application: cooperation among the agents, task decomposition and task allocation, and decentralised planning. An extension of the contract net protocol for task decomposition and task allocation is presented; we show that it can be used to obtain good initial solutions for complex resource allocation problems. By introducing global information based upon auction protocols, this initial solution can be improved significantly. We demonstrate that the auction mechanism used for schedule optimisation can also be used for implementing dynamic replanning. Experimental results are provided evaluating the performance of different scheduling strategies

Mechanisms for Automated Negotiation in State Oriented Domains

This paper lays part of the groundwork for a domain theory of negotiation, that is, a way of classifying interactions so that it is clear, given a domain, which negotiation mechanisms and strategies are appropriate. We define State Oriented Domains, a general category of interaction. Necessary and sufficient conditions for cooperation are outlined. We use the notion of worth in an altered definition of utility, thus enabling agreements in a wider class of joint-goal reachable situations. An approach is offered for conflict resolution, and it is shown that even in a conflict situation, partial cooperative steps can be taken by interacting agents (that is, agents in fundamental conflict might still agree to cooperate up to a certain point). A Unified Negotiation Protocol (UNP) is developed that can be used in all types of encounters. It is shown that in certain borderline cooperative situations, a partial cooperative agreement (i.e., one that does not achieve all agents' goals) might be preferred by all agents, even though there exists a rational agreement that would achieve all their goals. Finally, we analyze cases where agents have incomplete information on the goals and worth of other agents. First we consider the case where agents' goals are private information, and we analyze what goal declaration strategies the agents might adopt to increase their utility. Then, we consider the situation where the agents' goals (and therefore stand-alone costs) are common knowledge, but the worth they attach to their goals is private information. We introduce two mechanisms, one 'strict', the other 'tolerant', and analyze their affects on the stability and efficiency of negotiation outcomes.Comment: See http://www.jair.org/ for any accompanying file

Reflective mobile middleware for context-aware applications

The increasing popularity of mobile devices, such as mobile phones and personal digital assistants, and advances in wireless networking technologies, are enabling new classes of applications that present challenging problems to application designers. Applications have to be aware of, and adapt to, variations in the execution context, such as fluctuating network bandwidth and decreasing battery power, in order to deliver a good quality of service to their users. We argue that building applications directly on top of the network operating system would be extremely tedious and error-prone, as application developers would have to deal with these issues explicitly, and would consequently be distracted from the actual requirements of the application they are building. Rather, a middleware layered between the network operating system and the application should provide application developers with abstractions and mechanisms to deal with them. We investigate the principle of reflection and demonstrate how it can be used to support context-awareness and dynamic adaptation to context changes. We offer application engineers an abstraction of middleware as a dynamically customisable service provider, where each service can be delivered using different policies when requested in different contexts. Based on this abstraction, current middleware behaviour, with respect to a particular application, is reified in an application profile, and made accessible to the application for run-time inspection and adaptation. Applications can use the meta-interface that the middleware provides to change the information encoded in their profile, thus tailoring middleware behaviour to the user's needs. However, while doing so, conflicts may arise; different users may have different quality-of-service needs, and applications, in an attempt to full these needs, may customise middleware behaviour in conflicting ways. These conflicts have to be resolved in order to allow applications to come to an agreement, and thus be able to engage successful collaborations. We demonstrate how microeconomic techniques can be used to treat these kinds of conflicts. We offer an abstraction of the mobile setting as an economy, where applications compete to have a service delivered according to their quality-of-service needs. We have designed a mechanism where middleware plays the role of the auctioneer, collecting bids from the applications and delivering the service using the policy that maximises social welfare; that is, the one that delivers, on average, the best quality-of-service. We formalise the principles discussed above, namely reflection to support context-awareness and microeconomic techniques to support conflict resolution. To demonstrate their effectiveness in fostering the development of context-aware applications, we discuss a middleware architecture and implementation (CARISMA) that embed these principles, and report on performance and usability results obtained during a thorough evaluation stage