Predicting the expected behavior of agents that learn about agents: the CLRI framework

We describe a framework and equations used to model and predict the behavior of multi-agent systems (MASs) with learning agents. A difference equation is used for calculating the progression of an agent's error in its decision function, thereby telling us how the agent is expected to fare in the MAS. The equation relies on parameters which capture the agent's learning abilities, such as its change rate, learning rate and retention rate, as well as relevant aspects of the MAS such as the impact that agents have on each other. We validate the framework with experimental results using reinforcement learning agents in a market system, as well as with other experimental results gathered from the AI literature. Finally, we use PAC-theory to show how to calculate bounds on the values of the learning parameters

08461 Abstracts Collection -- Planning in Multiagent Systems

From the 9th of November to the 14th of November 2008 the Dagstuhl Seminar 08461 \u27`Planning in Multiagent Systems\u27\u27 was held in Schloss Dagstuhl~--~Leibniz Center for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

Improving Learning Performance by Applying Economic Knowledge

Digital information economies require information goods producers to learn how to position themselves within a potentially vast product space. Further, the topography of this space is often nonstationary, due to the interactive dynamics of multiple producers changing their position as they try to learn the distribution of consumer preferences and other features of the problem's economic structure. This presents a producer or its agent with a difficult learning problem: how to locate profitable niches in a very large space. In this paper, we present a model of an information goods duopoly and show that, under complete information, producers would prefer not to compete, instead acting as local monopolists and targeting separate niches in the consumer population. However, when producers have no information about the problem they are solving, it can be quite difficult for them to converge on this solution. We show how a modest amount of economic knowledge about the problem can make it much easier, either by reducing the search space, starting in a useful area of the space, or introducing a gradient. These experiments support the hypothesis that a producer using some knowledge of a problem's (economic) structure can outperform a producer that is performing a naive, knowledge-free form of learning.

A Formal Study of Distributed Meeting Scheduling

Automating routine organizational tasks, such as meeting scheduling, requires a careful balance between the individual (respecting his or her privacy and personal preferences) and the organization (making efficient use of time and other resources). We argue that meeting scheduling is an inherently distributed process, and that negotiating over meetings can be viewed as a distributed search process. Keeping the process tractable requires introducing heuristics to guide distributed schedulers' decisions about what information to exchange and whether or not to propose the same tentative time for several meetings. While we have intuitions about how such heuristics could affect scheduling performance and efficiency, verifying these intuitions requires a more formal model of the meeting schedule problem and process. We present our preliminary work toward this goal, as well as experimental results that validate some of the predictions of our formal model. We also investigate scheduling in overconstrained situations, namely, scheduling of high priority meetings at short notice, which requires cancellation and rescheduling of previously scheduled meetings. Our model provides a springboard into deeper investigations of important issues in distributed artificial intelligence as well, and we outline our ongoing work in this direction.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/42829/1/10726_2004_Article_153020.pd

Congregation Formation in Multiagent Systems

We present congregating both as a metaphor for describing and modeling multiagent systems (MAS) and as a means for reducing coordination costs in large-scale MAS. When agents must search for other agents to interact with, congregations provide a way for agents to bias this search towards groups of agents that have tended to produce successful interactions in the past. This causes each agent's search problem to scale with the size of a congregation rather than the size of the population as a whole. In this paper, we present a formal model of a congregation and then apply Vidal and Durfee's CLRI framework [24] to the congregating problem. We apply congregating to the affinity group domain, and show that if agents are unable to describe congregations to each other, the problem of forming optimal congregations grows exponentially with the number of agents. The introduction of labelers provides a means of coordinating agent decisions, thereby reducing the problem's complexity. We then show how a structured label space can be exploited to simplify the labeler's decision problem and make the congregating problem linear in the number of labels. We then present experimental evidence demonstrating how congregating can be used to reduce agents' search costs, thereby allowing the system to scale up. We conclude with a comparison to other methods for coordinating multiagent behavior, particularly teams and coalitions.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/44028/1/10458_2004_Article_5124857.pd

Rational Coordination in Multi-Agent Environments

We adopt the decision-theoretic principle of expected utility maximization as a paradigm for designing autonomous rational agents, and present a framework that uses this paradigm to determine the choice of coordinated action. We endow an agent with a specialized representation that captures the agent's knowledge about the environment and about the other agents, including its knowledge about their states of knowledge, which can include what they know about the other agents, and so on. This reciprocity leads to a recursive nesting of models. Our framework puts forth a representation for the recursive models and, under the assumption that the nesting of models is finite, uses dynamic programming to solve this representation for the agent's rational choice of action. Using a decision-theoretic approach, our work addresses concerns of agent decision-making about coordinated action in unpredictable situations, without imposing upon agents pre-designed prescriptions, or protocols, about standard rules of interaction. We implemented our method in a number of domains and we show results of coordination among our automated agents, among human-controlled agents, and among our agents coordinating with human-controlled agents.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/44002/1/10458_2004_Article_272540.pd

Rational Communication in Multi-Agent Environments

We address the issue of rational communicative behavior among autonomous self-interested agents that have to make decisions as to what to communicate, to whom, and how. Following decision theory, we postulate that a rational speaker should design a speech act so as to optimize the benefit it obtains as the result of the interaction. We quantify the gain in the quality of interaction in terms of the expected utility, and we present a framework that allows an agent to compute the expected utilities of various communicative actions. Our framework uses the Recursive Modeling Method as the specialized representation used for decision-making in a multi-agent environment. This representation includes information about the agent's state of knowledge, including the agent's preferences, abilities and beliefs about the world, as well as the beliefs the agent has about the other agents, the beliefs it has about the other agents' beliefs, and so on. Decision-theoretic pragmatics of a communicative act can be then defined as the transformation the act induces on the agent's state of knowledge about its decision-making situation. This transformation leads to a change in the quality of interaction, expressed in terms of the expected utilities of the agent's best actions before and after the communicative act. We analyze decision-theoretic pragmatics of a number of important kinds of communicative acts and investigate their expected utilities using examples. Finally, we report on the agreement between our method of message selection and messages that human subjects choose in various circumstances, and show an implementation and experimental validation of our framework in a simulated multi-agent environment.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/44016/1/10458_2004_Article_350961.pd