An Agent-Based Simulation for Investigating the Impact of Stereotypes on Task-Oriented Group Formation

In this paper, we introduce an agent-based simulation for investigating the impact of social factors on the formation and evolution of task-oriented groups. Task-oriented groups are created explicitly to perform a task, and all members derive benefits from task completion. However, even in cases when all group members act in a way that is locally optimal for task completion, social forces that have mild effects on choice of associates can have a measurable impact on task completion performance. In this paper, we show how our simulation can be used to model the impact of stereotypes on group formation. In our simulation, stereotypes are based on observable features, learned from prior experience, and only affect an agent\u27s link formation preferences. Even without assuming stereotypes affect the agents\u27 willingness or ability to complete tasks, the long-term modifications that stereotypes have on the agents\u27 social network impair the agents\u27 ability to form groups with sufficient diversity of skills, as compared to agents who form links randomly. An interesting finding is that this effect holds even in cases where stereotype preference and skill existence are completely uncorrelated. © 2011 Springer-Verlag Berlin Heidelberg

Identifying Influential Agents In Social Systems

This dissertation addresses the problem of influence maximization in social networks. In- fluence maximization is applicable to many types of real-world problems, including modeling contagion, technology adoption, and viral marketing. Here we examine an advertisement domain in which the overarching goal is to find the influential nodes in a social network, based on the network structure and the interactions, as targets of advertisement. The assumption is that advertisement budget limits prevent us from sending the advertisement to everybody in the network. Therefore, a wise selection of the people can be beneficial in increasing the product adoption. To model these social systems, agent-based modeling, a powerful tool for the study of phenomena that are difficult to observe within the confines of the laboratory, is used. To analyze marketing scenarios, this dissertation proposes a new method for propagating information through a social system and demonstrates how it can be used to develop a product advertisement strategy in a simulated market. We consider the desire of agents toward purchasing an item as a random variable and solve the influence maximization problem in steady state using an optimization method to assign the advertisement of available products to appropriate messenger agents. Our market simulation 1) accounts for the effects of group membership on agent attitudes 2) has a network structure that is similar to realistic human systems 3) models inter-product preference correlations that can be learned from market data. The results on synthetic data show that this method is significantly better than network analysis methods based on centrality measures. The optimized influence maximization (OIM) described above, has some limitations. For instance, it relies on a global estimation of the interaction among agents in the network, rendering it incapable of handling large networks. Although OIM is capable of finding the influential nodes in the social network in an optimized way and targeting them for advertising, in large networks, performing the matrix operations required to find the optimized solution is intractable. To overcome this limitation, we then propose a hierarchical influence maximization (HIM) iii algorithm for scaling influence maximization to larger networks. In the hierarchical method the network is partitioned into multiple smaller networks that can be solved exactly with optimization techniques, assuming a generalized IC model, to identify a candidate set of seed nodes. The candidate nodes are used to create a distance-preserving abstract version of the network that maintains an aggregate influence model between partitions. The budget limitation for the advertising dictates the algorithm’s stopping point. On synthetic datasets, we show that our method comes close to the optimal node selection, at substantially lower runtime costs. We present results from applying the HIM algorithm to real-world datasets collected from social media sites with large numbers of users (Epinions, SlashDot, and WikiVote) and compare it with two benchmarks, PMIA and DegreeDiscount, to examine the scalability and performance. Our experimental results reveal that HIM scales to larger networks but is outperformed by degreebased algorithms in highly-connected networks. However, HIM performs well in modular networks where the communities are clearly separable with small number of cross-community edges. This finding suggests that for practical applications it is useful to account for network properties when selecting an influence maximization method

Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems

Much research in artificial intelligence is concerned with the development of autonomous agents that can interact effectively with other agents. An important aspect of such agents is the ability to reason about the behaviours of other agents, by constructing models which make predictions about various properties of interest (such as actions, goals, beliefs) of the modelled agents. A variety of modelling approaches now exist which vary widely in their methodology and underlying assumptions, catering to the needs of the different sub-communities within which they were developed and reflecting the different practical uses for which they are intended. The purpose of the present article is to provide a comprehensive survey of the salient modelling methods which can be found in the literature. The article concludes with a discussion of open problems which may form the basis for fruitful future research.Comment: Final manuscript (46 pages), published in Artificial Intelligence Journal. The arXiv version also contains a table of contents after the abstract, but is otherwise identical to the AIJ version. Keywords: autonomous agents, multiagent systems, modelling other agents, opponent modellin

Learning to coordinate in sparse asymmetric multiagent systems

Multiagent learning offers a rich framework to address challenging real-world problems such as remote exploration and healthcare coordination, which require autonomous agents to express elaborate interactions. To be effective in such systems, agents must collectively reason about and pursue high-level, long-term, and possibly nebulous objectives while adapting their strategy to changing environments, inter-agent relationships, and team dynamics. This work introduces six contributions that address this multifaceted problem through the lens of two distinct perspectives: reward structures for high-level objectives that allow agents to consider behaviors before pursuing them, and diversity structures that incentivize asymmetric agents (agents with distinct capabilities and egocentric objectives) to discover complementary specializations required for robust teamwork. The first contribution, Asymmetric D++, distills sparse team feedback into dense informative rewards by encouraging agents to create asymmetric counterfactuals based on their likelihood to cooperate. The second contribution introduces an uncertainty-aware reward approximation that enables the application of Asymmetric D++ for exploration and learning in sparse reward settings. The third contribution, Behavior Refinement, presents a hierarchical framework that shifts focus from optimizing a single behavior to learning a repertoire of diverse behaviors required to complete variegated tasks. Behavior Refinement allows systematic exploration of the policy space via a combination of diversity search and team-objective maximization. The fourth contribution introduces the Island Model, a computational framework that builds on Behavior Refinement for informed behavior space exploration and team balancing for asymmetric agents. The final two contributions, expand upon the Island Model to develop an asynchronous learning framework that allows asymmetric agents to explore diverse environment-agnostic inter-agent relationships to balance multiple potentially conflicting objectives. The amalgamation of this work facilitates asymmetric agents to learn diverse specializations, express complex trade-offs, and discover robust inter-agent relationships required to solve challenging coordination problems. Additionally, the techniques introduced in this work aid in investigating the rich tapestry of agent synergies that evolve in response to changes in the environment and team objectives.Keywords: Multiagent Reinforcement Learning, Multiagent Coordination, Asymmetric Multiagent Systems, Multiagent Evolutionary Learnin

Synthesis of Strategies for Non-Zero-Sum Repeated Games

There are numerous applications that involve two or more self-interested autonomous agents that repeatedly interact with each other in order to achieve a goal or maximize their utilities. This dissertation focuses on the problem of how to identify and exploit useful structures in agents' behavior for the construction of good strategies for agents in multi-agent environments, particularly non-zero-sum repeated games. This dissertation makes four contributions to the study of this problem. First, this thesis describes a way to take a set of interaction traces produced by different pairs of players in a two-player repeated game, and then find the best way to combine them into a strategy. The strategy can then be incorporated into an existing agent, as an enhancement of the agent's original strategy. In cross-validated experiments involving 126 agents for the Iterated Prisoner's Dilemma, Iterated Chicken Game, and Iterated Battle of the Sexes, my technique was able to make improvement to the performance of nearly all of the agents. Second, this thesis investigates the issue of uncertainty about goals when a goal-based agent situated in a nondeterministic environment. The results of this investigation include the necessary and sufficiency conditions for such guarantee, and an algorithm for synthesizing a strategy from interaction traces that maximizes the probability of success of an agent even when no strategy can assure the success of the agent. Third, this thesis introduces a technique, Symbolic Noise Detection (SND), for detecting noise (i.e., mistakes or miscommunications) among agents in repeated games. The idea is that if we can build a model of the other agent's behavior, we can use this model to detect and correct actions that have been affected by noise. In the 20th Anniversary Iterated Prisoner's Dilemma competition, the SND agent placed third in the "noise" category, and was the best performer among programs that had no "slave" programs feeding points to them. Fourth, the thesis presents a generalization of SND that can be wrapped around any existing strategy. Finally, the thesis includes a general framework for synthesizing strategies from experience for repeated games in both noisy and noisy-free environments

Modified bargaining protocols for automated negotiation in open multi-agent systems

Current research in multi-agent systems (MAS) has advanced to the development of open MAS, which are characterized by the heterogeneity of agents, free exit/entry and decentralized control. Conflicts of interest among agents are inevitable, and hence automated negotiation to resolve them is one of the promising solutions. This thesis studies three modifications on alternating-offer bargaining protocols for automated negotiation in open MAS. The long-term goal of this research is to design negotiation protocols which can be easily used by intelligent agents in accommodating their need in resolving their conflicts. In particular, we propose three modifications: allowing non-monotonic offers during the bargaining (non-monotonic-offers bargaining protocol), allowing strategic delay (delay-based bargaining protocol), and allowing strategic ignorance to augment argumentation when the bargaining comprises argumentation (ignorance-based argumentation-based negotiation protocol). Utility theory and decision-theoretic approaches are used in the theoretical analysis part, with an aim to prove the benefit of these three modifications in negotiation among myopic agents under uncertainty. Empirical studies by means of computer simulation are conducted in analyzing the cost and benefit of these modifications. Social agents, who use common human bargaining strategies, are the subjects of the simulation. In general, we assume that agents are bounded rational with various degrees of belief and trust toward their opponents. In particular in the study of the non-monotonic-offers bargaining protocol, we assume that our agents have diminishing surplus. We further assume that our agents have increasing surplus in the study of delay-based bargaining protocol. And in the study of ignorance-based argumentation-based negotiation protocol, we assume that agents may have different knowledge and use different ontologies and reasoning engines. Through theoretical analysis under various settings, we show the benefit of allowing these modifications in terms of agents’ expected surplus. And through simulation, we show the benefit of allowing these modifications in terms of social welfare (total surplus). Several implementation issues are then discussed, and their potential solutions in terms of some additional policies are proposed. Finally, we also suggest some future work which can potentially improve the reliability of these modifications