Human-Agent Decision-making: Combining Theory and Practice

Extensive work has been conducted both in game theory and logic to model strategic interaction. An important question is whether we can use these theories to design agents for interacting with people? On the one hand, they provide a formal design specification for agent strategies. On the other hand, people do not necessarily adhere to playing in accordance with these strategies, and their behavior is affected by a multitude of social and psychological factors. In this paper we will consider the question of whether strategies implied by theories of strategic behavior can be used by automated agents that interact proficiently with people. We will focus on automated agents that we built that need to interact with people in two negotiation settings: bargaining and deliberation. For bargaining we will study game-theory based equilibrium agents and for argumentation we will discuss logic-based argumentation theory. We will also consider security games and persuasion games and will discuss the benefits of using equilibrium based agents.Comment: In Proceedings TARK 2015, arXiv:1606.0729

Unanimously acceptable agreements for negotiation teams in unpredictable domains

A negotiation team is a set of agents with common and possibly also conflicting preferences that forms one of the parties of a negotiation. A negotiation team is involved in two decision making processes simultaneously, a negotiation with the opponents, and an intra-team process to decide on the moves to make in the negotiation. This article focuses on negotiation team decision making for circumstances that require unanimity of team decisions. Existing agent-based approaches only guarantee unanimity in teams negotiating in domains exclusively composed of predictable and compatible issues. This article presents a model for negotiation teams that guarantees unanimous team decisions in domains consisting of predictable and compatible, and alsounpredictable issues. Moreover, the article explores the influence of using opponent, and team member models in the proposing strategies that team members use. Experimental results show that the team benefits if team members employ Bayesian learning to model their teammates’ preferences. 2014 Elsevier B.V. All rights reserved.This research is partially supported by TIN2012-36586-C03-01 of the Spanish government and PROMETEOII/2013/019 of Generalitat Valenciana. Other part of this research is supported by the Dutch Technology Foundation STW, applied science division of NWO and the Technology Program of the Ministry of Economic Affairs; the Pocket Negotiator Project with Grant No. VICI-Project 08075.Sánchez Anguix, V.; Aydogan, R.; Julian Inglada, VJ.; Jonker, C. (2014). Unanimously acceptable agreements for negotiation teams in unpredictable domains. Electronic Commerce Research and Applications. 13(4):243-265. https://doi.org/10.1016/j.elerap.2014.05.002S24326513

Challenges and Main Results of the Automated Negotiating Agents Competition (ANAC) 2019

The Automated Negotiating Agents Competition (ANAC) is a yearly-organized international contest in which participants from all over the world develop intelligent negotiating agents for a variety of negotiation problems. To facilitate the research on agent-based negotiation, the organizers introduce new research challenges every year. ANAC 2019 posed five negotiation challenges: automated negotiation with partial preferences, repeated human-agent negotiation, negotiation in supply-chain management, negotiating in the strategic game of Diplomacy, and in the Werewolf game. This paper introduces the challenges and discusses the main findings and lessons learnt per league

Generic Methods for Adaptive Management of Service Level Agreements in Cloud Computing

The adoption of cloud computing to build and deliver application services has been nothing less than phenomenal. Service oriented systems are being built using disparate sources composed of web services, replicable datastores, messaging, monitoring and analytics functions and more. Clouds augment these systems with advanced features such as high availability, customer affinity and autoscaling on a fair pay-per-use cost model. The challenge lies in using the utility paradigm of cloud beyond its current exploit. Major trends show that multi-domain synergies are creating added-value service propositions. This raises two questions on autonomic behaviors, which are specifically ad- dressed by this thesis. The first question deals with mechanism design that brings the customer and provider(s) together in the procurement process. The purpose is that considering customer requirements for quality of service and other non functional properties, service dependencies need to be efficiently resolved and legally stipulated. The second question deals with effective management of cloud infrastructures such that commitments to customers are fulfilled and the infrastructure is optimally operated in accordance with provider policies. This thesis finds motivation in Service Level Agreements (SLAs) to answer these questions. The role of SLAs is explored as instruments to build and maintain trust in an economy where services are increasingly interdependent. The thesis takes a wholesome approach and develops generic methods to automate SLA lifecycle management, by identifying and solving relevant research problems. The methods afford adaptiveness in changing business landscape and can be localized through policy based controls. A thematic vision that emerges from this work is that business models, services and the delivery technology are in- dependent concepts that can be finely knitted together by SLAs. Experimental evaluations support the message of this thesis, that exploiting SLAs as foundations for market innovation and infrastructure governance indeed holds win-win opportunities for both cloud customers and cloud providers

The Likeability-Success Tradeoff: Results of the 2nd Annual Human-Agent Automated Negotiating Agents Competition

We present the results of the 2nd Annual Human-Agent League of the Automated Negotiating Agent Competition. Building on the success of the previous year's results, a new challenge was issued that focused exploring the likeability-success tradeoff in negotiations. By examining a series of repeated negotiations, actions may affect the relationship between automated negotiating agents and their human competitors over time. The results presented herein support a more complex view of human-agent negotiation and capture of integrative potential (win-win solutions). We show that, although likeability is generally seen as a tradeoff to winning, agents are able to remain well-liked while winning if integrative potential is not discovered in a given negotiation. The results indicate that the top-performing agent in this competition took advantage of this loophole by engaging in favor exchange across negotiations (cross-game logrolling). These exploratory results provide information about the effects of different submitted 'black-box' agents in human-agent negotiation and provide a state-of-the-art benchmark for human-agent design.</p

Complex negotiations in multi-agent systems

Los sistemas multi-agente (SMA) son sistemas distribuidos donde entidades autónomas llamadas agentes, ya sean humanos o software, persiguen sus propios objetivos. El paradigma de SMA ha sido propuesto como la aproximación de modelo apropiada para aplicaciones como el comercio electrónico, los sistemas multi-robot, aplicaciones de seguridad, etc. En la comunidad de SMA, la visión de sistemas multi-agente abiertos, donde agentes heterogéneos pueden entrar y salir del sistema dinámicamente, ha cobrado fuerza como paradigma de modelado debido a su relación conceptual con tecnologías como la Web, la computación grid, y las organizaciones virtuales. Debido a la heterogeneidad de los agentes, y al hecho de dirigirse por sus propios objetivos, el conflicto es un fenómeno candidato a aparecer en los sistemas multi-agente. En los últimos años, el término tecnologías del acuerdo ha sido usado para referirse a todos aquellos mecanismos que, directa o indirectamente, promueven la resolución de conflictos en sistemas computacionales como los sistemas multi-agente. Entre las tecnologías del acuerdo, la negociación automática ha sido propuesta como uno de los mecanismos clave en la resolución de conflictos debido a su uso análogo en la resolución de conflictos entre humanos. La negociación automática consiste en el intercambio automático de propuestas llevado a cabo por agentes software en nombre de sus usuarios. El objetivo final es conseguir un acuerdo con todas las partes involucradas. Pese a haber sido estudiada por la Inteligencia Artificial durante años, distintos problemas todavía no han sido resueltos por la comunidad científica todavía. El principal objetivo de esta tesis es proponer modelos de negociación para escenarios complejos donde la complejidad deriva de (1) las limitaciones computacionales o (ii) la necesidad de representar las preferencias de múltiples individuos. En la primera parte de esta tesis proponemos un modelo de negociación bilateral para el problema deSánchez Anguix, V. (2013). Complex negotiations in multi-agent systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/21570Palanci

What to bid and when to stop

Negotiation is an important activity in human society, and is studied by various disciplines, ranging from economics and game theory, to electronic commerce, social psychology, and artificial intelligence. Traditionally, negotiation is a necessary, but also time-consuming and expensive activity. Therefore, in the last decades there has been a large interest in the automation of negotiation, for example in the setting of e-commerce. This interest is fueled by the promise of automated agents eventually being able to negotiate on behalf of human negotiators.Every year, automated negotiation agents are improving in various ways, and there is now a large body of negotiation strategies available, all with their unique strengths and weaknesses. For example, some agents are able to predict the opponent's preferences very well, while others focus more on having a sophisticated bidding strategy. The problem however, is that there is little incremental improvement in agent design, as the agents are tested in varying negotiation settings, using a diverse set of performance measures. This makes it very difficult to meaningfully compare the agents, let alone their underlying techniques. As a result, we lack a reliable way to pinpoint the most effective components in a negotiating agent.There are two major advantages of distinguishing between the different components of a negotiating agent's strategy: first, it allows the study of the behavior and performance of the components in isolation. For example, it becomes possible to compare the preference learning component of all agents, and to identify the best among them. Second, we can proceed to mix and match different components to create new negotiation strategies., e.g.: replacing the preference learning technique of an agent and then examining whether this makes a difference. Such a procedure enables us to combine the individual components to systematically explore the space of possible negotiation strategies.To develop a compositional approach to evaluate and combine the components, we identify structure in most agent designs by introducing the BOA architecture, in which we can develop and integrate the different components of a negotiating agent. We identify three main components of a general negotiation strategy; namely a bidding strategy (B), possibly an opponent model (O), and an acceptance strategy (A). The bidding strategy considers what concessions it deems appropriate given its own preferences, and takes the opponent into account by using an opponent model. The acceptance strategy decides whether offers proposed by the opponent should be accepted.The BOA architecture is integrated into a generic negotiation environment called Genius, which is a software environment for designing and evaluating negotiation strategies. To explore the negotiation strategy space of the negotiation research community, we amend the Genius repository with various existing agents and scenarios from literature. Additionally, we organize a yearly international negotiation competition (ANAC) to harvest even more strategies and scenarios. ANAC also acts as an evaluation tool for negotiation strategies, and encourages the design of negotiation strategies and scenarios.We re-implement agents from literature and ANAC and decouple them to fit into the BOA architecture without introducing any changes in their behavior. For each of the three components, we manage to find and analyze the best ones for specific cases, as described below. We show that the BOA framework leads to significant improvements in agent design by wining ANAC 2013, which had 19 participating teams from 8 international institutions, with an agent that is designed using the BOA framework and is informed by a preliminary analysis of the different components.In every negotiation, one of the negotiating parties must accept an offer to reach an agreement. Therefore, it is important that a negotiator employs a proficient mechanism to decide under which conditions to accept. When contemplating whether to accept an offer, the agent is faced with the acceptance dilemma: accepting the offer may be suboptimal, as better offers may still be presented before time runs out. On the other hand, accepting too late may prevent an agreement from being reached, resulting in a break off with no gain for either party. We classify and compare state-of-the-art generic acceptance conditions. We propose new acceptance strategies and we demonstrate that they outperform the other conditions. We also provide insight into why some conditions work better than others and investigate correlations between the properties of the negotiation scenario and the efficacy of acceptance conditions.Later, we adopt a more principled approach by applying optimal stopping theory to calculate the optimal decision on the acceptance of an offer. We approach the decision of whether to accept as a sequential decision problem, by modeling the bids received as a stochastic process. We determine the optimal acceptance policies for particular opponent classes and we present an approach to estimate the expected range of offers when the type of opponent is unknown. We show that the proposed approach is able to find the optimal time to accept, and improves upon all existing acceptance strategies.Another principal component of a negotiating agent's strategy is its ability to take the opponent's preferences into account. The quality of an opponent model can be measured in two different ways. One is to use the agent's performance as a benchmark for the model's quality. We evaluate and compare the performance of a selection of state-of-the-art opponent modeling techniques in negotiation. We provide an overview of the factors influencing the quality of a model and we analyze how the performance of opponent models depends on the negotiation setting. We identify a class of simple and surprisingly effective opponent modeling techniques that did not receive much previous attention in literature.The other way to measure the quality of an opponent model is to directly evaluate its accuracy by using similarity measures. We review all methods to measure the accuracy of an opponent model and we then analyze how changes in accuracy translate into performance differences. Moreover, we pinpoint the best predictors for good performance. This leads to new insights concerning how to construct an opponent model, and what we need to measure when optimizing performance.Finally, we take two different approaches to gain more insight into effective bidding strategies. We present a new classification method for negotiation strategies, based on their pattern of concession making against different kinds of opponents. We apply this technique to classify some well-known negotiating strategies, and we formulate guidelines on how agents should bid in order to be successful, which gives insight into the bidding strategy space of negotiating agents. Furthermore, we apply optimal stopping theory again, this time to find the concessions that maximize utility for the bidder against particular opponents. We show there is an interesting connection between optimal bidding and optimal acceptance strategies, in the sense that they are mirrored versions of each other.Lastly, after analyzing all components separately, we put the pieces back together again. We take all BOA components accumulated so far, including the best ones, and combine them all together to explore the space of negotiation strategies.We compute the contribution of each component to the overall negotiation result, and we study the interaction between components. We find that combining the best agent components indeed makes the strongest agents. This shows that the component-based view of the BOA architecture not only provides a useful basis for developing negotiating agents but also provides a useful analytical tool. By varying the BOA components we are able to demonstrate the contribution of each component to the negotiation result, and thus analyze the significance of each. The bidding strategy is by far the most important to consider, followed by the acceptance conditions and finally followed by the opponent model.Our results validate the analytical approach of the BOA framework to first optimize the individual components, and then to recombine them into a negotiating agent

Human-Machine Cooperative Decision Making

Diese Dissertation beschäftigt sich mit der gemeinsamen Entscheidungsfindung in der Mensch-Maschine-Kooperation und liefert neue Erkenntnisse, welche von der theoretischen Modellierung bis zu experimentellen Untersuchungen reichen. Zunächst wird eine methodische Klassifikation bestehender Forschung zur Mensch-Maschine-Kooperation vorgenommen und der Forschungsfokus dieser Dissertation mithilfe eines vorgestellten Taxonomiemodells der Mensch-Maschine-Kooperation, dem Butterfly-Modell, abgegrenzt. Darauffolgend stellt die Dissertation zwei mathematische Verhaltensmodelle der gemeinsamen Entscheidungsfindung von Mensch und Maschine vor: das Adaptive Verhandlungsmodell und den n-stufigen War of Attrition. Beide modellieren den Einigungsprozess zweier emanzipierter Kooperationspartner und unterscheiden sich hinsichtlich ihrer Ursprünge, welche in der Verhandlungs- beziehungsweise Spieltheorie liegen. Zusätzlich wird eine Studie vorgestellt, die die Eignung der vorgeschlagenen mathematischen Modelle zur Beschreibung des menschlichen Nachgebeverhaltens in kooperativen Entscheidungsfindungs-Prozessen nachweist. Darauf aufbauend werden zwei modellbasierte Automationsdesigns bereitgestellt, welche die Entwicklung von Maschinen ermöglichen, die an einem Einigungsprozess mit einem Menschen teilnehmen können. Zuletzt werden zwei experimentelle Untersuchungen der vorgeschlagenen Automationsdesigns im Kontext von teleoperierten mobilen Robotern in Such- und Rettungsszenarien und anhand einer Anwendung in einem hochautomatisierten Fahrzeug präsentiert. Die experimentellen Ergebnisse liefern empirische Evidenz für die Überlegenheit der vorgestellten modellbasierten Automationsdesigns gegenüber den bisherigen Ansätzen in den Aspekten der objektiven kooperativen Performanz, des menschlichen Vertrauens in die Interaktion mit der Maschine und der Nutzerzufriedenheit. So zeigt diese Dissertation, dass Menschen eine emanzipierte Interaktion mit Bezug auf die Entscheidungsfindung bevorzugen, und leistet einen wertvollen Beitrag zur vollumfänglichen Betrachtung und Verwirklichung von Mensch-Maschine-Kooperationen