The significance of bidding, accepting and opponent modeling in automated negotiation

Given the growing interest in automated negotiation, the search for effective strategies has produced a variety of different negotiation agents. Despite their diversity, there is a common structure to their design. A negotiation agent comprises three key components: the bidding strategy, the opponent model and the acceptance criteria. We show that this three-component view of a negotiating architecture not only provides a useful basis for developing such agents but also provides a useful analytical tool. By combining these components in varying ways, we are able to demonstrate the contribution of each component to the overall negotiation result, and thus determine the key contributing components. Moreover, we are able to study the interaction between components and present detailed interaction effects. Furthermore, we find that the bidding strategy in particular is of critical importance to the negotiator's success and far exceeds the importance of opponent preference modeling techniques. Our results contribute to the shaping of a research agenda for negotiating agent design by providing guidelines on how agent developers can spend their time most effectively

What to bid and when to stop

Negotiation is an important activity in human society, and is studied by various disciplines, ranging from economics and game theory, to electronic commerce, social psychology, and artificial intelligence. Traditionally, negotiation is a necessary, but also time-consuming and expensive activity. Therefore, in the last decades there has been a large interest in the automation of negotiation, for example in the setting of e-commerce. This interest is fueled by the promise of automated agents eventually being able to negotiate on behalf of human negotiators.Every year, automated negotiation agents are improving in various ways, and there is now a large body of negotiation strategies available, all with their unique strengths and weaknesses. For example, some agents are able to predict the opponent's preferences very well, while others focus more on having a sophisticated bidding strategy. The problem however, is that there is little incremental improvement in agent design, as the agents are tested in varying negotiation settings, using a diverse set of performance measures. This makes it very difficult to meaningfully compare the agents, let alone their underlying techniques. As a result, we lack a reliable way to pinpoint the most effective components in a negotiating agent.There are two major advantages of distinguishing between the different components of a negotiating agent's strategy: first, it allows the study of the behavior and performance of the components in isolation. For example, it becomes possible to compare the preference learning component of all agents, and to identify the best among them. Second, we can proceed to mix and match different components to create new negotiation strategies., e.g.: replacing the preference learning technique of an agent and then examining whether this makes a difference. Such a procedure enables us to combine the individual components to systematically explore the space of possible negotiation strategies.To develop a compositional approach to evaluate and combine the components, we identify structure in most agent designs by introducing the BOA architecture, in which we can develop and integrate the different components of a negotiating agent. We identify three main components of a general negotiation strategy; namely a bidding strategy (B), possibly an opponent model (O), and an acceptance strategy (A). The bidding strategy considers what concessions it deems appropriate given its own preferences, and takes the opponent into account by using an opponent model. The acceptance strategy decides whether offers proposed by the opponent should be accepted.The BOA architecture is integrated into a generic negotiation environment called Genius, which is a software environment for designing and evaluating negotiation strategies. To explore the negotiation strategy space of the negotiation research community, we amend the Genius repository with various existing agents and scenarios from literature. Additionally, we organize a yearly international negotiation competition (ANAC) to harvest even more strategies and scenarios. ANAC also acts as an evaluation tool for negotiation strategies, and encourages the design of negotiation strategies and scenarios.We re-implement agents from literature and ANAC and decouple them to fit into the BOA architecture without introducing any changes in their behavior. For each of the three components, we manage to find and analyze the best ones for specific cases, as described below. We show that the BOA framework leads to significant improvements in agent design by wining ANAC 2013, which had 19 participating teams from 8 international institutions, with an agent that is designed using the BOA framework and is informed by a preliminary analysis of the different components.In every negotiation, one of the negotiating parties must accept an offer to reach an agreement. Therefore, it is important that a negotiator employs a proficient mechanism to decide under which conditions to accept. When contemplating whether to accept an offer, the agent is faced with the acceptance dilemma: accepting the offer may be suboptimal, as better offers may still be presented before time runs out. On the other hand, accepting too late may prevent an agreement from being reached, resulting in a break off with no gain for either party. We classify and compare state-of-the-art generic acceptance conditions. We propose new acceptance strategies and we demonstrate that they outperform the other conditions. We also provide insight into why some conditions work better than others and investigate correlations between the properties of the negotiation scenario and the efficacy of acceptance conditions.Later, we adopt a more principled approach by applying optimal stopping theory to calculate the optimal decision on the acceptance of an offer. We approach the decision of whether to accept as a sequential decision problem, by modeling the bids received as a stochastic process. We determine the optimal acceptance policies for particular opponent classes and we present an approach to estimate the expected range of offers when the type of opponent is unknown. We show that the proposed approach is able to find the optimal time to accept, and improves upon all existing acceptance strategies.Another principal component of a negotiating agent's strategy is its ability to take the opponent's preferences into account. The quality of an opponent model can be measured in two different ways. One is to use the agent's performance as a benchmark for the model's quality. We evaluate and compare the performance of a selection of state-of-the-art opponent modeling techniques in negotiation. We provide an overview of the factors influencing the quality of a model and we analyze how the performance of opponent models depends on the negotiation setting. We identify a class of simple and surprisingly effective opponent modeling techniques that did not receive much previous attention in literature.The other way to measure the quality of an opponent model is to directly evaluate its accuracy by using similarity measures. We review all methods to measure the accuracy of an opponent model and we then analyze how changes in accuracy translate into performance differences. Moreover, we pinpoint the best predictors for good performance. This leads to new insights concerning how to construct an opponent model, and what we need to measure when optimizing performance.Finally, we take two different approaches to gain more insight into effective bidding strategies. We present a new classification method for negotiation strategies, based on their pattern of concession making against different kinds of opponents. We apply this technique to classify some well-known negotiating strategies, and we formulate guidelines on how agents should bid in order to be successful, which gives insight into the bidding strategy space of negotiating agents. Furthermore, we apply optimal stopping theory again, this time to find the concessions that maximize utility for the bidder against particular opponents. We show there is an interesting connection between optimal bidding and optimal acceptance strategies, in the sense that they are mirrored versions of each other.Lastly, after analyzing all components separately, we put the pieces back together again. We take all BOA components accumulated so far, including the best ones, and combine them all together to explore the space of negotiation strategies.We compute the contribution of each component to the overall negotiation result, and we study the interaction between components. We find that combining the best agent components indeed makes the strongest agents. This shows that the component-based view of the BOA architecture not only provides a useful basis for developing negotiating agents but also provides a useful analytical tool. By varying the BOA components we are able to demonstrate the contribution of each component to the negotiation result, and thus analyze the significance of each. The bidding strategy is by far the most important to consider, followed by the acceptance conditions and finally followed by the opponent model.Our results validate the analytical approach of the BOA framework to first optimize the individual components, and then to recombine them into a negotiating agent

RLBOA: A modular reinforcement learning framework for autonomous negotiating agents

Negotiation is a complex problem, in which the variety of settings and opponents that may be encountered prohibits the use of a single predefined negotiation strategy. Hence the agent should be able to learn such a strategy autonomously. To this end we propose RLBOA, a modular framework that facilitates the creation of autonomous negotiation agents using reinforcement learning. The framework allows for the creation of agents that are capable of negotiating effectively in many different scenarios. To be able to cope with the large size of the state and action spaces and diversity of settings, we leverage the modular BOA-framework. This decouples the negotiation strategy into a Bidding strategy, an Opponent model and an Acceptance condition. Furthermore, we map the multidimensional contract space onto the utility axis which enables a compact and generic state and action description. We demonstrate the value of the RLBOA framework by implementing an agent that uses tabular Q-learning on the compressed state and action space to learn a bidding strategy.We show that the resulting agent is able to learn well-performing bidding strategies in a range of negotiation settings and is able to generalize across opponents and domains

Addressing stability issues in mediated complex contract negotiations for constraint-based, non-monotonic utility spaces

Negotiating contracts with multiple interdependent issues may yield non- monotonic, highly uncorrelated preference spaces for the participating agents. These scenarios are specially challenging because the complexity of the agents’ utility functions makes traditional negotiation mechanisms not applicable. There is a number of recent research lines addressing complex negotiations in uncorrelated utility spaces. However, most of them focus on overcoming the problems imposed by the complexity of the scenario, without analyzing the potential consequences of the strategic behavior of the negotiating agents in the models they propose. Analyzing the dynamics of the negotiation process when agents with different strategies interact is necessary to apply these models to real, competitive environments. Specially problematic are high price of anarchy situations, which imply that individual rationality drives the agents towards strategies which yield low individual and social welfares. In scenarios involving highly uncorrelated utility spaces, “low social welfare” usually means that the negotiations fail, and therefore high price of anarchy situations should be avoided in the negotiation mechanisms. In our previous work, we proposed an auction-based negotiation model designed for negotiations about complex contracts when highly uncorrelated, constraint-based utility spaces are involved. This paper performs a strategy analysis of this model, revealing that the approach raises stability concerns, leading to situations with a high (or even infinite) price of anarchy. In addition, a set of techniques to solve this problem are proposed, and an experimental evaluation is performed to validate the adequacy of the proposed approaches to improve the strategic stability of the negotiation process. Finally, incentive-compatibility of the model is studied.Spain. Ministerio de Educación y Ciencia (grant TIN2008-06739-C04-04

The European Commission and International Trade Negotiations: A Principal-Agent Approach. IES WORKING PAPER 2/2011

Starting from the concept of delegation of power in external trade policy, this paper aims to investigate the dynamics surrounding the European Union’s position in international trade negotiations. The analysis centres on the role of the European Commission (the agent), which by means of Treaty-based delegation and as mandated by the Council (the principal) acts as the sole trade negotiator in the international sphere on behalf of the European Union (EU). The broader negotiating process is thus conceptualised as a threelevel game, where the Commission holds an intermediary position between the European and international levels and also interacts with the Member States in the Council. After an insight into the European decision-making process for external trade, the paper further analyses the Commission’s role during the multilateral trade negotiations of the Doha Development Round. By applying the principal-agent theory to international trade negotiations in general, and subsequently to the controversial agricultural negotiations, this paper seeks to investigate some of the potential sources of autonomy that the Commission can draw upon while upholding an EU position at the international level, in addition to the “hardball” job of balancing the interests of the Member States with those of World Trade Organisation (WTO) partners. Along these lines, the paper finally aims to contribute to the literature concerning agency autonomy in EU external trade relations but also to provide a better understanding of inter-institutional relations within the EU as they may unfold in practice

Intelligent Agents for Automated Cloud Computing Negotiation

Presently, cloud providers offer “off-the-shelf” Service Level Agreements (SLA), on a “take it or leave it” basis. This paper, alternatively, proposes customized SLAs. An automated negotiation is needed to establish customized SLAs between service providers and consumers with no previous knowledge of each other. Traditional negotiations between humans are often fraught with difficulty. Thus, in this work, the use of intelligent agents to represent cloud providers and consumers is advocated. Rubinstein’s Alternating Offers Protocol offers a suitable technical solution for this challenging problem. The purpose of this paper is to apply the state-of-the-art in negotiation automated algorithms/agents within a described Cloud Computing SLA framework, and to evaluate the most appropriate negotiation approach based on many criteria

The role of socio-technical experiments in introducing sustainable Product-Service System innovations

This is the pre-print version of the chapter published in 2015 by Springer in the book “The Handbook of Service Innovation” (edited by Renu Agarwal, Willem Selen, Göran Roos and Roy Green). The final publication is available at Springer via http://dx.doi.org/10.1007/978-1-4471-6590-3_18Product-Service System (PSS) innovations represent a promising approach to sustainability, but their implementation and diffusion are hindered by several cultural, corporate, and regulative barriers. Hence, an important challenge is not only to conceive sustainable PSS concepts, but also to understand how to manage, support, and orient the introduction and diffusion of these concepts. Building upon insights from transition studies (in particular, the concepts of Strategic Niche Management and Transition Management), and through an action research project, the chapter investigates the role of design in introducing sustainable radical service innovations. A key role is given to the implementation of socio-technical experiments, partially protected spaces where innovations can be incubated and tested, become more mature, and potentially favor the implementation and scaling up process