Rational bidding using reinforcement learning: an application in automated resource allocation

The application of autonomous agents by the provisioning and usage of computational resources is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic resource provisioning and usage of computational resources, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a framework for supporting consumers and providers in technical and economic preference elicitation and the generation of bids. Secondly, we introduce a consumer-side reinforcement learning bidding strategy which enables rational behavior by the generation and selection of bids. Thirdly, we evaluate and compare this bidding strategy against a truth-telling bidding strategy for two kinds of market mechanisms – one centralized and one decentralized

SOLACE: A framework for electronic negotiations

Copyright @ 2011 Walter de Gruyter GmbHMost existing frameworks for electronic negotiations today are tied to specific negotiation systems for which they were developed, preventing them from being applied to other negotiation scenarios. Thus, the evaluation of electronic negotiation systems is difficult as each one is based on a different framework. Additionally, each developer has to design a new framework for any system to be developed, leading to a ‘reinvention of the wheel’. This paper presents SOLACE—a generic framework for multi-issue negotiations, which can be applied to a variety of negotiation scenarios. In contrast with other frameworks for electronic negotiations, SOLACE supports hybrid systems in which the negotiation participants can be humans, agents or a combination of the two. By recognizing the importance of strategies in negotiations and incorporating a time attribute in negotiation proposals, SOLACE enhances existing approaches and provides a foundation for the flexible electronic negotiation systems of the future

A Rule-driven Approach for Defining the Behavior of Negotiating Software Agents

One problem with existing agent-mediated negotiation systems is that they rely on ad hoc, static, non-adaptive, and hardcoded schemes to represent the behaviour of agents. This limitation is probably due to the complexity of the negotiation task itself. Indeed, while negotiating, software (human) agents face tough decisions. These decisions are based not only on the information made available by the negotiation server, but on the behaviour of the other participants in the negotiation process as well. The information and the behaviour in question are constantly changing and highly uncertain. In the first part of the paper, we propose a rule-driven approach to represent, manage and explore negotiation strategies and coordination information. For that, we divide the behaviour of negotiating agents into protocols, strategies and coordination. Among the many advantages of the proposed solution, we can cite the high level of abstraction, the closeness to human understanding, the versatility, and the possibility to modify the agents' behaviour during the negotiation process. To validate our solution, we ran many agent tournaments, and used the rule-driven approach to implement bidding strategies that are common in the English and Dutch auctions. We also implemented simple coordination schemes across several auctions. The ongoing validation work is detailed and discussed in the second part of the paper. Un des inconvénients qu'on retrouve fréquemment dans les systèmes de négociation par agents est qu'ils reposent sur des schémas ad-hoc, non adaptatifs et figés dans le code pour représenter le comportement des agents. Cette limitation est probablement due à la complexité de l'activité de négociation elle-même. En effet, au cours de la négociation, les agents logiciels (humains) ont des décisions difficiles à prendre. Ces décisions ne sont pas seulement basées sur l'information disponible sur le serveur de négociation, mais aussi sur le comportement des autres participants durant le processus de négociation. L'information et le comportement en question changent constamment et sont très incertains. Dans la première partie de l'article, nous proposons une approche à base de règles pour représenter, gérer et explorer les stratégies de négociation ainsi que l'information de coordination. Parmi les nombreux avantages de la solution proposée, on peut citer le haut niveau d'abstraction, la proximité avec la compréhension humaine, la souplesse d'utilisation et la possibilité de modifier le comportement des agents durant le processus de négociation. Pour valider notre solution, nous avons effectué plusieurs tournois entre agents et utilisé l'approche à base de règles pour implémenter des stratégies simples applicables à l'enchère anglaise et à l'enchère hollandaise. Nous avons aussi implémenté des schémas simples de coordination impliquant plusieurs enchères. Le travail de validation, en cours, est détaillé et discuté dans la seconde partie de l'article.e-negotiation, online auction, software agent, negotiation strategy, coordination, rule-based system, rule engine, Négociation électronique, enchères en ligne, agents logiciels, stratégie de négociation, coordination, système à base de règles, moteur de règles

PACMAS: A Personalized, Adaptive, and Cooperative MultiAgent System Architecture

In this paper, a generic architecture, designed to support the implementation of applications aimed at managing information among different and heterogeneous sources, is presented. Information is filtered and organized according to personal interests explicitly stated by the user. User pro- files are improved and refined throughout time by suitable adaptation techniques. The overall architecture has been called PACMAS, being a support for implementing Personalized, Adaptive, and Cooperative MultiAgent Systems. PACMAS agents are autonomous and flexible, and can be made personal, adaptive and cooperative, depending on the given application. The peculiarities of the architecture are highlighted by illustrating three relevant case studies focused on giving a support to undergraduate and graduate students, on predicting protein secondary structure, and on classifying newspaper articles, respectively

Human-Agent Decision-making: Combining Theory and Practice

Extensive work has been conducted both in game theory and logic to model strategic interaction. An important question is whether we can use these theories to design agents for interacting with people? On the one hand, they provide a formal design specification for agent strategies. On the other hand, people do not necessarily adhere to playing in accordance with these strategies, and their behavior is affected by a multitude of social and psychological factors. In this paper we will consider the question of whether strategies implied by theories of strategic behavior can be used by automated agents that interact proficiently with people. We will focus on automated agents that we built that need to interact with people in two negotiation settings: bargaining and deliberation. For bargaining we will study game-theory based equilibrium agents and for argumentation we will discuss logic-based argumentation theory. We will also consider security games and persuasion games and will discuss the benefits of using equilibrium based agents.Comment: In Proceedings TARK 2015, arXiv:1606.0729