Autonomous agents in bargaining games : an evolutionary investigation of fundamentals, strategies, and business applications

Bargaining is becoming increasingly important due to developments within the field of electronic commerce, especially the development of autonomous software agents. Software agents are programs which, given instructions from a user, are capable of autonomously and intelligently realise a given task. By means of such agents, the bargaining process can be automated, allowing products and services together with related conditions, such as warranty and delivery time, to be flexible and tuned to the individual preferences of the people concerned. In this theses we concentrate on both fundamental aspects of bargaining as well as business-related applications of automated bargaining using software agents. The fundamental part investigates bargaining outcomes within a stylised world, and the factors that influence these outcomes. This can provide insights for the production of software agents, strategies, and setting up bargaining rules for practical situations. We study these aspects using computational simulations of bargaining agents. Hereby we consider adaptive systems, i.e., where agents learn to adjust their bargaining strategy given past experience. This learning behaviour is simulated using evolutionary algorithms. These algorithms originate from the field of artificial intelligence, and are inspired by the biological theory of evolution. Originally, evolutionary algorithms were designed for solving optimisation problems, but they are now increasingly being used within economics for modelling human learning behaviour. Besides computational simulations, we also consider mathematical solutions from game theory for relatively simple cases. Game theory is mainly concerned with the “rational man”, that is, with optimal outcomes within an stylised setting (or game) where people act rationally. We use the game-theoretic outcomes to validate the computational experiments. The advantage of computer simulations is that less strict assumptions are necessary, and that more complex interactions that are closer to real-world settings can be investigated. First of all, we study a bargaining setting where two players exchange offers and counter offers, the so-called alternating-offers game. This game is frequently used for modelling bargaining about for instance the price of a product or service. It is also important, however, to allow other product- and service-related aspects to be negotiated, such as quality, delivery time, and warranty. This enables compromises by conceding on less important issues and demanding a higher value for relatively important aspects. This way, bargaining is less competitive and the resulting outcome can be mutually beneficial. Therefore, we investigate using computational simulations an extended version of the alternating-offers game, where multiple aspects are negotiated concurrently. Moreover, we apply game theory to validate the results of the computational experiments. The simulation shows that learning agents are capable of quickly finding optimal compromises, also called Pareto-efficient outcomes. In addition, we study the effects of time pressure that arise if negotiations are broken off with a small probability, for example due to external eventualities. In absence of time pressure and a maximum number of negotiation rounds, outcomes are very unbalanced: the player that has the opportunity to make a final offer proposes a take-it-or-leave-it offer in the last round, which leaves the other player with a deal that is only slightly better than no deal at all. With relatively high time pressure, on the other hand, the first offer is most important and almost all agreements are reached in the first round. Another interesting result is that the simulation outcomes after a long period of learning in general coincide with the results from game theory, in spite of the fact that the learning agents are not “rational”. In reality, not only the final outcome is important, but also other factors play a role, such as the fairness of an offer. Using the simulation we study the influence of such fairness norms on the bargaining outcomes. The fairness norms result in much more balanced outcomes, even with no time pressure, and seem to be closer outcomes in the real world. Negotiations are rarely isolated, but can also be influenced by external factors such as additional bargaining opportunities. We therefore also consider bargaining within a market-like setting, where both buyers and sellers can bargain with several opponents before reaching an agreement. The negotiations are executed consecutively until an agreement is reached or no more opportunities are available. Each bargaining game is reduced to a single round, where player 1 makes an offer and player 2 can only respond by rejecting or accepting this offer. Using an evolutionary simulation we study several properties of this market game. It appears that the outcomes depend on the information that is available to the players. If players are informed about the bargaining opportunities of their opponents, the first player in turn has the advantage and always proposes a take-it-or-leave-it deal that leaves the other player with a relatively poor outcome. This outcome is consistent with a game-theoretic analysis which we also present in this thesis. If this information is not available, a theoretical analysis is very hard. The evolutionary simulation, however, shows that in this case the responder obtains a better deal. This occurs because the first player can no longer anticipate the response of the other player, and therefore bids lower to avoid a disagreement. In this thesis, we additionally consider other factors that influence the outcomes of the market game, such as negotiation over multiple issues simultaneously, search costs, and break off probabilities. Besides fundamental issues, this thesis presents a number of business-related applications of automated bargaining, as well as generic bargaining strategies for agents that can be employed in related areas. As a first application, we introduce a framework where negotiation is used for recommending shops to customers, for example on a web page of an electronic shopping mall. Through a market-driven auction a relevant selection of shops is determined in a distributed fashion. This is achieved by selling a limited number of banner spaces in an electronic auction. For each arriving customer on the web page, shops can automatically place bids for this “customer attention space” through their shop agents. These software agents bid based on a customer profile, containing personal data of the customer, such as age, interests, and/or keywords in a search query. The shop agents are adaptive and learn, given feedback from the customers, which profiles to target and how much to bid in the auction. The highest bidders are then selected and displayed to the customer. The feasibility of this distributed approach for matching shops to customers is demonstrated using an evolutionary simulation. Several customer models and auction mechanisms are studied, and we show that the market-based approach results in a proper selection of shops for the customers. Bargaining can be especially beneficial if not only the price, but other aspects are considered as well. This allows for example to customise products and services to the personal preferences of a user. We developed a system makes use of these properties for selling and personalising so-called information goods, such as news articles, software, and music. Using the alternating-offers protocol, a seller agent negotiates with several buyers simultaneously about a fixed price, a per-item price, and the quality of a bundle of information goods. The system is capable of taking into account important business-related conditions such as the fairness of the negotiation. The agents combine a search strategy and a concession strategy to generate offers in the negotiations. The concession strategy determines the amount the agent will concede each round, whereas the search strategy takes care of the personalisation of the offer. We introduce two search strategies in this thesis, and show through computer experiments that the use of these strategies by a buyer and seller agent, result in personalised outcomes, also when combined with various concession strategies. The search strategies presented here can be easily applied to other domains where personalisation is important. In addition, we also developed concession strategies for the seller agent that can be used in settings where a single seller agent bargains with several buyer agents simultaneously. Even if bargaining itself is bilateral (i.e., between two parties), a seller agent can actually benefit from the fact that several such negotiations occur concurrently. The developed strategies are focussed on domains where supply is flexible and can be adjusted to meet demand, like for information goods. We study fixed strategies, time-dependent strategies and introduce several auction-inspired strategies. Auctions are often used when one party negotiates with several opponents simultaneously. Although the latter strategies benefit from the advantages of auctions, the actual negotiation remains bilateral and consists of exchanging offers and counter offers. We developed an evolutionary simulation environment to evaluate the seller agent’s strategies. We especially consider the case where buyers are time-impatient and under pressure to reach agreements early. The simulations show that the auction-inspired strategies are able to obtain almost maximum profits from the negotiations, given sufficient time pressure of the buyers

POLICY options

End of project reportThe incomes of Irish cattle farmers benefited greatly from the reform of the CAP for beef and cereals in 1992 and more recently under Agenda 2000. In both of these reforms the institutional support prices were reduced and animal-based direct payments (DPs) were used to compensate farmers for the anticipated market price reductions

Performance Evaluation - Annual Report Year 3

This report describes the work done and results obtained in third year of the CATNETS project. Experiments carried out with the different configurations of the prototype are reported and simulation results are evaluated with the CATNETS metrics framework. The applicability of the Catallactic approach as market model for service and resource allocation in application layer networks is assessed based on the results and experience gained both from the prototype development and simulations. --Grid Computing

New perspectives on realism, tractability, and complexity in economics

Fuzzy logic and genetic algorithms are used to rework more realistic (and more complex) models of competitive markets. The resulting equilibria are significantly different from the ones predicted from the usual static analysis; the methodology solves the Walrasian problem of how markets can reach equilibrium, starting with firms trading at disparate prices. The modified equilibria found in these complex market models involve some mutual self-restraint on the part of the agents involved, relative to economically rational behaviour. Research (using similar techniques) into the evolution of collaborative behaviours in economics, and of altruism generally, is summarized; and the joint significance of these two bodies of work for public policy is reviewed. The possible extension of the fuzzy/ genetic methodology to other technical aspects of economics (including international trade theory, and development) is also discussed, as are the limitations to the usefulness of any type of theory in political domains. For the latter purpose, a more differentiated concept of rationality, appropriate to ill-structured choices, is developed. The philosophical case for laissez-faire policies is considered briefly; and the prospects for change in the way we ‘do economics’ are analysed

Distributed, decentralised and compensational mechanisms for platoon formation

Verkehrsprobleme nehmen mit der weltweiten Urbanisierung und der Zunahme der Anzahl der Fahrzeuge pro Kopf zu. Platoons, eine Formation von eng hintereinander fahrenden Fahrzeugen, stellen sich als mögliche Lösung dar, da bestehende Forschungen darauf hinweisen, dass sie zu einer besseren Straßenauslastung beitragen, den Kraftstoffverbrauch und die Emissionen reduzieren und Engpässe schneller entlasten können. Rund um das Thema Platooning gibt es viele Aspekte zu erforschen: Sicherheit, Stabilität, Kommunikation, Steuerung und Betrieb, die allesamt notwendig sind, um den Einsatz von Platooning im Alltagsverkehr näher zu bringen. Während in allen genannten Bereichen bereits umfangreiche Forschungen durchgeführt wurden, gibt es bisher nur wenige Arbeiten, die sich mit der logischen Gruppierung von Fahrzeugen in Platoons beschäftigen. Daher befasst sich diese Arbeit mit dem noch wenig erforschten Problem der Platoonbildung, wobei sich die vorhandenen Beispiele mit auf Autobahnen fahrenden Lastkraftwagen beschäftigen. Diese Fälle befinden sich auf der strategischen und taktischen Ebene der Planung, da sie von einem großen Zeithorizont profitieren und die Gruppierung entsprechend optimiert werden kann. Die hier vorgestellten Ansätze befinden sich hingegen auf der operativen Ebene, indem Fahrzeuge aufgrund der verteilten und dezentralen Natur dieser Ansätze spontan und organisch gruppiert und gesteuert werden. Dadurch entstehen sogenannte opportunistische Platoons, die aufgrund ihrer Flexibilität eine vielversprechende Voraussetzung für alle Netzwerkarte bieten könnten. Insofern werden in dieser Arbeit zwei neuartige Algorithmen zur Bildung von Platoons vorgestellt: ein verteilter Ansatz, der von klassischen Routing-Problemen abgeleitet wurde, und ein ergänzender dezentraler kompensatorischer Ansatz. Letzteres nutzt automatisierte Verhandlungen, um es den Fahrzeugen zu erleichtern, sich auf der Basis eines monetären Austausches in einem Platoon zu organisieren. In Anbetracht der Tatsache, dass alle Verkehrsteilnehmer über eine Reihe von Präferenzen, Einschränkungen und Zielen verfügen, muss das vorgeschlagene System sicherstellen, dass jede angebotene Lösung für die einzelnen Fahrzeuge akzeptabel und vorteilhaft ist und den möglichen Aufwand, die Kosten und die Opfer überwiegt. Dies wird erreicht, indem den Platooning-Fahrzeugen eine Form von Anreiz geboten wird, im Sinne von entweder Kostensenkung oder Ampelpriorisierung. Um die vorgeschlagenen Algorithmen zu testen, wurde eine Verkehrssimulation unter Verwendung realer Netzwerke mit realistischer Verkehrsnachfrage entwickelt. Die Verkehrsteilnehmer wurden in Agenten umgewandelt und mit der notwendigen Funktionalität ausgestattet, um Platoons zu bilden und innerhalb dieser zu operieren. Die Anwendbarkeit und Eignung beider Ansätze wurde zusammen mit verschiedenen anderen Aspekten untersucht, die den Betrieb von Platoons betreffen, wie Größe, Verkehrszustand, Netzwerkpositionierung und Anreizmethoden. Die Ergebnisse zeigen, dass die vorgeschlagenen Mechanismen die Bildung von spontanen Platoons ermöglichen. Darüber hinaus profitierten die teilnehmenden Fahrzeuge mit dem auf verteilter Optimierung basierenden Ansatz und unter Verwendung kostensenkender Anreize unabhängig von der Platoon-Größe, dem Verkehrszustand und der Positionierung, mit Nutzenverbesserungen von 20% bis über 50% im Vergleich zur untersuchten Baseline. Bei zeitbasierten Anreizen waren die Ergebnisse uneinheitlich, wobei sich der Nutzen einiger Fahrzeuge verbesserte, bei einigen keine Veränderung eintrat und bei anderen eine Verschlechterung zu verzeichnen war. Daher wird die Verwendung solcher Anreize aufgrund ihrer mangelnden Pareto-Effizienz nicht empfohlen. Der kompensatorische und vollständig dezentralisierte Ansatz weißt einige Vorteile auf, aber die daraus resultierende Verbesserung war insgesamt vernachlässigbar. Die vorgestellten Mechanismen stellen einen neuartigen Ansatz zur Bildung von Platoons dar und geben einen aussagekräftigen Einblick in die Mechanik und Anwendbarkeit von Platoons. Dies schafft die Voraussetzungen für zukünftige Erweiterungen in der Planung, Konzeption und Implementierung effektiverer Infrastrukturen und Verkehrssysteme.Traffic problems have been on the rise corresponding with the increase in worldwide urbanisation and the number of vehicles per capita. Platoons, which are a formation of vehicles travelling close together, present themselves as a possible solution, as existing research indicates that they can contribute to better road usage, reduce fuel consumption and emissions and decongest bottlenecks faster. There are many aspects to be explored pertaining to the topic of platooning: safety, stability, communication, controllers and operations, all of which are necessary to bring platoons closer to use in everyday traffic. While extensive research has already made substantial strides in all the aforementioned fields, there is so far little work on the logical grouping of vehicles in platoons. Therefore, this work addresses the platoon formation problem, which has not been heavily researched, with existing examples being focused on large, freight vehicles travelling on highways. These cases find themselves on the strategic and tactical level of planning since they benefit from a large time horizon and the grouping can be optimised accordingly. The approaches presented here, however, are on the operational level, grouping and routing vehicles spontaneously and organically thanks to their distributed and decentralised nature. This creates so-called opportunistic platoons which could provide a promising premise for all networks given their flexibility. To this extent, this thesis presents two novel platoon forming algorithms: a distributed approach derived from classical routing problems, and a supplementary decentralised compensational approach. The latter uses automated negotiation to facilitate vehicles organising themselves in a platoon based on monetary exchanges. Considering that all traffic participants have a set of preferences, limitations and goals, the proposed system must ensure that any solution provided is acceptable and beneficial for the individual vehicles, outweighing any potential effort, cost and sacrifices. This is achieved by offering platooning vehicles some form of incentivisation, either cost reductions or traffic light prioritisation. To test the proposed algorithms, a traffic simulation was developed using real networks with realistic traffic demand. The traffic participants were transformed into agents and given the necessary functionality to build platoons and operate within them. The applicability and suitability of both approaches were investigated along with several other aspects pertaining to platoon operations such as size, traffic state, network positioning and incentivisation methods. The results indicate that the mechanisms proposed allow for spontaneous platoons to be created. Moreover, with the distributed optimisation-based approach and using cost-reducing incentives, participating vehicles benefited regardless of the platoon size, traffic state and positioning, with utility improvements ranging from 20% to over 50% compared to the studied baseline. For time-based incentives the results were mixed, with the utility of some vehicles improving, some seeing no change and for others, deteriorating. Therefore, the usage of such incentives would not be recommended due to their lack of Pareto-efficiency. The compensational and completely decentralised approach shows some benefits, but the resulting improvement was overall negligible. The presented mechanisms are a novel approach to platoon formation and provide meaningful insight into the mechanics and applicability of platoons. This sets the stage for future expansions into planning, designing and implementing more effective infrastructures and traffic systems

New perspectives on realism, tractability, and complexity in economics

Fuzzy logic and genetic algorithms are used to rework more realistic (and more complex) models of competitive markets. The resulting equilibria are significantly different from the ones predicted from the usual static analysis; the methodology solves the Walrasian problem of how markets can reach equilibrium, starting with firms trading at disparate prices. The modified equilibria found in these complex market models involve some mutual self-restraint on the part of the agents involved, relative to economically rational behaviour. Research (using similar techniques) into the evolution of collaborative behaviours in economics, and of altruism generally, is summarized; and the joint significance of these two bodies of work for public policy is reviewed. The possible extension of the fuzzy/ genetic methodology to other technical aspects of economics (including international trade theory, and development) is also discussed, as are the limitations to the usefulness of any type of theory in political domains. For the latter purpose, a more differentiated concept of rationality, appropriate to ill-structured choices, is developed. The philosophical case for laissez-faire policies is considered briefly; and the prospects for change in the way we ‘do economics’ are analysed.Fuzzy logic; genetic algorithms; complexity; emergence; rationality; ill-structured choice; equilibrium; Walrasian Crier; paradigm change;