Rational bidding using reinforcement learning: an application in automated resource allocation

The application of autonomous agents by the provisioning and usage of computational resources is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic resource provisioning and usage of computational resources, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a framework for supporting consumers and providers in technical and economic preference elicitation and the generation of bids. Secondly, we introduce a consumer-side reinforcement learning bidding strategy which enables rational behavior by the generation and selection of bids. Thirdly, we evaluate and compare this bidding strategy against a truth-telling bidding strategy for two kinds of market mechanisms – one centralized and one decentralized

Q-Strategy: A Bidding Strategy for Market-Based Allocation of Grid Services

The application of autonomous agents by the provisioning and usage of computational services is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic service provisioning and usage of Grid services, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a bidding agent framework for implementing artificial bidding agents, supporting consumers and providers in technical and economic preference elicitation as well as automated bid generation by the requesting and provisioning of Grid services. Secondly, we introduce a novel consumer-side bidding strategy, which enables a goal-oriented and strategic behavior by the generation and submission of consumer service requests and selection of provider offers. Thirdly, we evaluate and compare the Q-strategy, implemented within the presented framework, against the Truth-Telling bidding strategy in three mechanisms – a centralized CDA, a decentralized on-line machine scheduling and a FIFO-scheduling mechanisms

Building an Artificial Stock Market Populated by Reinforcement-Learning Agents

In this paper we propose an artificial stock market model based on interaction of heterogeneous agents whose forward-looking behaviour is driven by the reinforcement learning algorithm combined with some evolutionary selection mechanism. We use the model for the analysis of market self-regulation abilities, market efficiency and determinants of emergent properties of the financial market. Distinctive and novel features of the model include strong emphasis on the economic content of individual decision making, application of the Q-learning algorithm for driving individual behaviour, and rich market setup.agent-based financial modelling, artificial stock market, complex dynamical system, emergent properties, market efficiency, agent heterogeneity, reinforcement learning

The Spatial Agent-based Competition Model (SpAbCoM)

The paper presents a detailed documentation of the underlying concepts and methods of the Spatial Agent-based Competition Model (SpAbCoM). For instance, SpAbCoM is used to study firms' choices of spatial pricing policy (GRAUBNER et al., 2011a) or pricing and location under a framework of multi-firm spatial competition and two-dimensional markets (GRAUBNER et al., 2011b). While the simulation model is briefly introduced by means of relevant examples within the corresponding papers, the present paper serves two objectives. First, it presents a detailed discussion of the computational concepts that are used, particularly with respect to genetic algorithms (GAs). Second, it documents SpAbCoM and provides an overview of the structure of the simulation model and its dynamics. -- Das vorliegende Papier dokumentiert die zugrundeliegenden Konzepte und Methoden des Räumlichen Agenten-basierten Wettbewerbsmodells (Spatial Agent-based Competition Model) SpAbCoM. Anwendungsbeispiele dieses Simulationsmodells untersuchen die Entscheidung bezüglich der räumlichen Preisstrategie von Unternehmen (GRAUBNER et al., 2011a) oder Preissetzung und Standortwahl im Rahmen eines räumlichen Wettbewerbsmodells, welches mehr als einen Wettbewerber und zweidimensionalen Marktgebiete berücksichtigt. Während das Simulationsmodell in den jeweiligen Arbeiten kurz anhand eines Beispiels eingeführt wird, dient das vorliegende Papier zwei Zielen. Zum Einen sollen die verwendeten computergestützten Konzepte, hier speziell Genetische Algorithmen (GA), detailliert vorgestellt werden. Zum Anderen besteht die Absicht dieser Dokumentation darin, einen Überblick über die Struktur von SpAbCoM und die während einer Simulation ablaufenden Prozesse zu gegeben.Agent-based modelling,genetic algorithms,spatial pricing,location model.,Agent-basierte Modellierung,Genetische Algorithmen,räumliche Preissetzung,Standortmodell.

Adaptive microfoundations for emergent macroeconomics

In this paper we present the basics of a research program aimed at providing microfoundations to macroeconomic theory on the basis of computational agentbased adaptive descriptions of individual behavior. To exemplify our proposal, a simple prototype model of decentralized multi-market transactions is offered. We show that a very simple agent-based computational laboratory can challenge more structured dynamic stochastic general equilibrium models in mimicking comovements over the business cycle.Microfoundations of macroeconomics, Agent-based economics, Adaptive behavior

Reviewing agent-based modelling of socio-ecosystems: a methodology for the analysis of climate change adaptation and sustainability

The integrated - environmental, economic and social - analysis of climate change calls for a paradigm shift as it is fundamentally a problem of complex, bottom-up and multi-agent human behaviour. There is a growing awareness that global environmental change dynamics and the related socio-economic implications involve a degree of complexity that requires an innovative modelling of combined social and ecological systems. Climate change policy can no longer be addressed separately from a broader context of adaptation and sustainability strategies. A vast body of literature on agent-based modelling (ABM) shows its potential to couple social and environmental models, to incorporate the influence of micro-level decision making in the system dynamics and to study the emergence of collective responses to policies. However, there are few publications which concretely apply this methodology to the study of climate change related issues. The analysis of the state of the art reported in this paper supports the idea that today ABM is an appropriate methodology for the bottom-up exploration of climate policies, especially because it can take into account adaptive behaviour and heterogeneity of the system's components.Review, Agent-Based Modelling, Socio-Ecosystems, Climate Change, Adaptation, Complexity.

Preliminary specification and design documentation for software components to achieve catallaxy in computational systems

This Report is about the preliminary specifications and design documentation for software components to achieve Catallaxy in computational systems. -- Die Arbeit beschreibt die Spezifikation und das Design von Softwarekomponenten, um das Konzept der Katallaxie in Grid Systemen umzusetzen. Eine Einführung ordnet das Konzept der Katallaxie in bestehende Grid Taxonomien ein und stellt grundlegende Komponenten vor. Anschließend werden diese Komponenten auf ihre Anwendbarkeit in bestehenden Application Layer Netzwerken untersucht.Grid Computing

Social Simulation of Stock Markets: Taking It to the Next Level

This paper studies the use of social simulation in linking micro level investor behaviour and macro level stock market dynamics. Empirical data from a survey on individual investors\' decision-making and social interaction was used to formalize the trading and interaction rules of the agents of the artificial stock market SimStockExchange. Multiple simulation runs were performed with this artificial stock market, which generated macro level results, like stock market prices and returns over time. These outcomes were subsequently compared to empirical macro level data from real stock markets. Partial qualitative as well as quantitative agreement between the simulated asset returns distributions and the asset returns distributions of the real stock markets was found.Agent-Based Computational Finance, Artificial Stock Markets, Behavioral Finance, Micro-Macro Links, Multi-Agent Simulation, Stock Market Characteristics