Rational bidding using reinforcement learning: an application in automated resource allocation

The application of autonomous agents by the provisioning and usage of computational resources is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic resource provisioning and usage of computational resources, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a framework for supporting consumers and providers in technical and economic preference elicitation and the generation of bids. Secondly, we introduce a consumer-side reinforcement learning bidding strategy which enables rational behavior by the generation and selection of bids. Thirdly, we evaluate and compare this bidding strategy against a truth-telling bidding strategy for two kinds of market mechanisms – one centralized and one decentralized

Some Econometric Evidence on the Effectiveness of Active Labour Market Programmes in East Germany

In this paper we summarise our previous results on the effectiveness of different kinds of labour market training programmes as well as employment programmes in East Germany after unification. All the studies use the microeconometric evaluation approach and are based on different types of matching estimators. We find some positive earnings effect for on-the-job training and also some positive employment effects for employment programmes. No such effects appear for public sector sponsored (off-the-job) training programmes. Generally, the scope of such analysis is very much hampered by the insufficient quality and quantity of the data available for East Germany. Although in particular the results for public sector sponsored training programmes raise serious doubts about the effectiveness of these programmes, any definite policy conclusion from this and other studies about active labour market policy in East Germany would probably be premature.http://deepblue.lib.umich.edu/bitstream/2027.42/39702/3/wp318.pd

CONSUMERS' WILLINGNESS TO PAY FOR THE COLOR OF SALMON:A CHOICE EXPERIMENT WITH REAL ECONOMIC INCENTIVES

We designed an experimental market with posted prices to investigate consumers' willingness to pay for the color of salmon. Salmon fillets varying in color and price were displayed in 20 choice scenarios. In each scenario, the participants chose which of two salmon fillets they wanted to buy. To induce real economic incentives, each participant drew one unique binding scenario; the participants then had to buy the salmon fillet they had chosen in their binding scenario.Consumer/Household Economics,

Stochastic simulation framework for the Limit Order Book using liquidity motivated agents

In this paper we develop a new form of agent-based model for limit order books based on heterogeneous trading agents, whose motivations are liquidity driven. These agents are abstractions of real market participants, expressed in a stochastic model framework. We develop an efficient way to perform statistical calibration of the model parameters on Level 2 limit order book data from Chi-X, based on a combination of indirect inference and multi-objective optimisation. We then demonstrate how such an agent-based modelling framework can be of use in testing exchange regulations, as well as informing brokerage decisions and other trading based scenarios

Measuring and Optimizing Cultural Markets

Social influence has been shown to create significant unpredictability in cultural markets, providing one potential explanation why experts routinely fail at predicting commercial success of cultural products. To counteract the difficulty of making accurate predictions, "measure and react" strategies have been advocated but finding a concrete strategy that scales for very large markets has remained elusive so far. Here we propose a "measure and optimize" strategy based on an optimization policy that uses product quality, appeal, and social influence to maximize expected profits in the market at each decision point. Our computational experiments show that our policy leverages social influence to produce significant performance benefits for the market, while our theoretical analysis proves that our policy outperforms in expectation any policy not displaying social information. Our results contrast with earlier work which focused on showing the unpredictability and inequalities created by social influence. Not only do we show for the first time that dynamically showing consumers positive social information under our policy increases the expected performance of the seller in cultural markets. We also show that, in reasonable settings, our policy does not introduce significant unpredictability and identifies "blockbusters". Overall, these results shed new light on the nature of social influence and how it can be leveraged for the benefits of the market

A Collaborative Mechanism for Crowdsourcing Prediction Problems

Machine Learning competitions such as the Netflix Prize have proven reasonably successful as a method of "crowdsourcing" prediction tasks. But these competitions have a number of weaknesses, particularly in the incentive structure they create for the participants. We propose a new approach, called a Crowdsourced Learning Mechanism, in which participants collaboratively "learn" a hypothesis for a given prediction task. The approach draws heavily from the concept of a prediction market, where traders bet on the likelihood of a future event. In our framework, the mechanism continues to publish the current hypothesis, and participants can modify this hypothesis by wagering on an update. The critical incentive property is that a participant will profit an amount that scales according to how much her update improves performance on a released test set.Comment: Full version of the extended abstract which appeared in NIPS 201