Rational bidding using reinforcement learning: an application in automated resource allocation

The application of autonomous agents by the provisioning and usage of computational resources is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic resource provisioning and usage of computational resources, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a framework for supporting consumers and providers in technical and economic preference elicitation and the generation of bids. Secondly, we introduce a consumer-side reinforcement learning bidding strategy which enables rational behavior by the generation and selection of bids. Thirdly, we evaluate and compare this bidding strategy against a truth-telling bidding strategy for two kinds of market mechanisms – one centralized and one decentralized

Constraint rule-based programming of norms for electronic institutions

Peer reviewedPostprin

Q-Strategy: A Bidding Strategy for Market-Based Allocation of Grid Services

The application of autonomous agents by the provisioning and usage of computational services is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic service provisioning and usage of Grid services, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a bidding agent framework for implementing artificial bidding agents, supporting consumers and providers in technical and economic preference elicitation as well as automated bid generation by the requesting and provisioning of Grid services. Secondly, we introduce a novel consumer-side bidding strategy, which enables a goal-oriented and strategic behavior by the generation and submission of consumer service requests and selection of provider offers. Thirdly, we evaluate and compare the Q-strategy, implemented within the presented framework, against the Truth-Telling bidding strategy in three mechanisms – a centralized CDA, a decentralized on-line machine scheduling and a FIFO-scheduling mechanisms

"On Detail-Free Mechanism Design and Rationality"

The study of mechanism design is sometimes criticized, because the designed mechanisms depend on the fine detail of the model specification, and agents' behavior relies on the strong common knowledge assumptions on their rationality and others. Hence, the study of 'detail-free' mechanism design with weak informational assumptions is the most important to make as the first step towards a practically useful theory. This paper will emphasize that even if we confine our attentions to detail-free mechanisms with week rationality, there still exist a plenty of scope for development of new ideas on how to design a mechanism to play the powerful role. We briefly explain my recent works on this line, and argue that the use of stochastic decision works much in large exchange economics, and agents' moral preferences can drastically improve implementability of social choice functions.

Auction-based Bandwidth Allocation Mechanisms for Wireless Future Internet

An important aspect of the Future Internet is the efficient utilization of (wireless) network resources. In order for the - demanding in terms of QoS - Future Internet services to be provided, the current trend is evolving towards an "integrated" wireless network access model that enables users to enjoy mobility, seamless access and high quality of service in an all-IP network on an "Anytime, Anywhere" basis. The term "integrated" is used to denote that the Future Internet wireless "last mile" is expected to comprise multiple heterogeneous geographically coexisting wireless networks, each having different capacity and coverage radius. The efficient management of the wireless access network resources is crucial due to their scarcity that renders wireless access a potential bottleneck for the provision of high quality services. In this paper we propose an auction mechanism for allocating the bandwidth of such a network so that efficiency is attained, i.e. social welfare is maximized. In particular, we propose an incentive-compatible, efficient auction-based mechanism of low computational complexity. We define a repeated game to address user utilities and incentives issues. Subsequently, we extend this mechanism so that it can also accommodate multicast sessions. We also analyze the computational complexity and message overhead of the proposed mechanism. We then show how user bids can be replaced from weights generated by the network and transform the auction to a cooperative mechanism capable of prioritizing certain classes of services and emulating DiffServ and time-of-day pricing schemes. The theoretical analysis is complemented by simulations that assess the proposed mechanisms properties and performance. We finally provide some concluding remarks and directions for future research

Allocative and Informational Externalities in Auctions and Related Mechanisms

We study the effects of allocative and informational externalities in (multi-object) auctions and related mechanisms. Such externalities naturally arise in models that embed auctions in larger economic contexts. In particular, they appear when there is downstream interaction among bidders after the auction has closed. The endogeneity of valuations is the main driving force behind many new, specific phenomena with allocative externalities: even in complete information settings, traditional auction formats need not be efficient, and they may give rise to multiple equilibria and strategic non-participation. But, in the absence of informational externalities, welfare maximization can be achieved by Vickrey-Clarke- Groves mechanisms. Welfare-maximizing Bayes-Nash implementation is, however, impossible in multi-object settings with informational externalities, unless the allocation problem is separable across objects (e.g. there are no allocative externalities nor complementarities) or signals are one-dimensional. Moreover, implementation of any choice function via ex-post equilibrium is generically impossible with informational externalities and multidimensional types. A theory of information constraints with multidimensional signals is rather complex, but indispensable for our study

On Detail-Free Mechanism Design and Rationality

The study of mechanism design is sometimes criticized, because the designed mechanisms depend on the fine detail of the model specification, and agents' behavior relies on the strong common knowledge assumptions on their rationality and others. Hence, the study of 'detail-free' mechanism design with weak informational assumptions is the most important to make as the first step towards a practically useful theory. This paper will emphasize that even if we confine our attentions to detail-free mechanisms with week rationality, there still exist a plenty of scope for development of new ideas on how to design a mechanism to play the powerful role. We briefly explain my recent works on this line, and argue that the use of stochastic decision works much in large exchange economics, and agents' moral preferences can drastically improve implementability of social choice functions.

Price-Based Combinatorial Auction: Connectedness and Representative Valuations

We investigate combinatorial auctions from a practical perspective. The auctioneer gathers information according to a dynamical protocol termed ask price procedure. We demonstrate a method for elucidating whether a procedure gathers sufficient information for deriving a VCG mechanism. We calculate representative valuation functions in a history-contingent manner, and show that it is necessary and sufficient to examine whether efficient allocations with and without any buyer associated with the profile of representative valuation functions were revealed. This method is tractable, and can be applied to general procedures with connectedness. The representative valuation functions could be the sufficient statistics for privacy preservation.

Evolutionary Optimization of ZIP60: A Controlled Explosion in Hyperspace

The “ZIP” adaptive trading algorithm has been demonstrated to out-perform human traders in experimental studies of continuous double auction (CDA) markets. The original ZIP algorithm requires the values of eight control parameters to be set correctly. A new extension of the ZIP algorithm, called ZIP60, requires the values of 60 parameters to be set correctly. ZIP60 is shown here to produce significantly better results than the original ZIP (called “ZIP8” hereafter), for negligable additional computational costs. A genetic algorithm (GA) is used to search the 60-dimensional ZIP60 parameter space, and it finds parameter vectors that yield ZIP60 traders with mean scores significantly better than those of ZIP8s. This paper shows that the optimizing evolutionary search works best when the GA itself controls the dimensionality of the search-space, so that the search commences in an 8-d space and thereafter the dimensionality of the search-space is gradually increased by the GA until it is exploring a 60-d space. Furthermore, the results from ZIP60 cast some doubt on prior ZIP8 results concerning the evolution of new ‘hybrid’ auction mechanisms that appeared to be better than the CDA