Learning to bid in revenue-maximizing auctions

We consider the problem of the optimization of bidding strategies in prior-dependent revenue-maximizing auctions, when the seller fixes the reserve prices based on the bid distributions. Our study is done in the setting where one bidder is strategic. Using a variational approach, we study the complexity of the original objective and we introduce a relaxation of the objective functional in order to use gradient descent methods. Our approach is simple, general and can be applied to various value distributions and revenue-maximizing mechanisms. The new strategies we derive yield massive uplifts compared to the traditional truthfully bidding strategy

Private Monitoring in Auctions

We study collusion in repeated first-price auctions under the condition of minimal information release by the auctioneer. In each auction a bidder only learns whether or not he won the object. Bidders do not observe other bidders’ bids, who participates or who wins in case they are not the winner. We show that for large enough discount factors collusion can nevertheless be supported in the infinitely repeated game. While there is a unique Nash equilibrium in public strategies, in which bidders bid competitively in every period, there are simple Nash equilibria in private strategies that support bid rotation. Equilibria that either improve on bid rotation or satisfy the requirement of Bayesian perfection, but not both, are only slightly more complex. Our main result is the construction of perfect Bayesian equilibria that improve on bid rotation. These equilibria require complicated inferences off the equilibrium path. A deviator may not know who has observed his deviation and consequently may have an incentive to use strategic experimentation to learn about the bidding behavior of his rivals. ZUSAMMENFASSUNG - (Privates Monitoring in Auktionen) Der Beitrag untersucht, inwieweit Bieter Kollusion, bzw. stillschweigende Abkommen, in wiederholten Erstpreisauktionen aufrecht erhalten können, in welchen der Auktionator alle Informationen zurückhält. Nach jeder Auktion lernt ein Bieter nur, ob er das Objekt gewonnen hat oder nicht. Ein Bieter kann weder die Gebote der anderen Bieter beobachten, noch kann er beobachten, welche Bieter an der Auktion teilgenommen haben und wer gewonnen hat - solange er nicht selbst das Objekt erhält. Wir zeigen, dass in dem unendlich wiederholten Spiel für hinreichend geduldige Bieter Kollusion möglich ist. Es existiert zwar ein eindeutiges Gleichgewicht in öffentlichen Strategien, in welchem die Bieter in jeder Periode kompetitiv bieten, aber es gibt einfache Nash-Gleichgewichte in privaten Strategien, die Bieterrotation durchsetzen. Wir zeigen auch, dass Bieterrotation das Ergebnis eines perfekt bayesianischen Gleichgewichtes sein kann. Nash-Gleichgewichte, die höhere erwartete Gewinne als Bieterrotation erzielen, sind nur ein wenig komplexer. Das Hauptergebnis ist die Konstruktion von (essentiell) perfekt bayesianischen Gleichgewichten, welche höhere Gewinne als Bieterrotation erzielen. Nach Abweichungen vom Gleichgewichtspfad, müssen die Bieter in diesen Gleichgewichten komplizierte Rückschlüsse auf das Verhalten Ihrer Wettbewerber ziehen. So weiß ein Bieter nach bestimmten Abweichungen nicht, ob diese von seinen Mitspielern beobachtet wurden, und hat ein Interesse daran, durch strategisches experimentieren das Bietverhalten seiner Rivalen kennenzulernen.Itacit collusion, repeated auctions, supergames, contagion, bid-rotation, trigger strategies.

Private monitoring in auctions

"We study collusion in repeated first-price auctions under the condition of minimal information release by the auctioneer. In each auction a bidder only learns whether or not he won the object. Bidders do not observe other bidders? bids, who participates or who wins in case they are not the winner. We show that for large enough discount factors collusion can nevertheless be supported in the infinitely repeated game. While there is a unique Nash equilibrium in public strategies, in which bidders bid competitively in every period, there are simple Nash equilibria in private strategies that support bid rotation. Equilibria that either improve on bid rotation or satisfy the requirement of Bayesian perfection, but not both, are only slightly more complex. Our main result is the construction of perfect Bayesian equilibria that improve on bid rotation. These equilibria require complicated inferences off the equilibrium path. A deviator may not know who has observed his deviation and consequently may have an incentive to use strategic experimentation to learn about the bidding behavior of his rivals." (author's abstract)"Der Beitrag untersucht, inwieweit Bieter Kollusion, bzw. stillschweigende Abkommen, in wiederholten Erstpreisauktionen aufrecht erhalten können, in welchen der Auktionator alle Informationen zurückhält. Nach jeder Auktion lernt ein Bieter nur, ob er das Objekt gewonnen hat oder nicht. Ein Bieter kann weder die Gebote der anderen Bieter beobachten, noch kann er beobachten, welche Bieter an der Auktion teilgenommen haben und wer gewonnen hat - solange er nicht selbst das Objekt erhält. Wir zeigen, dass in dem unendlich wiederholten Spiel für hinreichend geduldige Bieter Kollusion möglich ist. Es existiert zwar ein eindeutiges Gleichgewicht in öffentlichen Strategien, in welchem die Bieter in jeder Periode kompetitiv bieten, aber es gibt einfache Nash-Gleichgewichte in privaten Strategien, die Bieterrotation durchsetzen. Wir zeigen auch, dass Bieterrotation das Ergebnis eines perfekt bayesianischen Gleichgewichtes sein kann. Nash-Gleichgewichte, die höhere erwartete Gewinne als Bieterrotation erzielen, sind nur ein wenig komplexer. Das Hauptergebnis ist die Konstruktion von (essentiell) perfekt bayesianischen Gleichgewichten, welche höhere Gewinne als Bieterrotation erzielen. Nach Abweichungen vom Gleichgewichtspfad, müssen die Bieter in diesen Gleichgewichten komplizierte Rückschlüsse auf das Verhalten Ihrer Wettbewerber ziehen. So weiß ein Bieter nach bestimmten Abweichungen nicht, ob diese von seinen Mitspielern beobachtet wurden, und hat ein Interesse daran, durch strategisches experimentieren das Bietverhalten seiner Rivalen kennen zu lernen." (Autorenreferat

Thresholding at the monopoly price: an agnostic way to improve bidding strategies in revenue-maximizing auctions

We address the problem of improving bidders' strategies in prior-dependent revenue-maximizing auctions. We introduce a simple and generic method to design novel bidding strategies if the seller uses past bids to optimize her mechanism. This strategy works with general value distributions, with asymmetric bidders and for different revenue-maximizing mechanisms. Furthermore, it can be made robust to sample approximation errors on the seller part. This results in a large increase in utility for bidders whether they have a full or partial knowledge of their competitors. In the case where the buyer has no information about the competition, we propose a simple and agnostic strategy that is robust to mechanism changes and local (as opposed to global) optimization of e.g. reserve prices by the seller. In textbook-style examples, for instance with uniform value distributions and two bidders, this no-side-information and mechanism-independent strategy yields an enormous 57% increase in buyer utility for lazy second price auctions with no reserves. In the i.i.d symmetric case, we show existence and uniqueness of a Nash equilibrium in the class of strategy we consider for lazy second price auctions, as well as the corresponding explicit shading strategies. Our approach also works for Myerson auctions for instance. At this Nash equilibrium, buyer's utility is the same as in a second price auction with no reserve. Our approach also yields optimal solutions when buyer are constrained in the class of shading strategies they can use, a realistic constraint in practical applications. The heart of our approach is to see optimal auctions in practice as a Stackelberg game where the buyer is the leader, as he is the first one to move (here bid) when the seller is the follower as she has no prior information on the bidder

Supply Side Optimisation in Online Display Advertising

On the Internet there are publishers (the supply side) who provide free contents (e.g., news) and services (e.g., email) to attract users. Publishers get paid by selling ad displaying opportunities (i.e., impressions) to advertisers. Advertisers then sell products to users who are converted by ads. Better supply side revenue allows more free content and services to be created, thus, benefiting the entire online advertising ecosystem. This thesis addresses several optimisation problems for the supply side. When a publisher creates an ad-supported website, he needs to decide the percentage of ads first. The thesis reports a large-scale empirical study of Internet ad density over past seven years, then presents a model that includes many factors, especially the competition among similar publishers, and gives an optimal dynamic ad density that generates the maximum revenue over time. This study also unveils the tragedy of the commons in online advertising where users' attention has been overgrazed which results in a global sub-optimum. After deciding the ad density, the publisher retrieves ads from various sources, including contracts, ad networks, and ad exchanges. This forms an exploration-exploitation problem when ad sources are typically unknown before trail. This problem is modelled using Partially Observable Markov Decision Process (POMDP), and the exploration efficiency is increased by utilising the correlation of ads. The proposed method reports 23.4% better than the best performing baseline in the real-world data based experiments. Since some ad networks allow (or expect) an input of keywords, the thesis also presents an adaptive keyword extraction system using BM25F algorithm and the multi-armed bandits model. This system has been tested by a domain service provider in crowdsourcing based experiments. If the publisher selects a Real-Time Bidding (RTB) ad source, he can use reserve price to manipulate auctions for better payoff. This thesis proposes a simplified game model that considers the competition between seller and buyer to be one-shot instead of repeated and gives heuristics that can be easily implemented. The model has been evaluated in a production environment and reported 12.3% average increase of revenue. The documentation of a prototype system for reserve price optimisation is also presented in the appendix of the thesis

Auction Fever: Theory and Experimental Evidence

It is not a secret that certain auction formats yield on average higher prices than others. The phenomenon that dynamic auctions are more likely to elicit higher bids than static one-shot auctions is often associated with the term ''auction fever.'' On a psychological level, we consider the so-called pseudo-endowment effect as largely responsible for peoples’ tendency to submit higher bids, potentially amplified by the source-dependence effect. The phenomenon of auction fever is replicated in an experimental investigation of different auction formats within a private values framework where bidders have private but incomplete knowledge of their valuation for a hypothetical good. We suggest this assumption to be more realistic than definite private values, as assumed in the traditional IPV model. An additional experimental investigation within the traditional IPV framework does not either reveal any indication for the appearance of auction fever. On the basis of our experimental observations we present a model of reference-dependent utility theory that comprehends the phenomenon by assuming that bidders' reference points are shifted by the pseudo-endowment and the source-dependence effect.

Learning Prices for Repeated Auctions with Strategic Buyers

Inspired by real-time ad exchanges for online display advertising, we consider the problem of inferring a buyer's value distribution for a good when the buyer is repeatedly interacting with a seller through a posted-price mechanism. We model the buyer as a strategic agent, whose goal is to maximize her long-term surplus, and we are interested in mechanisms that maximize the seller's long-term revenue. We define the natural notion of strategic regret --- the lost revenue as measured against a truthful (non-strategic) buyer. We present seller algorithms that are no-(strategic)-regret when the buyer discounts her future surplus --- i.e. the buyer prefers showing advertisements to users sooner rather than later. We also give a lower bound on strategic regret that increases as the buyer's discounting weakens and shows, in particular, that any seller algorithm will suffer linear strategic regret if there is no discounting.Comment: Neural Information Processing Systems (NIPS 2013

Selling to a No-Regret Buyer

We consider the problem of a single seller repeatedly selling a single item to a single buyer (specifically, the buyer has a value drawn fresh from known distribution $D$ in every round). Prior work assumes that the buyer is fully rational and will perfectly reason about how their bids today affect the seller's decisions tomorrow. In this work we initiate a different direction: the buyer simply runs a no-regret learning algorithm over possible bids. We provide a fairly complete characterization of optimal auctions for the seller in this domain. Specifically: - If the buyer bids according to EXP3 (or any "mean-based" learning algorithm), then the seller can extract expected revenue arbitrarily close to the expected welfare. This auction is independent of the buyer's valuation $D$, but somewhat unnatural as it is sometimes in the buyer's interest to overbid. - There exists a learning algorithm $\mathcal{A}$ such that if the buyer bids according to $\mathcal{A}$ then the optimal strategy for the seller is simply to post the Myerson reserve for $D$ every round. - If the buyer bids according to EXP3 (or any "mean-based" learning algorithm), but the seller is restricted to "natural" auction formats where overbidding is dominated (e.g. Generalized First-Price or Generalized Second-Price), then the optimal strategy for the seller is a pay-your-bid format with decreasing reserves over time. Moreover, the seller's optimal achievable revenue is characterized by a linear program, and can be unboundedly better than the best truthful auction yet simultaneously unboundedly worse than the expected welfare

Private Information in Sequential Common-Value Auctions

We study an infinitely-repeated ?rst-price auction with common values. Initially, bid- ders receive independent private signals about the objects' value, which itself does not change over time. Learning occurs only through observation of the bids. Under one-sided incomplete information, this information is eventually revealed and the seller extracts es- sentially the entire rent (for large discount factors). Both players?payo¤s tend to zero as the discount factor tends to one. However, the uninformed bidder does relatively better than the informed bidder. We discuss the case of two-sided incomplete information, and argue that, under a Markovian re?nement, the outcome is pooling: information is revealed only insofar as it does not affect prices. Bidders submit a common, low bid in the tradition of collusion without conspiracy.repeated game with incomplete information; private information; ratchet effect; first-price auction; dynamic auctions