15 research outputs found
Sequential Selection of Correlated Ads by POMDPs
Online advertising has become a key source of revenue for both web search
engines and online publishers. For them, the ability of allocating right ads to
right webpages is critical because any mismatched ads would not only harm web
users' satisfactions but also lower the ad income. In this paper, we study how
online publishers could optimally select ads to maximize their ad incomes over
time. The conventional offline, content-based matching between webpages and ads
is a fine start but cannot solve the problem completely because good matching
does not necessarily lead to good payoff. Moreover, with the limited display
impressions, we need to balance the need of selecting ads to learn true ad
payoffs (exploration) with that of allocating ads to generate high immediate
payoffs based on the current belief (exploitation). In this paper, we address
the problem by employing Partially observable Markov decision processes
(POMDPs) and discuss how to utilize the correlation of ads to improve the
efficiency of the exploration and increase ad incomes in a long run. Our
mathematical derivation shows that the belief states of correlated ads can be
naturally updated using a formula similar to collaborative filtering. To test
our model, a real world ad dataset from a major search engine is collected and
categorized. Experimenting over the data, we provide an analyse of the effect
of the underlying parameters, and demonstrate that our algorithms significantly
outperform other strong baselines
Real-Time Bidding by Reinforcement Learning in Display Advertising
The majority of online display ads are served through real-time bidding (RTB)
--- each ad display impression is auctioned off in real-time when it is just
being generated from a user visit. To place an ad automatically and optimally,
it is critical for advertisers to devise a learning algorithm to cleverly bid
an ad impression in real-time. Most previous works consider the bid decision as
a static optimization problem of either treating the value of each impression
independently or setting a bid price to each segment of ad volume. However, the
bidding for a given ad campaign would repeatedly happen during its life span
before the budget runs out. As such, each bid is strategically correlated by
the constrained budget and the overall effectiveness of the campaign (e.g., the
rewards from generated clicks), which is only observed after the campaign has
completed. Thus, it is of great interest to devise an optimal bidding strategy
sequentially so that the campaign budget can be dynamically allocated across
all the available impressions on the basis of both the immediate and future
rewards. In this paper, we formulate the bid decision process as a
reinforcement learning problem, where the state space is represented by the
auction information and the campaign's real-time parameters, while an action is
the bid price to set. By modeling the state transition via auction competition,
we build a Markov Decision Process framework for learning the optimal bidding
policy to optimize the advertising performance in the dynamic real-time bidding
environment. Furthermore, the scalability problem from the large real-world
auction volume and campaign budget is well handled by state value approximation
using neural networks.Comment: WSDM 201
A dynamic pricing model for unifying programmatic guarantee and real-time bidding in display advertising
There are two major ways of selling impressions in display advertising. They
are either sold in spot through auction mechanisms or in advance via guaranteed
contracts. The former has achieved a significant automation via real-time
bidding (RTB); however, the latter is still mainly done over the counter
through direct sales. This paper proposes a mathematical model that allocates
and prices the future impressions between real-time auctions and guaranteed
contracts. Under conventional economic assumptions, our model shows that the
two ways can be seamless combined programmatically and the publisher's revenue
can be maximized via price discrimination and optimal allocation. We consider
advertisers are risk-averse, and they would be willing to purchase guaranteed
impressions if the total costs are less than their private values. We also
consider that an advertiser's purchase behavior can be affected by both the
guaranteed price and the time interval between the purchase time and the
impression delivery date. Our solution suggests an optimal percentage of future
impressions to sell in advance and provides an explicit formula to calculate at
what prices to sell. We find that the optimal guaranteed prices are dynamic and
are non-decreasing over time. We evaluate our method with RTB datasets and find
that the model adopts different strategies in allocation and pricing according
to the level of competition. From the experiments we find that, in a less
competitive market, lower prices of the guaranteed contracts will encourage the
purchase in advance and the revenue gain is mainly contributed by the increased
competition in future RTB. In a highly competitive market, advertisers are more
willing to purchase the guaranteed contracts and thus higher prices are
expected. The revenue gain is largely contributed by the guaranteed selling.Comment: Chen, Bowei and Yuan, Shuai and Wang, Jun (2014) A dynamic pricing
model for unifying programmatic guarantee and real-time bidding in display
advertising. In: The Eighth International Workshop on Data Mining for Online
Advertising, 24 - 27 August 2014, New York Cit
A lattice framework for pricing display advertisement options with the stochastic volatility underlying model
Advertisement (abbreviated ad) options are a recent development in online
advertising. Simply, an ad option is a first look contract in which a publisher
or search engine grants an advertiser a right but not obligation to enter into
transactions to purchase impressions or clicks from a specific ad slot at a
pre-specified price on a specific delivery date. Such a structure provides
advertisers with more flexibility of their guaranteed deliveries. The valuation
of ad options is an important topic and previous studies on ad options pricing
have been mostly restricted to the situations where the underlying prices
follow a geometric Brownian motion (GBM). This assumption is reasonable for
sponsored search; however, some studies have also indicated that it is not
valid for display advertising. In this paper, we address this issue by
employing a stochastic volatility (SV) model and discuss a lattice framework to
approximate the proposed SV model in option pricing. Our developments are
validated by experiments with real advertising data: (i) we find that the SV
model has a better fitness over the GBM model; (ii) we validate the proposed
lattice model via two sequential Monte Carlo simulation methods; (iii) we
demonstrate that advertisers are able to flexibly manage their guaranteed
deliveries by using the proposed options, and publishers can have an increased
revenue when some of their inventories are sold via ad options.Comment: Bowei Chen and Jun Wang. A lattice framework for pricing display
advertisement options with the stochastic volatility underlying model.
Electronic Commerce Research and Applications, 2015, Volume 14, Issue 6,
pages 465-479, ISSN: 1567-422
Optimal Real-Time Bidding Strategies
The ad-trading desks of media-buying agencies are increasingly relying on
complex algorithms for purchasing advertising inventory. In particular,
Real-Time Bidding (RTB) algorithms respond to many auctions -- usually Vickrey
auctions -- throughout the day for buying ad-inventory with the aim of
maximizing one or several key performance indicators (KPI). The optimization
problems faced by companies building bidding strategies are new and interesting
for the community of applied mathematicians. In this article, we introduce a
stochastic optimal control model that addresses the question of the optimal
bidding strategy in various realistic contexts: the maximization of the
inventory bought with a given amount of cash in the framework of audience
strategies, the maximization of the number of conversions/acquisitions with a
given amount of cash, etc. In our model, the sequence of auctions is modeled by
a Poisson process and the \textit{price to beat} for each auction is modeled by
a random variable following almost any probability distribution. We show that
the optimal bids are characterized by a Hamilton-Jacobi-Bellman equation, and
that almost-closed form solutions can be found by using a fluid limit.
Numerical examples are also carried out
Pricing average price advertising options when underlying spot market prices are discontinuous
Advertising options have been recently studied as a special type of
guaranteed contracts in online advertising, which are an alternative sales
mechanism to real-time auctions. An advertising option is a contract which
gives its buyer a right but not obligation to enter into transactions to
purchase page views or link clicks at one or multiple pre-specified prices in a
specific future period. Different from typical guaranteed contracts, the option
buyer pays a lower upfront fee but can have greater flexibility and more
control of advertising. Many studies on advertising options so far have been
restricted to the situations where the option payoff is determined by the
underlying spot market price at a specific time point and the price evolution
over time is assumed to be continuous. The former leads to a biased calculation
of option payoff and the latter is invalid empirically for many online
advertising slots. This paper addresses these two limitations by proposing a
new advertising option pricing framework. First, the option payoff is
calculated based on an average price over a specific future period. Therefore,
the option becomes path-dependent. The average price is measured by the power
mean, which contains several existing option payoff functions as its special
cases. Second, jump-diffusion stochastic models are used to describe the
movement of the underlying spot market price, which incorporate several
important statistical properties including jumps and spikes, non-normality, and
absence of autocorrelations. A general option pricing algorithm is obtained
based on Monte Carlo simulation. In addition, an explicit pricing formula is
derived for the case when the option payoff is based on the geometric mean.
This pricing formula is also a generalized version of several other option
pricing models discussed in related studies.Comment: IEEE Transactions on Knowledge and Data Engineering, 201
Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising
Real-time bidding (RTB) is an important mechanism in online display
advertising, where a proper bid for each page view plays an essential role for
good marketing results. Budget constrained bidding is a typical scenario in RTB
where the advertisers hope to maximize the total value of the winning
impressions under a pre-set budget constraint. However, the optimal bidding
strategy is hard to be derived due to the complexity and volatility of the
auction environment. To address these challenges, in this paper, we formulate
budget constrained bidding as a Markov Decision Process and propose a
model-free reinforcement learning framework to resolve the optimization
problem. Our analysis shows that the immediate reward from environment is
misleading under a critical resource constraint. Therefore, we innovate a
reward function design methodology for the reinforcement learning problems with
constraints. Based on the new reward design, we employ a deep neural network to
learn the appropriate reward so that the optimal policy can be learned
effectively. Different from the prior model-based work, which suffers from the
scalability problem, our framework is easy to be deployed in large-scale
industrial applications. The experimental evaluations demonstrate the
effectiveness of our framework on large-scale real datasets.Comment: In The 27th ACM International Conference on Information and Knowledge
Management (CIKM 18), October 22-26, 2018, Torino, Italy. ACM, New York, NY,
USA, 9 page
Multi-keyword multi-click advertisement option contracts for sponsored search
In sponsored search, advertisement (abbreviated ad) slots are usually sold by
a search engine to an advertiser through an auction mechanism in which
advertisers bid on keywords. In theory, auction mechanisms have many desirable
economic properties. However, keyword auctions have a number of limitations
including: the uncertainty in payment prices for advertisers; the volatility in
the search engine's revenue; and the weak loyalty between advertiser and search
engine. In this paper we propose a special ad option that alleviates these
problems. In our proposal, an advertiser can purchase an option from a search
engine in advance by paying an upfront fee, known as the option price. He then
has the right, but no obligation, to purchase among the pre-specified set of
keywords at the fixed cost-per-clicks (CPCs) for a specified number of clicks
in a specified period of time. The proposed option is closely related to a
special exotic option in finance that contains multiple underlying assets
(multi-keyword) and is also multi-exercisable (multi-click). This novel
structure has many benefits: advertisers can have reduced uncertainty in
advertising; the search engine can improve the advertisers' loyalty as well as
obtain a stable and increased expected revenue over time. Since the proposed ad
option can be implemented in conjunction with the existing keyword auctions,
the option price and corresponding fixed CPCs must be set such that there is no
arbitrage between the two markets. Option pricing methods are discussed and our
experimental results validate the development. Compared to keyword auctions, a
search engine can have an increased expected revenue by selling an ad option.Comment: Chen, Bowei and Wang, Jun and Cox, Ingemar J. and Kankanhalli, Mohan
S. (2015) Multi-keyword multi-click advertisement option contracts for
sponsored search. ACM Transactions on Intelligent Systems and Technology, 7
(1). pp. 1-29. ISSN: 2157-690