Real-Time Bidding by Reinforcement Learning in Display Advertising

The majority of online display ads are served through real-time bidding (RTB) --- each ad display impression is auctioned off in real-time when it is just being generated from a user visit. To place an ad automatically and optimally, it is critical for advertisers to devise a learning algorithm to cleverly bid an ad impression in real-time. Most previous works consider the bid decision as a static optimization problem of either treating the value of each impression independently or setting a bid price to each segment of ad volume. However, the bidding for a given ad campaign would repeatedly happen during its life span before the budget runs out. As such, each bid is strategically correlated by the constrained budget and the overall effectiveness of the campaign (e.g., the rewards from generated clicks), which is only observed after the campaign has completed. Thus, it is of great interest to devise an optimal bidding strategy sequentially so that the campaign budget can be dynamically allocated across all the available impressions on the basis of both the immediate and future rewards. In this paper, we formulate the bid decision process as a reinforcement learning problem, where the state space is represented by the auction information and the campaign's real-time parameters, while an action is the bid price to set. By modeling the state transition via auction competition, we build a Markov Decision Process framework for learning the optimal bidding policy to optimize the advertising performance in the dynamic real-time bidding environment. Furthermore, the scalability problem from the large real-world auction volume and campaign budget is well handled by state value approximation using neural networks.Comment: WSDM 201

Statistical Arbitrage Mining for Display Advertising

We study and formulate arbitrage in display advertising. Real-Time Bidding (RTB) mimics stock spot exchanges and utilises computers to algorithmically buy display ads per impression via a real-time auction. Despite the new automation, the ad markets are still informationally inefficient due to the heavily fragmented marketplaces. Two display impressions with similar or identical effectiveness (e.g., measured by conversion or click-through rates for a targeted audience) may sell for quite different prices at different market segments or pricing schemes. In this paper, we propose a novel data mining paradigm called Statistical Arbitrage Mining (SAM) focusing on mining and exploiting price discrepancies between two pricing schemes. In essence, our SAMer is a meta-bidder that hedges advertisers' risk between CPA (cost per action)-based campaigns and CPM (cost per mille impressions)-based ad inventories; it statistically assesses the potential profit and cost for an incoming CPM bid request against a portfolio of CPA campaigns based on the estimated conversion rate, bid landscape and other statistics learned from historical data. In SAM, (i) functional optimisation is utilised to seek for optimal bidding to maximise the expected arbitrage net profit, and (ii) a portfolio-based risk management solution is leveraged to reallocate bid volume and budget across the set of campaigns to make a risk and return trade-off. We propose to jointly optimise both components in an EM fashion with high efficiency to help the meta-bidder successfully catch the transient statistical arbitrage opportunities in RTB. Both the offline experiments on a real-world large-scale dataset and online A/B tests on a commercial platform demonstrate the effectiveness of our proposed solution in exploiting arbitrage in various model settings and market environments.Comment: In the proceedings of the 21st ACM SIGKDD international conference on Knowledge discovery and data mining (KDD 2015

Smart Pacing for Effective Online Ad Campaign Optimization

In targeted online advertising, advertisers look for maximizing campaign performance under delivery constraint within budget schedule. Most of the advertisers typically prefer to impose the delivery constraint to spend budget smoothly over the time in order to reach a wider range of audiences and have a sustainable impact. Since lots of impressions are traded through public auctions for online advertising today, the liquidity makes price elasticity and bid landscape between demand and supply change quite dynamically. Therefore, it is challenging to perform smooth pacing control and maximize campaign performance simultaneously. In this paper, we propose a smart pacing approach in which the delivery pace of each campaign is learned from both offline and online data to achieve smooth delivery and optimal performance goals. The implementation of the proposed approach in a real DSP system is also presented. Experimental evaluations on both real online ad campaigns and offline simulations show that our approach can effectively improve campaign performance and achieve delivery goals.Comment: KDD'15, August 10-13, 2015, Sydney, NSW, Australi

Display Advertising with Real-Time Bidding (RTB) and Behavioural Targeting

The most significant progress in recent years in online display advertising is what is known as the Real-Time Bidding (RTB) mechanism to buy and sell ads. RTB essentially facilitates buying an individual ad impression in real time while it is still being generated from a user’s visit. RTB not only scales up the buying process by aggregating a large amount of available inventories across publishers but, most importantly, enables direct targeting of individual users. As such, RTB has fundamentally changed the landscape of digital marketing. Scientifically, the demand for automation, integration and optimisation in RTB also brings new research opportunities in information retrieval, data mining, machine learning and other related fields. In this monograph, an overview is given of the fundamental infrastructure, algorithms, and technical solutions of this new frontier of computational advertising. The covered topics include user response prediction, bid landscape forecasting, bidding algorithms, revenue optimisation, statistical arbitrage, dynamic pricing, and ad fraud detection

Display Advertising with Real-Time Bidding (RTB) and Behavioural Targeting

The most significant progress in recent years in online display advertising is what is known as the Real-Time Bidding (RTB) mechanism to buy and sell ads. RTB essentially facilitates buying an individual ad impression in real time while it is still being generated from a user’s visit. RTB not only scales up the buying process by aggregating a large amount of available inventories across publishers but, most importantly, enables direct targeting of individual users. As such, RTB has fundamentally changed the landscape of digital marketing. Scientifically, the demand for automation, integration and optimisation in RTB also brings new research opportunities in information retrieval, data mining, machine learning and other related fields. In this monograph, an overview is given of the fundamental infrastructure, algorithms, and technical solutions of this new frontier of computational advertising. The covered topics include user response prediction, bid landscape forecasting, bidding algorithms, revenue optimisation, statistical arbitrage, dynamic pricing, and ad fraud detection

Demystifying Advertising Campaign Bid Recommendation: A Constraint target CPA Goal Optimization

In cost-per-click (CPC) or cost-per-impression (CPM) advertising campaigns, advertisers always run the risk of spending the budget without getting enough conversions. Moreover, the bidding on advertising inventory has few connections with propensity one that can reach to target cost-per-acquisition (tCPA) goals. To address this problem, this paper presents a bid optimization scenario to achieve the desired tCPA goals for advertisers. In particular, we build the optimization engine to make a decision by solving the rigorously formalized constrained optimization problem, which leverages the bid landscape model learned from rich historical auction data using non-parametric learning. The proposed model can naturally recommend the bid that meets the advertisers' expectations by making inference over advertisers' historical auction behaviors, which essentially deals with the data challenges commonly faced by bid landscape modeling: incomplete logs in auctions, and uncertainty due to the variation and fluctuations in advertising bidding behaviors. The bid optimization model outperforms the baseline methods on real-world campaigns, and has been applied into a wide range of scenarios for performance improvement and revenue liftup

PERFORMANCE BASED PRICING MODELS IN ONLINE ADVERTISING: CLICK PER MILLE (CPM) AND COST PER CLICK (CPC)

The project is focusing on enhancing pricing scheme for Publishers on online advertising method thru Click Per Mille (CPM) and Cost Per Click (CPC). Both have strength and weakness that might affect profitability / revenue on the Publishers side. Dengler, Brian. (2011) state that Internet advertising revenues jumped 23 percent in the United State for the first quarter of 2011 over the same period last year, according to figures released May 27, 2011 by the Interactive Advertising Bureau (IAB) and PricewaterhouseCoopers (PwC). In the other words, online advertising is the best way to making money for the player within the industry; Publishers and Advertisers. From the Malaysian perspectives, in order to the players in the online advertising industry is following the guidelines and code of practices, they must referring and dealing with Advertising Standards Authority Malaysia who provide Malaysian Code of Advertising Practice. The Advertising Standards Authority Malaysia (ASA) is the independent body responsible for ensuring that the self-regulatory system works in the public interest. The ASA’s activities include investigating complaints and copy advice on your advertising. The Malaysian Code of Advertising Practice has the support of the following organizations whose representatives constitute the Advertising Standards Authority Malaysia. The Malaysian Code of Advertising Practice has the support of the following organizations whose representatives constitute the Advertising Standards Authority Malaysia; Association of Accredited Advertising Agents Malaysia, Malaysian Advertisers Association, Malaysian Newspaper Publishers Association, and Media Specialists Association. Yuan, S., Abidin, A.Z., Sloan, M., and Wang, J. (2012) state that towards this goal mathematically well grounded Computational Advertising methods are becoming necessary and will continue to develop as a fundamental tool towards the Web. As a vibrant new discipline, Internet advertising requires effort from different research domains including Information Retrieval, Machine Learning, Data Mining and Analytic, Statistics, Economics, and even Psychology to predict and understand user vi behaviors. In this paper, we provide a comprehensive survey on Internet advertising, discussing and classifying the research issues, identifying the recent technologies, and suggesting its future directions. To have a comprehensive picture, we start with a brief history, introduction, and classification of the industry and present a schematic view of the new advertising ecosystem. We then introduce four major participants, namely advertisers, online publishers, ad exchanges and web users; and through analyzing and discussing the major research problems and existing solutions from their perspectives respectively, we discover and aggregate the fundamental problems that characterize the newly formed researched and capture its potential future prospects. Performance based advertising is a form of advertising in which the purchaser pays only when there are measurable results. Performance based advertising is becoming more common with the spread of electronic media, notably the Internet, where it is possible to measure user actions resulting from advertisement. Publishers act as the body who will appoint another body (Advertisers) to advertise their products and services to the publics. However, there still inefficiency between Click Per Mille (CPM) and Cost Per Click (CPC). This project will do research, analyze, evaluate, calculate, enhance and develop current advertising approach for a better performance based pricing models. As the result, the new improvised pricing model where as CPM, CPC or combination of both (Hybrid) might contribute much to the small business and entrepreneurship player’s in Malaysia especially to the Bumiputra’s. The advanced and effective pricing model will help much to them in order to promote and increased the amount of business revenue by applying and practicing the new pricing model on online advertising. The collaboration between Publishers and Advertisers also important to make sure the direct negotiation is realized between them (No more agents) so the win - win situation could be the objective of the pricing model project

Polysemy in Advertising

The article reviews the conceptual foundations of advertising polysemy – the occurrence of different interpretations for the same advertising message. We discuss how disciplines as diverse as psychology, semiotics and literary theory have dealt with the issue of polysemy, and provide translations and integration among these multiple perspectives. From such review we draw recurrent themes to foster future research in the area and to show how seemingly opposed methodological and theoretical perspectives complement and extend each other. Implications for advertising research and practice are discussed.Advertising;Polysemy;Semiotics