Learning Adaptive Display Exposure for Real-Time Advertising

In E-commerce advertising, where product recommendations and product ads are presented to users simultaneously, the traditional setting is to display ads at fixed positions. However, under such a setting, the advertising system loses the flexibility to control the number and positions of ads, resulting in sub-optimal platform revenue and user experience. Consequently, major e-commerce platforms (e.g., Taobao.com) have begun to consider more flexible ways to display ads. In this paper, we investigate the problem of advertising with adaptive exposure: can we dynamically determine the number and positions of ads for each user visit under certain business constraints so that the platform revenue can be increased? More specifically, we consider two types of constraints: request-level constraint ensures user experience for each user visit, and platform-level constraint controls the overall platform monetization rate. We model this problem as a Constrained Markov Decision Process with per-state constraint (psCMDP) and propose a constrained two-level reinforcement learning approach to decompose the original problem into two relatively independent sub-problems. To accelerate policy learning, we also devise a constrained hindsight experience replay mechanism. Experimental evaluations on industry-scale real-world datasets demonstrate the merits of our approach in both obtaining higher revenue under the constraints and the effectiveness of the constrained hindsight experience replay mechanism.Comment: accepted by CIKM201

PREDICTING ONLINE USER BEHAVIOR BASED ON REAL-TIME ADVERTISING DATA

Generating economic value from big data is a challenge for many companies these days. On the Internet, a major source of big data is structured and unstructured data generated by users. Companies can use this data to better understand patterns of user behavior and to improve marketing decisions. In this paper, we focus on data generated in real-time advertising where billions of advertising slots are sold by auction. The auctions are triggered by user activity on websites that use this form of advertising to sell their advertising slots. During an auction, so-called bid requests are sent to advertisers who bid for the advertising slots. We develop a model that uses bid requests to predict whether a user will visit a certain website during his or her user journey. These predictions can be used by advertisers to derive user interests early in the sales funnel and, thus, to increase profits from branding campaigns. By iteratively applying a Bayesian multinomial logistic model to data from a case study, we show how to constantly improve the predictive accuracy of the model. We calculate the economic value of our model and show that it can be beneficial for advertisers in the context of cross-channel advertising

REAL TIME ADVERTISING ON SOCIAL NETWORKS

Ph.DDOCTOR OF PHILOSOPH

Statistical modelling of clickstream behaviour to inform real-time advertising decisions

Online user browsing generates vast quantities of typically unexploited data. Investigating this data and uncovering the valuable information it contains can be of substantial value to online businesses, and statistics plays a key role in this process. The data takes the form of an anonymous digital footprint associated with each unique visitor, resulting in $10^{6}$ unique profiles across $10^{7}$ individual page visits on a daily basis. Exploring, cleaning and transforming data of this scale and high dimensionality (2TB+ of memory) is particularly challenging, and requires cluster computing. We outline a variable selection method to summarise clickstream behaviour with a single value, and make comparisons to other dimension reduction techniques. We illustrate how to apply generalised linear models and zero-inflated models to predict sponsored search advert clicks based on keywords. We consider the problem of predicting customer purchases (known as conversions), from the customer’s journey or clickstream, which is the sequence of pages seen during a single visit to a website. We consider each page as a discrete state with probabilities of transitions between the pages, providing the basis for a simple Markov model. Further, Hidden Markov models (HMMs) are applied to relate the observed clickstream to a sequence of hidden states, uncovering meta-states of user activity. We can also apply conventional logistic regression to model conversions in terms of summaries of the profile’s browsing behaviour and incorporate both into a set of tools to solve a wide range of conversion types where we can directly compare the predictive capability of each model. In real-time, predicting profiles that are likely to follow similar behaviour patterns to known conversions, will have a critical impact on targeted advertising. We illustrate these analyses with results from real data collected by an Audience Management Platform (AMP) - Carbon

Parallel Ranking of Ads and Creatives in Real-Time Advertising Systems

"Creativity is the heart and soul of advertising services". Effective creatives can create a win-win scenario: advertisers can reach target users and achieve marketing objectives more effectively, users can more quickly find products of interest, and platforms can generate more advertising revenue. With the advent of AI-Generated Content, advertisers now can produce vast amounts of creative content at a minimal cost. The current challenge lies in how advertising systems can select the most pertinent creative in real-time for each user personally. Existing methods typically perform serial ranking of ads or creatives, limiting the creative module in terms of both effectiveness and efficiency. In this paper, we propose for the first time a novel architecture for online parallel estimation of ads and creatives ranking, as well as the corresponding offline joint optimization model. The online architecture enables sophisticated personalized creative modeling while reducing overall latency. The offline joint model for CTR estimation allows mutual awareness and collaborative optimization between ads and creatives. Additionally, we optimize the offline evaluation metrics for the implicit feedback sorting task involved in ad creative ranking. We conduct extensive experiments to compare ours with two state-of-the-art approaches. The results demonstrate the effectiveness of our approach in both offline evaluations and real-world advertising platforms online in terms of response time, CTR, and CPM.Comment: 9 pages, 4 figures, AAAI202

Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising

Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by ad placements, advertisers not only need to estimate the relevance between the ads and user's interests, but most importantly require a strategic response with respect to other advertisers bidding in the market. In this paper, we formulate bidding optimization with multi-agent reinforcement learning. To deal with a large number of advertisers, we propose a clustering method and assign each cluster with a strategic bidding agent. A practical Distributed Coordinated Multi-Agent Bidding (DCMAB) has been proposed and implemented to balance the tradeoff between the competition and cooperation among advertisers. The empirical study on our industry-scaled real-world data has demonstrated the effectiveness of our methods. Our results show cluster-based bidding would largely outperform single-agent and bandit approaches, and the coordinated bidding achieves better overall objectives than purely self-interested bidding agents

Fuzzy Content Mining for Targeted Advertisement

Content-targeted advertising system is becoming an increasingly important part of the funding source of free web services. Highly efficient content analysis is the pivotal key of such a system. This project aims to establish a content analysis engine involving fuzzy logic that is able to automatically analyze real user-posted Web documents such as blog entries. Based on the analysis result, the system matches and retrieves the most appropriate Web advertisements. The focus and complexity is on how to better estimate and acquire the keywords that represent a given Web document. Fuzzy Web mining concept will be applied to synthetically consider multiple factors of Web content. A Fuzzy Ranking System is established based on certain fuzzy (and some crisp) rules, fuzzy sets, and membership functions to get the best candidate keywords. Once it is has obtained the keywords, the system will retrieve corresponding advertisements from certain providers through Web services as matched advertisements, similarly to retrieving a products list from Amazon.com. In 87% of the cases, the results of this system can match the accuracy of the Google Adwords system. Furthermore, this expandable system will also be a solid base for further research and development on this topic

SAFE Newsletter : 2013, Q3

Research: Joachim Weber, Benjamin Loos, Steffen Meyer, Andreas Hackethal "Individual Investors' Trading Motives and Security Selling Behavior" Ignazio Angeloni, Ester Faia "Monetary Policy and Prudential Regulations with Bank Runs" Helmut Siekmann "Legal Limits to Quantitative Easing" Policy Margit Vanberg "SAFE Summer Academy 2013 on 'International Financial Stability'" Guest Commentary Peter Praet "Cooperation between the ECB and Academia

Pricing average price advertising options when underlying spot market prices are discontinuous

Advertising options have been recently studied as a special type of guaranteed contracts in online advertising, which are an alternative sales mechanism to real-time auctions. An advertising option is a contract which gives its buyer a right but not obligation to enter into transactions to purchase page views or link clicks at one or multiple pre-specified prices in a specific future period. Different from typical guaranteed contracts, the option buyer pays a lower upfront fee but can have greater flexibility and more control of advertising. Many studies on advertising options so far have been restricted to the situations where the option payoff is determined by the underlying spot market price at a specific time point and the price evolution over time is assumed to be continuous. The former leads to a biased calculation of option payoff and the latter is invalid empirically for many online advertising slots. This paper addresses these two limitations by proposing a new advertising option pricing framework. First, the option payoff is calculated based on an average price over a specific future period. Therefore, the option becomes path-dependent. The average price is measured by the power mean, which contains several existing option payoff functions as its special cases. Second, jump-diffusion stochastic models are used to describe the movement of the underlying spot market price, which incorporate several important statistical properties including jumps and spikes, non-normality, and absence of autocorrelations. A general option pricing algorithm is obtained based on Monte Carlo simulation. In addition, an explicit pricing formula is derived for the case when the option payoff is based on the geometric mean. This pricing formula is also a generalized version of several other option pricing models discussed in related studies.Comment: IEEE Transactions on Knowledge and Data Engineering, 201