Estimating Causal Installed-Base Effects: A Bias-Correction Approach

New empirical models of consumer demand that incorporate social preferences, observational learning, word-of-mouth or network effects have the feature that the adoption of others in the reference group - the “installed-base” - has a causal effect on current adoption behavior. Estimation of such causal installed-base effects is challenging due to the potential for spurious correlation between the adoption of agents, arising from endogenous assortive matching into social groups (or homophily) and from the existence of unobservables across agents that are correlated. In the absence of experimental variation, the preferred solution is to control for these using a rich specification of fixed-effects, which is feasible with panel data. We show that fixedeffects estimators of this sort are inconsistent in the presence of installed-base effects; in our simulations, random-effects specifications perform even worse. Our analysis reveals the tension faced by the applied empiricist in this area: a rich control for unobservables increases the credibility of the reported causal effects, but the incorporation of these controls introduces biases of a new kind in this class of models. We present two solutions: an instrumental variable approach, and a new bias-correction approach, both of which deliver consistent estimates of causal installed-base effects. The bias-correction approach is tractable in this context because we are able to exploit the structure of the problem to solve analytically for the asymptotic bias of the installed-base estimator, and to incorporate it into the estimation routine. Our approach has implications for the measurement of social effects using non-experimental data, and for measuring marketing-mix effects in the presence of state-dependence in demand, more generally. Our empirical application to the adoption of the Toyota Prius Hybrid in California reveals evidence for social influence in diffusion, and demonstrates the importance of incorporating proper controls for the biases we identify

Online Evaluation of Audiences for Targeted Advertising via Bandit Experiments

Firms implementing digital advertising campaigns face a complex problem in determining the right match between their advertising creatives and target audiences. Typical solutions to the problem have leveraged non-experimental methods, or used "split-testing" strategies that have not explicitly addressed the complexities induced by targeted audiences that can potentially overlap with one another. This paper presents an adaptive algorithm that addresses the problem via online experimentation. The algorithm is set up as a contextual bandit and addresses the overlap issue by partitioning the target audiences into disjoint, non-overlapping sub-populations. It learns an optimal creative display policy in the disjoint space, while assessing in parallel which creative has the best match in the space of possibly overlapping target audiences. Experiments show that the proposed method is more efficient compared to naive "split-testing" or non-adaptive "A/B/n" testing based methods. We also describe a testing product we built that uses the algorithm. The product is currently deployed on the advertising platform of JD.com, an eCommerce company and a publisher of digital ads in China

Advertising Media and Target Audience Optimization via High-dimensional Bandits

We present a data-driven algorithm that advertisers can use to automate their digital ad-campaigns at online publishers. The algorithm enables the advertiser to search across available target audiences and ad-media to find the best possible combination for its campaign via online experimentation. The problem of finding the best audience-ad combination is complicated by a number of distinctive challenges, including (a) a need for active exploration to resolve prior uncertainty and to speed the search for profitable combinations, (b) many combinations to choose from, giving rise to high-dimensional search formulations, and (c) very low success probabilities, typically just a fraction of one percent. Our algorithm (designated LRDL, an acronym for Logistic Regression with Debiased Lasso) addresses these challenges by combining four elements: a multiarmed bandit framework for active exploration; a Lasso penalty function to handle high dimensionality; an inbuilt debiasing kernel that handles the regularization bias induced by the Lasso; and a semi-parametric regression model for outcomes that promotes cross-learning across arms. The algorithm is implemented as a Thompson Sampler, and to the best of our knowledge, it is the first that can practically address all of the challenges above. Simulations with real and synthetic data show the method is effective and document its superior performance against several benchmarks from the recent high-dimensional bandit literature.Comment: 39 pages, 8 figure

Estimating Causal Installed-Base Effects: A Bias-Correction Approach

New empirical models of consumer demand that incorporate social preferences, observational learning, word-of-mouth or network effects have the feature that the adoption of others in the reference group - the “installed-base” - has a causal effect on current adoption behavior. Estimation of such causal installed-base effects is challenging due to the potential for spurious correlation between the adoption of agents, arising from endogenous assortive matching into social groups (or homophily) and from the existence of unobservables across agents that are correlated. In the absence of experimental variation, the preferred solution is to control for these using a rich specification of fixed-effects, which is feasible with panel data. We show that fixedeffects estimators of this sort are inconsistent in the presence of installed-base effects; in our simulations, random-effects specifications perform even worse. Our analysis reveals the tension faced by the applied empiricist in this area: a rich control for unobservables increases the credibility of the reported causal effects, but the incorporation of these controls introduces biases of a new kind in this class of models. We present two solutions: an instrumental variable approach, and a new bias-correction approach, both of which deliver consistent estimates of causal installed-base effects. The bias-correction approach is tractable in this context because we are able to exploit the structure of the problem to solve analytically for the asymptotic bias of the installed-base estimator, and to incorporate it into the estimation routine. Our approach has implications for the measurement of social effects using non-experimental data, and for measuring marketing-mix effects in the presence of state-dependence in demand, more generally. Our empirical application to the adoption of the Toyota Prius Hybrid in California reveals evidence for social influence in diffusion, and demonstrates the importance of incorporating proper controls for the biases we identify

Online Causal Inference for Advertising in Real-Time Bidding Auctions

Real-time bidding (RTB) systems, which leverage auctions to programmatically allocate user impressions to multiple competing advertisers, continue to enjoy widespread success in digital advertising. Assessing the effectiveness of such advertising remains a lingering challenge in research and practice. This paper presents a new experimental design to perform causal inference on advertising bought through such mechanisms. Our method leverages the economic structure of first- and second-price auctions, which are ubiquitous in RTB systems, embedded within a multi-armed bandit (MAB) setup for online adaptive experimentation. We implement it via a modified Thompson sampling (TS) algorithm that estimates causal effects of advertising while minimizing the costs of experimentation to the advertiser by simultaneously learning the optimal bidding policy that maximizes her expected payoffs from auction participation. Simulations show that not only the proposed method successfully accomplishes the advertiser's goals, but also does so at a much lower cost than more conventional experimentation policies aimed at performing causal inference

Parallel Experimentation in a Competitive Advertising Marketplace

When multiple firms are simultaneously running experiments on a platform, the treatment effects for one firm may depend on the experimentation policies of others. This paper presents a set of causal estimands that are relevant to such an environment. We also present an experimental design that is suitable for facilitating experimentation across multiple competitors in such an environment. Together, these can be used by a platform to run experiments "as a service," on behalf of its participating firms. We show that the causal estimands we develop are identified nonparametrically by the variation induced by the design, and present two scalable estimators that help measure them in typical high-dimensional situations. We implement the design on the advertising platform of JD.com, an eCommerce company, which is also a publisher of digital ads in China. We discuss how the design is engineered within the platform's auction-driven ad-allocation system, which is typical of modern, digital advertising marketplaces. Finally, we present results from a parallel experiment involving 16 advertisers and millions of JD.com users. These results showcase the importance of accommodating a role for interactions across experimenters and demonstrates the viability of the framework

Social Ties and User Generated Content: Evidence from an Online Social Network

We use variation in wind speeds at surfing locations in Switzerland as exogenous shifters of users' propensity to post content about their surfing activity onto an online social network. We exploit this variation to test whether users' social ties on the network have a causal effect on their content generation, and whether conent generation in turn has a similar causal effect on the users' abilty to form social ties. Economically significant causal effects of this kind can produce positive feedback that generate multiplier e¤ects to interventions that subsidize tie formation. We argue these interventions can therefore be the basis of a strategy by the rm to indirectly faciliate content generation on the site. The exogenous variation provided by wind speeds enable us to measure this feedback empirically and to assess the return on investment from such policies. We use a detailed dataset from an online social network that comprises the complete details of social tie formation and content generation on the site. The richness of he data enable us to control for several spurious confounds that have typically plagued empirical analysis of social interactions. Our results show evidence for significant positive feedback in user generated content. We discuss the implications of the estimates for the management of the content and the growth of the network

Comparison Lift: Bandit-based Experimentation System for Online Advertising

Comparison Lift is an experimentation-as-a-service (EaaS) application for testing online advertising audiences and creatives at JD.com. Unlike many other EaaS tools that focus primarily on fixed sample A/B testing, Comparison Lift deploys a custom bandit-based experimentation algorithm. The advantages of the bandit-based approach are two-fold. First, it aligns the randomization induced in the test with the advertiser's goals from testing. Second, by adapting experimental design to information acquired during the test, it reduces substantially the cost of experimentation to the advertiser. Since launch in May 2019, Comparison Lift has been utilized in over 1,500 experiments. We estimate that utilization of the product has helped increase click-through rates of participating advertising campaigns by 46% on average. We estimate that the adaptive design in the product has generated 27% more clicks on average during testing compared to a fixed sample A/B design. Both suggest significant value generation and cost savings to advertisers from the product

Social Ties and User Generated Content: Evidence from an Online Social Network

We use variation in wind speeds at surfing locations in Switzerland as exogenous shifters of users' propensity to post content about their surfing activity onto an online social network. We exploit this variation to test whether users' social ties on the network have a causal effect on their content generation, and whether conent generation in turn has a similar causal effect on the users' abilty to form social ties. Economically significant causal effects of this kind can produce positive feedback that generate multiplier e¤ects to interventions that subsidize tie formation. We argue these interventions can therefore be the basis of a strategy by the rm to indirectly faciliate content generation on the site. The exogenous variation provided by wind speeds enable us to measure this feedback empirically and to assess the return on investment from such policies. We use a detailed dataset from an online social network that comprises the complete details of social tie formation and content generation on the site. The richness of he data enable us to control for several spurious confounds that have typically plagued empirical analysis of social interactions. Our results show evidence for significant positive feedback in user generated content. We discuss the implications of the estimates for the management of the content and the growth of the network

Intertemporal Price Discrimination with Forward-Looking Consumers: Application to the US Market for Console Video-Games

Firms in durable good product markets face incentives to intertemporally price discriminate, by setting high initial prices to sell to consumers with the highest willingness to pay, and cutting prices thereafter to appeal to those with lower willingness to pay. A critical determinant of the profitability of such pricing policies is the extent to which consumers anticipate future price declines, and delay purchases. We develop a framework to investigate empirically the optimal pricing over time of a firm selling a durable-good product to such strategic consumers. Prices in our model are equilibrium outcomes of a game played between forward-looking consumers who strategically delay purchases to avail of lower prices in the future, and a forward-looking firm that takes this consumer behavior into account in formulating its optimal pricing policy. The model incorporates first, a method to infer estimates of demand under dynamic consumer behavior, and second, an algorithm to compute the optimal sequence of prices given these demand estimates. The model is solved using numerical dynamic programming techniques. We present an empirical application to the market for video-games in the US. The results indicate that consumer forward-looking behavior has a significant effect on optimal pricing and profits of games in the industry. Simulations reveal that the profit losses of ignoring forward-looking behavior by consumers are large and economically significant, and suggest that market research that provides information regarding the extent of discounting by consumers is valuable to video-game firms.