90 research outputs found
Estimating Causal Installed-Base Effects: A Bias-Correction Approach
New empirical models of consumer demand that incorporate social
preferences, observational learning, word-of-mouth or network effects
have the feature that the adoption of others in the reference group -
the “installed-base” - has a causal effect on current
adoption behavior. Estimation of such causal installed-base effects is
challenging due to the potential for spurious correlation between the
adoption of agents, arising from endogenous assortive matching into
social groups (or homophily) and from the existence of unobservables
across agents that are correlated. In the absence of experimental
variation, the preferred solution is to control for these using a rich
specification of fixed-effects, which is feasible with panel data. We
show that fixedeffects estimators of this sort are inconsistent in the
presence of installed-base effects; in our simulations, random-effects
specifications perform even worse. Our analysis reveals the tension
faced by the applied empiricist in this area: a rich control for
unobservables increases the credibility of the reported causal effects,
but the incorporation of these controls introduces biases of a new kind
in this class of models. We present two solutions: an instrumental
variable approach, and a new bias-correction approach, both of which
deliver consistent estimates of causal installed-base effects. The
bias-correction approach is tractable in this context because we are
able to exploit the structure of the problem to solve analytically for
the asymptotic bias of the installed-base estimator, and to incorporate
it into the estimation routine. Our approach has implications for the
measurement of social effects using non-experimental data, and for
measuring marketing-mix effects in the presence of state-dependence in
demand, more generally. Our empirical application to the adoption of the
Toyota Prius Hybrid in California reveals evidence for social influence
in diffusion, and demonstrates the importance of incorporating proper
controls for the biases we identify
Advertising Content and Consumer Engagement on Social Media: Evidence from Facebook
We describe the effect of social media advertising content on customer engagement using data from Facebook. We content-code 106,316 Facebook messages across 782 companies, using a combination of Amazon Mechanical Turk and natural language processing algorithms. We use this data set to study the association of various kinds of social media marketing content with user engagement—defined as Likes, comments, shares, and click-throughs—with the messages. We find that inclusion of widely used content related to brand personality—like humor and emotion—is associated with higher levels of consumer engagement (Likes, comments, shares) with a message. We find that directly informative content—like mentions of price and deals—is associated with lower levels of engagement when included in messages in isolation, but higher engagement levels when provided in combination with brand personality–related attributes. Also, certain directly informative content, such as deals and promotions, drive consumers’ path to conversion (click-throughs). These results persist after incorporating corrections for the nonrandom targeting of Facebook’s EdgeRank (News Feed) algorithm and so reflect more closely user reaction to content than Facebook’s behavioral targeting. Our results suggest that there are benefits to content engineering that combines informative characteristics that help in obtaining immediate leads (via improved click-throughs) with brand personality–related content that helps in maintaining future reach and branding on the social media site (via improved engagement). These results inform content design strategies. Separately, the methodology we apply to content-code text is useful for future studies utilizing unstructured data such as advertising content or product reviews
Online Evaluation of Audiences for Targeted Advertising via Bandit Experiments
Firms implementing digital advertising campaigns face a complex problem in
determining the right match between their advertising creatives and target
audiences. Typical solutions to the problem have leveraged non-experimental
methods, or used "split-testing" strategies that have not explicitly addressed
the complexities induced by targeted audiences that can potentially overlap
with one another. This paper presents an adaptive algorithm that addresses the
problem via online experimentation. The algorithm is set up as a contextual
bandit and addresses the overlap issue by partitioning the target audiences
into disjoint, non-overlapping sub-populations. It learns an optimal creative
display policy in the disjoint space, while assessing in parallel which
creative has the best match in the space of possibly overlapping target
audiences. Experiments show that the proposed method is more efficient compared
to naive "split-testing" or non-adaptive "A/B/n" testing based methods. We also
describe a testing product we built that uses the algorithm. The product is
currently deployed on the advertising platform of JD.com, an eCommerce company
and a publisher of digital ads in China
Advertising Media and Target Audience Optimization via High-dimensional Bandits
We present a data-driven algorithm that advertisers can use to automate their
digital ad-campaigns at online publishers. The algorithm enables the advertiser
to search across available target audiences and ad-media to find the best
possible combination for its campaign via online experimentation. The problem
of finding the best audience-ad combination is complicated by a number of
distinctive challenges, including (a) a need for active exploration to resolve
prior uncertainty and to speed the search for profitable combinations, (b) many
combinations to choose from, giving rise to high-dimensional search
formulations, and (c) very low success probabilities, typically just a fraction
of one percent. Our algorithm (designated LRDL, an acronym for Logistic
Regression with Debiased Lasso) addresses these challenges by combining four
elements: a multiarmed bandit framework for active exploration; a Lasso penalty
function to handle high dimensionality; an inbuilt debiasing kernel that
handles the regularization bias induced by the Lasso; and a semi-parametric
regression model for outcomes that promotes cross-learning across arms. The
algorithm is implemented as a Thompson Sampler, and to the best of our
knowledge, it is the first that can practically address all of the challenges
above. Simulations with real and synthetic data show the method is effective
and document its superior performance against several benchmarks from the
recent high-dimensional bandit literature.Comment: 39 pages, 8 figure
Online Causal Inference for Advertising in Real-Time Bidding Auctions
Real-time bidding (RTB) systems, which leverage auctions to programmatically
allocate user impressions to multiple competing advertisers, continue to enjoy
widespread success in digital advertising. Assessing the effectiveness of such
advertising remains a lingering challenge in research and practice. This paper
presents a new experimental design to perform causal inference on advertising
bought through such mechanisms. Our method leverages the economic structure of
first- and second-price auctions, which are ubiquitous in RTB systems, embedded
within a multi-armed bandit (MAB) setup for online adaptive experimentation. We
implement it via a modified Thompson sampling (TS) algorithm that estimates
causal effects of advertising while minimizing the costs of experimentation to
the advertiser by simultaneously learning the optimal bidding policy that
maximizes her expected payoffs from auction participation. Simulations show
that not only the proposed method successfully accomplishes the advertiser's
goals, but also does so at a much lower cost than more conventional
experimentation policies aimed at performing causal inference
Estimating Causal Installed-Base Effects: A Bias-Correction Approach
New empirical models of consumer demand that incorporate social
preferences, observational learning, word-of-mouth or network effects
have the feature that the adoption of others in the reference group -
the “installed-base” - has a causal effect on current
adoption behavior. Estimation of such causal installed-base effects is
challenging due to the potential for spurious correlation between the
adoption of agents, arising from endogenous assortive matching into
social groups (or homophily) and from the existence of unobservables
across agents that are correlated. In the absence of experimental
variation, the preferred solution is to control for these using a rich
specification of fixed-effects, which is feasible with panel data. We
show that fixedeffects estimators of this sort are inconsistent in the
presence of installed-base effects; in our simulations, random-effects
specifications perform even worse. Our analysis reveals the tension
faced by the applied empiricist in this area: a rich control for
unobservables increases the credibility of the reported causal effects,
but the incorporation of these controls introduces biases of a new kind
in this class of models. We present two solutions: an instrumental
variable approach, and a new bias-correction approach, both of which
deliver consistent estimates of causal installed-base effects. The
bias-correction approach is tractable in this context because we are
able to exploit the structure of the problem to solve analytically for
the asymptotic bias of the installed-base estimator, and to incorporate
it into the estimation routine. Our approach has implications for the
measurement of social effects using non-experimental data, and for
measuring marketing-mix effects in the presence of state-dependence in
demand, more generally. Our empirical application to the adoption of the
Toyota Prius Hybrid in California reveals evidence for social influence
in diffusion, and demonstrates the importance of incorporating proper
controls for the biases we identify
Parallel Experimentation in a Competitive Advertising Marketplace
When multiple firms are simultaneously running experiments on a platform, the
treatment effects for one firm may depend on the experimentation policies of
others. This paper presents a set of causal estimands that are relevant to such
an environment. We also present an experimental design that is suitable for
facilitating experimentation across multiple competitors in such an
environment. Together, these can be used by a platform to run experiments "as a
service," on behalf of its participating firms. We show that the causal
estimands we develop are identified nonparametrically by the variation induced
by the design, and present two scalable estimators that help measure them in
typical high-dimensional situations. We implement the design on the advertising
platform of JD.com, an eCommerce company, which is also a publisher of digital
ads in China. We discuss how the design is engineered within the platform's
auction-driven ad-allocation system, which is typical of modern, digital
advertising marketplaces. Finally, we present results from a parallel
experiment involving 16 advertisers and millions of JD.com users. These results
showcase the importance of accommodating a role for interactions across
experimenters and demonstrates the viability of the framework
Social Ties and User Generated Content: Evidence from an Online Social Network
We use variation in wind speeds at surfing locations in Switzerland as
exogenous shifters of users' propensity to post content about their
surfing activity onto an online social network. We exploit this
variation to test whether users' social ties on the network have a
causal effect on their content generation, and whether conent generation
in turn has a similar causal effect on the users' abilty to form social
ties. Economically significant causal effects of this kind can produce
positive feedback that generate multiplier e¤ects to
interventions that subsidize tie formation. We argue these interventions
can therefore be the basis of a strategy by the rm to indirectly
faciliate content generation on the site. The exogenous variation
provided by wind speeds enable us to measure this feedback empirically
and to assess the return on investment from such policies. We use a
detailed dataset from an online social network that comprises the
complete details of social tie formation and content generation on the
site. The richness of he data enable us to control for several spurious
confounds that have typically plagued empirical analysis of social
interactions. Our results show evidence for significant positive
feedback in user generated content. We discuss the implications of the
estimates for the management of the content and the growth of the network
Comparison Lift: Bandit-based Experimentation System for Online Advertising
Comparison Lift is an experimentation-as-a-service (EaaS) application for
testing online advertising audiences and creatives at JD.com. Unlike many other
EaaS tools that focus primarily on fixed sample A/B testing, Comparison Lift
deploys a custom bandit-based experimentation algorithm. The advantages of the
bandit-based approach are two-fold. First, it aligns the randomization induced
in the test with the advertiser's goals from testing. Second, by adapting
experimental design to information acquired during the test, it reduces
substantially the cost of experimentation to the advertiser. Since launch in
May 2019, Comparison Lift has been utilized in over 1,500 experiments. We
estimate that utilization of the product has helped increase click-through
rates of participating advertising campaigns by 46% on average. We estimate
that the adaptive design in the product has generated 27% more clicks on
average during testing compared to a fixed sample A/B design. Both suggest
significant value generation and cost savings to advertisers from the product
- …