The Value of Field Experiments

The feasibility of using field experiments to optimize marketing decisions remains relatively unstudied. We investigate category pricing decisions that require estimating a large matrix of cross-product demand elasticities and ask the following question: How many experiments are required as the number of products in the category grows? Our main result demonstrates that if the categories have a favorable structure, we can learn faster and reduce the number of experiments that are required: the number of experiments required may grow just logarithmically with the number of products. These findings potentially have important implications for the application of field experiments. Firms may be able to obtain meaningful estimates using a practically feasible number of experiments, even in categories with a large number of products. We also provide a relatively simple mechanism that firms can use to evaluate whether a category has a structure that makes it feasible to use field experiments to set prices. We illustrate how to accomplish this using either a sample of historical data or a pilot set of experiments. We also discuss how to evaluate whether field experiments can help optimize other marketing decisions, such as selecting which products to advertise or promote.National Science Foundation (U.S.) (Grant CMMI-0856063)National Science Foundation (U.S.) (Grant CMMI-1158658

How Sales Taxes Affect Customer and Firm Behavior: The Role of Search on the Internet

When a multichannel retailer opens its first retail store in a state, the firm is obligated to collect sales taxes on all Internet and catalog orders shipped to that state. This article assesses how opening a store affects Internet and catalog demand. The authors analyze purchase behavior among customers who live far from the retail store but must now pay sales taxes on catalog and Internet purchases. A comparable group of customers in a neighboring state serves as a control. The results show that Internet sales decrease significantly, but catalog sales are unaffected. Further investigation indicates that the difference in these outcomes is partly attributable to the ease with which customers can search for lower prices at competing retailers. The authors extend the analysis to a panel of multichannel firms and show that retailers that earn a large proportion of their revenue from direct channels avoid opening a first store in high-tax states. They conclude that current U.S. sales taxes laws have significant effects on both customer and firm behavior

Analytical essays on marketing

Thesis (Ph. D.)--Massachusetts Institute of Technology, Sloan School of Management, 1993.Includes bibliographical references (leaves 102-103).Duncan I. Simester.Ph.D

Determinants of Asset Ownership: A Study of the Carpentry Trade

We use a data set describing ownership of productive assets in the carpentry trade to evaluate several factors influencing the allocation of asset ownership between an employer and his employees. The findings suggest that the allocation involves a tradeoff between two incentive effects influencing how the employee uses the asset and what the employer decides it should be used for. In particular, the allocation of ownership hinges on whether an asset is easily lost or stolen, which favors employee ownership, and whether the employer's task assignment affects the asset's depreciation, which favors employer ownership. There is also evidence that more expensive assets and assets that are shared by more than one employee are more likely to be owned by the employer. The results suggest that a general theory of asset ownership should be able to take account of at least these effects. © 2005 President and Fellows of Harvard College and the Massachusetts Institute of Technology.

—Does Demand Fall When Customers Perceive That Prices Are Unfair? The Case of Premium Pricing for Large Sizes

We analyze a large-scale field test conducted with a mail-order catalog firm to investigate how customers react to premium prices for larger sizes of women's apparel. We find that customers who demand large sizes react unfavorably to paying a higher price than customers for small sizes. Further investigation suggests that these consumers perceive that the price premium is unfair. Overall, premium pricing led to a 6% to 8% decrease in gross profits.product line pricing, price discrimination, fairness, price promotion, experimental economics

Price Discrimination as an Adverse Signal: Why an Offer to Spread Payments May Hurt Demand

Firms often search enthusiastically for distinguishing traits that they may use to price discriminate between segments. Yet there are occasions in which firms forgo the opportunity to price discriminate and instead charge a single price. Traditional explanations for why retailers forgo the opportunity to price discriminate focus on the cost of discriminating, including operational costs, explicit discrimination costs, and implicit discrimination costs. In this paper we identify an additional reason for why firms may forgo an opportunity to price discriminate. By revealing that a product is being sold to a broad range of segments, a retailer implicitly claims that the product is suitable for each segment. However, claiming that a premium-quality product is suitable for price-sensitive consumers undermines the credibility of a retailer's quality claim. The signaling explanation was motivated by extensive discussions over more than a year with a major catalog retailer that sells premium-quality jewelry and gifts. Discussions with managers revealed that they were reluctant to use any price-discrimination mechanism that signals their products are targeted at price-sensitive customers. For example, the catalog does not include sale or clearance sections and does not target more price-sensitive customers by using separate items. However, management was under some pressure to consider installment-billing offers, which allow customers to pay over a series of periods rather than in a lump sum. Management feared that offering installment billing may adversely affect customers' quality perceptions and demand. To investigate this issue, we develop a general game-the-oretic model, illustrate how the model extends to installment billing, and conduct a large-scale field test. The general model illustrates how selling to multiple segments may lead to an adverse quality signal. We illustrate how the model extends to installment-billing offers in a direct-mail catalog. Installment-billing offers allow customers to spread the total payment over a series of payments. All customers have the option of using installment billing, and customers who use the plan receive an economic benefit (an interest-free loan). We would normally expect this type of offer to increase demand or, at a minimum, leave demand unchanged. However, because installment-billing offers target credit-constrained customers, we predicted that the introduction of installment billing would prompt an unfavorable quality inference and reduce demand among quality-sensitive customers. We empirically investigated this prediction in a large-scale field test with a catalog that offers premium-quality jewelry and gifts. Two versions of the catalog were created: a test version that contained an installment-billing offer, and a control version in which installment billing was not offered. Importantly, the prices in both the test version and control version were identical. Approximately 240,000 catalogs were mailed, and customers were randomly assigned to either the test version or control version. Results show that the installment-billing offer (test version) was associated with both a reduction in the number of orders received and a reduction in aggregate revenue. Offering installment billing resulted in approximately $15,000 in lost revenue. The only plausible explanation for this counterintuitive finding appears to be the signaling theory. To investigate the long-term effects, the catalog agreed to survey their customers to measure how an offer of installment billing affects their customers' quality perceptions. Similar to the field test, two versions of a catalog were created, and customers were randomly mailed a catalog, along with a short survey. Respondents were asked to browse through the catalog and return their responses in a replypaid envelope. The findings are consistent with customer beliefs in the signaling model: Offering installment billing lowers the perceived quality of the items in the catalog. The field test and survey findings were both statistically significant and managerially relevant. Together, the results convinced the catalog not to include installment-billing offers in future catalogs.Signaling, Price Discrimination, Installment Billing, Promotions, Quality Perceptions, Retailing

Efficiently Evaluating Targeting Policies: Improving on Champion vs. Challenger Experiments

© 2020 INFORMS Champion versus challenger field experiments are widely used to compare the performance of different targeting policies. These experiments randomly assign customers to receive marketing actions recommended by either the existing (champion) policy or the new (challenger) policy, and then compare the aggregate outcomes. We recommend an alternative experimental design and propose an alternative estimation approach to improve the evaluation of targeting policies. The recommended experimental design randomly assigns customers to marketing actions. This allows evaluation of any targeting policy without requiring an additional experiment, including policies designed after the experiment is implemented. The proposed estimation approach identifies customers for whom different policies recommend the same action and recognizes that for these customers there is no difference in performance. This allows for a more precise comparison of the policies. We illustrate the advantages of the experimental design and estimation approach using data from an actual field experiment. We also demonstrate that the grouping of customers, which is the foundation of our estimation approach, can help to improve the training of new targeting policies

Targeting Prospective Customers: Robustness of Machine-Learning Methods to Typical Data Challenges

We investigate how firms can use the results of field experiments to optimize the targeting of promotions when prospecting for new customers. We evaluate seven widely used machine-learning methods using a series of two large-scale field experiments. The first field experiment generates a common pool of training data for each of the seven methods. We then validate the seven optimized policies provided by each method together with uniform benchmark policies in a second field experiment. The findings not only compare the performance of the targeting methods, but also demonstrate how well the methods address common data challenges. Our results reveal that when the training data are ideal, model-driven methods perform better than distance-driven methods and classification methods. However, the performance advantage vanishes in the presence of challenges that affect the quality of the training data, including the extent to which the training data captures details of the implementation setting. The challenges we study are covariate shift, concept shift, information loss through aggregation, and imbalanced data. Intuitively, the model-driven methods make better use of the information available in the training data, but the performance of these methods is more sensitive to deterioration in the quality of this information. The classification methods we tested performed relatively poorly. We explain the poor performance of the classification methods in our setting and describe how the performance of these methods could be improved

Price Stickiness and Customer Antagonism

Managers often state that they are reluctant to vary prices for fear of "antagonizing customers." However, there is no empirical evidence that antagonizing customers through price adjustments reduces demand or profits. We use a 28-month randomized field experiment involving over 50,000 customers to investigate how customers react if they buy a product and later observe the same retailer selling it for less. We find that customers react by making fewer subsequent purchases from the firm. The effect is largest among the firm's most valuable customers: those whose prior purchases were most recent and at the highest prices. (c) 2010 by the President and Fellows of Harvard College and the Massachusetts Institute of Technology..