Online Model Evaluation in a Large-Scale Computational Advertising Platform

Online media provides opportunities for marketers through which they can deliver effective brand messages to a wide range of audiences. Advertising technology platforms enable advertisers to reach their target audience by delivering ad impressions to online users in real time. In order to identify the best marketing message for a user and to purchase impressions at the right price, we rely heavily on bid prediction and optimization models. Even though the bid prediction models are well studied in the literature, the equally important subject of model evaluation is usually overlooked. Effective and reliable evaluation of an online bidding model is crucial for making faster model improvements as well as for utilizing the marketing budgets more efficiently. In this paper, we present an experimentation framework for bid prediction models where our focus is on the practical aspects of model evaluation. Specifically, we outline the unique challenges we encounter in our platform due to a variety of factors such as heterogeneous goal definitions, varying budget requirements across different campaigns, high seasonality and the auction-based environment for inventory purchasing. Then, we introduce return on investment (ROI) as a unified model performance (i.e., success) metric and explain its merits over more traditional metrics such as click-through rate (CTR) or conversion rate (CVR). Most importantly, we discuss commonly used evaluation and metric summarization approaches in detail and propose a more accurate method for online evaluation of new experimental models against the baseline. Our meta-analysis-based approach addresses various shortcomings of other methods and yields statistically robust conclusions that allow us to conclude experiments more quickly in a reliable manner. We demonstrate the effectiveness of our evaluation strategy on real campaign data through some experiments.Comment: Accepted to ICDM201

Development Strategy, Viability, and Economic Institutions: The Case of China

development strategy, institution, viability, trinity system

The New Food Safety

A safe food supply is essential for a healthy society. Our food system is replete with different types of risk, yet food safety is often narrowly understood as encompassing only foodborne illness and other risks related directly to food ingestion. This Article argues for a more comprehensive definition of food safety, one that includes not just acute, ingestion-related risks, but also whole-diet cumulative ingestion risks, and cradle-to-grave risks of food production and disposal. This broader definition, which we call “Food System Safety,” draws under the header of food safety a variety of historically siloed, and under-regulated, food system issues including nutrition, environmental protection, and workplace safety. The current narrow approach to food safety is inadequate. First, it contributes to irrational resource allocation among food system risks. Second, it has collateral consequences for other food system risks, and, third, its limited focus can undermine efforts to achieve narrow food safety. A comprehensive understanding of food safety illuminates the complex interactions between narrow food safety and other areas of food system health risks. We argue that such an understanding could facilitate improved allocation of resources and assessment of tradeoffs, and ultimately support better health and safety outcomes for more people. We offer a variety of structural and institutional mechanisms for embedding this approach into federal agency action

ARE HOUSEHOLD PRODUCTION DECISIONS COOPERATIVE? EVIDENCE ON MIGRATION AND MILK SALES FROM NORTHERN KENYA

Replaced with revised version of paper 08/29/02.Consumer/Household Economics,

Inadequacies in the water reforms in the Kyrgyz Republic: an institutional analysis

Water resource management / Analysis / Irrigation management / Participatory management / Water users’ associations / Research methods / Agrarian reform / Irrigation programs / Operations / Maintenance / Conflict / Rivers / Kyrgyzstan

Media mix modeling: a case study on optimizing television and digital media spend for a retailer

Project Work presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Marketing IntelligenceRetailers invest most of their advertising budget in traditional channels, namely Television, even though the percentage of budget allocated towards digital media has been increasing. Since the largest part of sales still happen in physical stores, marketers face the challenge of optimizing their media mix to maximize revenue. To address this challenge, media mix models were developed using the traditional modeling approach, based on linear regressions, with data from a retailer’s advertising campaign, specifically the online and offline investments per channel and online conversion metrics. The models were influenced by the selection bias regarding funnel effects, which was exacerbated by the use of the last-touch attribution model that tends to disproportionately skew marketer investment away from higher funnel channels to lower-funnel. Nonetheless, results from the models suggest that online channels were more effective in explaining the variance of the number of participations, which were a proxy to sales. To managers, this thesis highlights that there are factors specific to their own campaigns that influence the media mix models, which they must consider and, if possible, control for. One factor is the selection biases, such as ad targeting that may arise from using the paid search channel or remarketing tactics, seasonality or the purchase funnel effects bias that undermines the contribution of higher-funnel channels like TV, which generates awareness in the target audience. Therefore, companies should assess which of these biases might have a bigger influence on their results and design their models accordingly. Data limitations are the most common constraint for marketing mix modeling. In this case, we did not have access to sales and media spend historical data. Therefore, it was not possible to understand what the uplift in sales caused by the promotion was, as well as to verify the impact of the promotion on items that were eligible to participate in the promotion, versus the items that were not. Also, we were not able to reduce the bias from the paid search channel because we lacked the search query data necessary to control for it and improve the accuracy of the models. Moreover, this project is not the ultimate solution for the “company’s” marketing measurement challenges but rather informs its next initiatives. It describes the state of the art in marketing mix modeling, reveals the limitations of the models developed and suggests ways to improve future models. In turn, this is expected to provide more accurate marketing measurement, and as a result, a media budget allocation that improves business performance