Inefficiencies in Digital Advertising Markets

Digital advertising markets are growing and attracting increased scrutiny. This article explores four market inefficiencies that remain poorly understood: ad effect measurement, frictions between and within advertising channel members, ad blocking, and ad fraud. Although these topics are not unique to digital advertising, each manifests in unique ways in markets for digital ads. The authors identify relevant findings in the academic literature, recent developments in practice, and promising topics for future research

A novel machine learning approach for identifying the drivers of domestic electricity users’ price responsiveness

Time-based pricing programs for domestic electricity users have been effective in reducing peak demand and facilitating renewables integration. Nevertheless, high cost, price non-responsiveness and adverse selection may create the possible challenges. To overcome these challenges, it can be fruitful to investigate the ‘high-potential’ users, which are more responsive to price changes and apply time-based pricing to these users. Few studies have investigated how to identify which users are more price-responsive. We aim to fill this gap by comprehensively identifying the drivers of domestic users’ price responsiveness, in order to facilitate the selection of the high-potential users. We adopt a novel data-driven approach, first by a feed forward neural network model to accurately determine the baseline monthly peak consumption of individual households, followed by an integrated machine-learning variable selection methodology to identify the drivers of price responsiveness applied to Irish smart meter data from 2009-10 as part of a national Time of Use trial. This methodology substantially outperforms traditional variable selection methods by combining three advanced machine-learning techniques. Our results show that the response of energy users to price change is affected by a number of factors, ranging from demographic and dwelling characteristics, psychological factors, historical electricity consumption, to appliance ownership. In particular, historical electricity consumption, income, the number of occupants, perceived behavioural control, and adoption of specific appliances, including immersion water heater and dishwasher, are found to be significant drivers of price responsiveness. We also observe that continual price increase within a moderate range does not drive additional peak demand reduction, and that there is an intention-behaviour gap, whereby stated intention does not lead to actual peak reduction behavior. Based on our findings, we have conducted scenario analysis to demonstrate the feasibility of selecting the high potential users to achieve significant peak reduction

Stability Analysis of Wholesale Electricity Markets under Dynamic Consumption Models and Real-Time Pricing

This paper analyzes stability conditions for wholesale electricity markets under real-time retail pricing and realistic consumption models with memory, which explicitly take into account previous electricity prices and consumption levels. By passing on the current retail price of electricity from supplier to consumer and feeding the observed consumption back to the supplier, a closed-loop dynamical system for electricity prices and consumption arises whose stability is to be investigated. Under mild assumptions on the generation cost of electricity and consumers' backlog disutility functions, we show that, for consumer models with price memory only, market stability is achieved if the ratio between the consumers' marginal backlog disutility and the suppliers' marginal cost of supply remains below a fixed threshold. Further, consumer models with price and consumption memory can result in greater stability regions and faster convergence to the equilibrium compared to models with price memory alone, if consumption deviations from nominal demand are adequately penalized.Comment: 8 pages, 7 Figures, accepted to the 2017 American Control Conferenc

A Primer on Causality in Data Science

Many questions in Data Science are fundamentally causal in that our objective is to learn the effect of some exposure, randomized or not, on an outcome interest. Even studies that are seemingly non-causal, such as those with the goal of prediction or prevalence estimation, have causal elements, including differential censoring or measurement. As a result, we, as Data Scientists, need to consider the underlying causal mechanisms that gave rise to the data, rather than simply the pattern or association observed in those data. In this work, we review the 'Causal Roadmap' of Petersen and van der Laan (2014) to provide an introduction to some key concepts in causal inference. Similar to other causal frameworks, the steps of the Roadmap include clearly stating the scientific question, defining of the causal model, translating the scientific question into a causal parameter, assessing the assumptions needed to express the causal parameter as a statistical estimand, implementation of statistical estimators including parametric and semi-parametric methods, and interpretation of our findings. We believe that using such a framework in Data Science will help to ensure that our statistical analyses are guided by the scientific question driving our research, while avoiding over-interpreting our results. We focus on the effect of an exposure occurring at a single time point and highlight the use of targeted maximum likelihood estimation (TMLE) with Super Learner.Comment: 26 pages (with references); 4 figure

Targeted demand response for flexible energy communities using clustering techniques

The present study proposes clustering techniques for designing demand response (DR) programs for commercial and residential prosumers. The goal is to alter the consumption behavior of the prosumers within a distributed energy community in Italy. This aggregation aims to: a) minimize the reverse power flow at the primary substation, occuring when generation from solar panels in the local grid exceeds consumption, and b) shift the system wide peak demand, that typically occurs during late afternoon. Regarding the clustering stage, we consider daily prosumer load profiles and divide them across the extracted clusters. Three popular machine learning algorithms are employed, namely k-means, k-medoids and agglomerative clustering. We evaluate the methods using multiple metrics including a novel metric proposed within this study, namely peak performance score (PPS). The k-means algorithm with dynamic time warping distance considering 14 clusters exhibits the highest performance with a PPS of 0.689. Subsequently, we analyze each extracted cluster with respect to load shape, entropy, and load types. These characteristics are used to distinguish the clusters that have the potential to serve the optimization objectives by matching them to proper DR schemes including time of use, critical peak pricing, and real-time pricing. Our results confirm the effectiveness of the proposed clustering algorithm in generating meaningful flexibility clusters, while the derived DR pricing policy encourages consumption during off-peak hours. The developed methodology is robust to the low availability and quality of training datasets and can be used by aggregator companies for segmenting energy communities and developing personalized DR policies