Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR

There has been much discussion of the right to explanation in the EU General Data Protection Regulation, and its existence, merits, and disadvantages. Implementing a right to explanation that opens the black box of algorithmic decision-making faces major legal and technical barriers. Explaining the functionality of complex algorithmic decision-making systems and their rationale in specific cases is a technically challenging problem. Some explanations may offer little meaningful information to data subjects, raising questions around their value. Explanations of automated decisions need not hinge on the general public understanding how algorithmic systems function. Even though such interpretability is of great importance and should be pursued, explanations can, in principle, be offered without opening the black box. Looking at explanations as a means to help a data subject act rather than merely understand, one could gauge the scope and content of explanations according to the specific goal or action they are intended to support. From the perspective of individuals affected by automated decision-making, we propose three aims for explanations: (1) to inform and help the individual understand why a particular decision was reached, (2) to provide grounds to contest the decision if the outcome is undesired, and (3) to understand what would need to change in order to receive a desired result in the future, based on the current decision-making model. We assess how each of these goals finds support in the GDPR. We suggest data controllers should offer a particular type of explanation, unconditional counterfactual explanations, to support these three aims. These counterfactual explanations describe the smallest change to the world that can be made to obtain a desirable outcome, or to arrive at the closest possible world, without needing to explain the internal logic of the system

Counterfactual analysis in macroeconometrics: an empirical investigation into the effects of quantitative easing

This paper is concerned with ex ante and ex post counterfactual analyses in the case of macroeconometric applications where a single unit is observed before and after a given policy intervention. It distinguishes between cases where the policy change affects the model’s parameters and where it does not. It is argued that for ex post policy evaluation it is important that outcomes are conditioned on ex post realized variables that are invariant to the policy change but nevertheless influence the outcomes. The effects of the control variables that are determined endogenously with the policy outcomes can be solved out for the policy evaluation exercise. An ex post policy ineffectiveness test statistic is proposed. The analysis is applied to the evaluation of the effects of the quantitative easing (QE) in the UK after March 2009. It is estimated that a 100 basis points reduction in the spread due to QE has an impact effect on output growth of about one percentage point, but the policy impact is very quickly reversed with no statistically significant effects remaining within 9-12 months of the policy intervention

Exclusive dealing as a barrier to entry? Evidence from automobiles.

Exclusive dealing contracts between manufacturers and retailers force new entrants to set up their own costly dealer networks to enter the market. We ask whether such contracts may act as an entry barrier, and provide an empirical analysis of the European car market. We first estimate a demand model with product and spatial differentiation, and quantify the role of a dense distribution network in explaining the car manufacturers’ market shares. We then perform policy counterfactuals to assess the pro.t incentives and entry-deterring effects of exclusive dealing. We find that there are no individual incentives to maintain exclusive dealing, but there can be a collective incentive by the industry as a whole, even absent efficiencies. Furthermore, a ban on exclusive dealing would shift market shares from the larger European firms to the smaller entrants. More importantly, consumers would gain substantially, mainly because of the increased spatial availability and less so because of intensified price competition. Our findings suggest that the European Commission’s recent decision to facilitate exclusive dealing in the car market may not have been warranted.

Double Bottom Line Progress Report: Assessing Social Impact in Double Bottom Line Ventures, Methods Catalog

Outlines methods for social entrepreneurs and their investors to define, measure and communicate social impact and return in early-stage ventures

Local Rule-Based Explanations of Black Box Decision Systems

The recent years have witnessed the rise of accurate but obscure decision systems which hide the logic of their internal decision processes to the users. The lack of explanations for the decisions of black box systems is a key ethical issue, and a limitation to the adoption of machine learning components in socially sensitive and safety-critical contexts. %Therefore, we need explanations that reveals the reasons why a predictor takes a certain decision. In this paper we focus on the problem of black box outcome explanation, i.e., explaining the reasons of the decision taken on a specific instance. We propose LORE, an agnostic method able to provide interpretable and faithful explanations. LORE first leans a local interpretable predictor on a synthetic neighborhood generated by a genetic algorithm. Then it derives from the logic of the local interpretable predictor a meaningful explanation consisting of: a decision rule, which explains the reasons of the decision; and a set of counterfactual rules, suggesting the changes in the instance's features that lead to a different outcome. Wide experiments show that LORE outperforms existing methods and baselines both in the quality of explanations and in the accuracy in mimicking the black box

The long term effect of vocational qualifications on labour market outcomes

"London Economics were commissioned by the Department for Business, Innovation and Skills to undertake an assessment of the long-term effect of vocational education and training on labour market outcomes. We combined learner attainment information from the Individual Learner Record (ILR) between 2002/03 and 2005/06, annual earnings information (between 2003/04 and 2009/10) and employment information (between 1999/00 and 2009/10) from HM Revenue and Customs, and benefit receipt and duration information (between 1999/00 and 2009/10) from the Department for Work and Pensions. The number of individuals contained in the ILR totalled almost 6.9 million learners that could be subsequently matched to the HMRC and DWP data sources (presented in Figure 1). For the analysis we retained both achievers and individuals who enrolled in the course but failed to achieve the qualification aim." - page 8

Double Bottom Line Project Report: Assessing Social Impact in Double Bottom Line Ventures

This tool expresses costs and social impacts of an investment in monetary terms. Quantification is achieved according to one or more of three measures: NPV (the aggregate value of all costs, revenues and social impacts discounted), benefit-cost ratio (the discounted value of revenues and positive impacts divided by discounted value of costs and negative impacts) and internal rate of return (the net value of revenues plus impacts expressed as an annual percentage return on the total costs of the investment)