16,908 research outputs found
Counterfactual Explanations for Data-Driven Decisions
Users’ lack of understanding of systems that use predictive models to make automated decisions is one of the main barriers for their adoption. We adopt the increasingly accepted view of a counterfactual explanation for a system decision: a set of the system inputs that is causal (meaning that removing them changes the decision) and irreducible (meaning that removing any subset of the inputs in the explanation does not change the decision). We generalize previous work on counterfactual explanations in three ways: we explain system decisions rather than model predictions; we do not enforce any specific method for removing inputs, and our explanations can incorporate inputs with arbitrary data structures. We also show how model-agnostic algorithms can be tweaked to find the most useful explanations depending on the context. Finally, we showcase our approach using a real data set to illustrate its advantages over other explanation methods when the goal is to understand system decisions better
Explaining Data-Driven Decisions made by AI Systems: The Counterfactual Approach
We examine counterfactual explanations for explaining the decisions made by
model-based AI systems. The counterfactual approach we consider defines an
explanation as a set of the system's data inputs that causally drives the
decision (i.e., changing the inputs in the set changes the decision) and is
irreducible (i.e., changing any subset of the inputs does not change the
decision). We (1) demonstrate how this framework may be used to provide
explanations for decisions made by general, data-driven AI systems that may
incorporate features with arbitrary data types and multiple predictive models,
and (2) propose a heuristic procedure to find the most useful explanations
depending on the context. We then contrast counterfactual explanations with
methods that explain model predictions by weighting features according to their
importance (e.g., SHAP, LIME) and present two fundamental reasons why we should
carefully consider whether importance-weight explanations are well-suited to
explain system decisions. Specifically, we show that (i) features that have a
large importance weight for a model prediction may not affect the corresponding
decision, and (ii) importance weights are insufficient to communicate whether
and how features influence decisions. We demonstrate this with several concise
examples and three detailed case studies that compare the counterfactual
approach with SHAP to illustrate various conditions under which counterfactual
explanations explain data-driven decisions better than importance weights
Decisions, Counterfactual Explanations and Strategic Behavior
As data-driven predictive models are increasingly used to inform decisions,
it has been argued that decision makers should provide explanations that help
individuals understand what would have to change for these decisions to be
beneficial ones. However, there has been little discussion on the possibility
that individuals may use the above counterfactual explanations to invest effort
strategically and maximize their chances of receiving a beneficial decision. In
this paper, our goal is to find policies and counterfactual explanations that
are optimal in terms of utility in such a strategic setting. We first show
that, given a pre-defined policy, the problem of finding the optimal set of
counterfactual explanations is NP-hard. Then, we show that the corresponding
objective is nondecreasing and satisfies submodularity and this allows a
standard greedy algorithm to enjoy approximation guarantees. In addition, we
further show that the problem of jointly finding both the optimal policy and
set of counterfactual explanations reduces to maximizing a non-monotone
submodular function. As a result, we can use a recent randomized algorithm to
solve the problem, which also offers approximation guarantees. Finally, we
demonstrate that, by incorporating a matroid constraint into the problem
formulation, we can increase the diversity of the optimal set of counterfactual
explanations and incentivize individuals across the whole spectrum of the
population to self improve. Experiments on synthetic and real lending and
credit card data illustrate our theoretical findings and show that the
counterfactual explanations and decision policies found by our algorithms
achieve higher utility than several competitive baselines.Comment: New data preprocessing method, experiments on credit card data and
experiments under a matroid constrain
LIMEtree: Interactively Customisable Explanations Based on Local Surrogate Multi-output Regression Trees
Systems based on artificial intelligence and machine learning models should
be transparent, in the sense of being capable of explaining their decisions to
gain humans' approval and trust. While there are a number of explainability
techniques that can be used to this end, many of them are only capable of
outputting a single one-size-fits-all explanation that simply cannot address
all of the explainees' diverse needs. In this work we introduce a
model-agnostic and post-hoc local explainability technique for black-box
predictions called LIMEtree, which employs surrogate multi-output regression
trees. We validate our algorithm on a deep neural network trained for object
detection in images and compare it against Local Interpretable Model-agnostic
Explanations (LIME). Our method comes with local fidelity guarantees and can
produce a range of diverse explanation types, including contrastive and
counterfactual explanations praised in the literature. Some of these
explanations can be interactively personalised to create bespoke, meaningful
and actionable insights into the model's behaviour. While other methods may
give an illusion of customisability by wrapping, otherwise static, explanations
in an interactive interface, our explanations are truly interactive, in the
sense of allowing the user to "interrogate" a black-box model. LIMEtree can
therefore produce consistent explanations on which an interactive exploratory
process can be built
The Repurchase Behavior of Individual Investors: An Experimental Investigation
We analyze two recently documented follow-on purchase and repurchase patterns experimentally: Individual investors’ preference for purchasing additional shares of a stock that decreased rather than increased in value succeeding an initial purchase (pattern 1) and investors’ tendency for purchasing stocks that they previously sold at a higher price (pattern 2). Similar to the field data study by Odean, Strahilevitz, and Barber (2004), subjects in our experiment are about 2.5 to 3 times as likely to purchase units of a single fictitious good if the price of the good declined following a purchase or sale in the previous period. As an assignment of choices clearly reduces the effect, we ar-gue that investors are involved in counterfactual thinking: They refrain from purchasing additional shares or repurchasing shares at a higher price because doing so means admitting to their ex post wrong decision.
Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR
There has been much discussion of the right to explanation in the EU General
Data Protection Regulation, and its existence, merits, and disadvantages.
Implementing a right to explanation that opens the black box of algorithmic
decision-making faces major legal and technical barriers. Explaining the
functionality of complex algorithmic decision-making systems and their
rationale in specific cases is a technically challenging problem. Some
explanations may offer little meaningful information to data subjects, raising
questions around their value. Explanations of automated decisions need not
hinge on the general public understanding how algorithmic systems function.
Even though such interpretability is of great importance and should be pursued,
explanations can, in principle, be offered without opening the black box.
Looking at explanations as a means to help a data subject act rather than
merely understand, one could gauge the scope and content of explanations
according to the specific goal or action they are intended to support. From the
perspective of individuals affected by automated decision-making, we propose
three aims for explanations: (1) to inform and help the individual understand
why a particular decision was reached, (2) to provide grounds to contest the
decision if the outcome is undesired, and (3) to understand what would need to
change in order to receive a desired result in the future, based on the current
decision-making model. We assess how each of these goals finds support in the
GDPR. We suggest data controllers should offer a particular type of
explanation, unconditional counterfactual explanations, to support these three
aims. These counterfactual explanations describe the smallest change to the
world that can be made to obtain a desirable outcome, or to arrive at the
closest possible world, without needing to explain the internal logic of the
system
Axiomatic Characterization of Data-Driven Influence Measures for Classification
We study the following problem: given a labeled dataset and a specific
datapoint x, how did the i-th feature influence the classification for x? We
identify a family of numerical influence measures - functions that, given a
datapoint x, assign a numeric value phi_i(x) to every feature i, corresponding
to how altering i's value would influence the outcome for x. This family, which
we term monotone influence measures (MIM), is uniquely derived from a set of
desirable properties, or axioms. The MIM family constitutes a provably sound
methodology for measuring feature influence in classification domains; the
values generated by MIM are based on the dataset alone, and do not make any
queries to the classifier. While this requirement naturally limits the scope of
our framework, we demonstrate its effectiveness on data
- …