395 research outputs found
Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR
There has been much discussion of the right to explanation in the EU General
Data Protection Regulation, and its existence, merits, and disadvantages.
Implementing a right to explanation that opens the black box of algorithmic
decision-making faces major legal and technical barriers. Explaining the
functionality of complex algorithmic decision-making systems and their
rationale in specific cases is a technically challenging problem. Some
explanations may offer little meaningful information to data subjects, raising
questions around their value. Explanations of automated decisions need not
hinge on the general public understanding how algorithmic systems function.
Even though such interpretability is of great importance and should be pursued,
explanations can, in principle, be offered without opening the black box.
Looking at explanations as a means to help a data subject act rather than
merely understand, one could gauge the scope and content of explanations
according to the specific goal or action they are intended to support. From the
perspective of individuals affected by automated decision-making, we propose
three aims for explanations: (1) to inform and help the individual understand
why a particular decision was reached, (2) to provide grounds to contest the
decision if the outcome is undesired, and (3) to understand what would need to
change in order to receive a desired result in the future, based on the current
decision-making model. We assess how each of these goals finds support in the
GDPR. We suggest data controllers should offer a particular type of
explanation, unconditional counterfactual explanations, to support these three
aims. These counterfactual explanations describe the smallest change to the
world that can be made to obtain a desirable outcome, or to arrive at the
closest possible world, without needing to explain the internal logic of the
system
Ordered Counterfactual Explanation by Mixed-Integer Linear Optimization
Post-hoc explanation methods for machine learning models have been widely
used to support decision-making. One of the popular methods is Counterfactual
Explanation (CE), also known as Actionable Recourse, which provides a user with
a perturbation vector of features that alters the prediction result. Given a
perturbation vector, a user can interpret it as an "action" for obtaining one's
desired decision result. In practice, however, showing only a perturbation
vector is often insufficient for users to execute the action. The reason is
that if there is an asymmetric interaction among features, such as causality,
the total cost of the action is expected to depend on the order of changing
features. Therefore, practical CE methods are required to provide an
appropriate order of changing features in addition to a perturbation vector.
For this purpose, we propose a new framework called Ordered Counterfactual
Explanation (OrdCE). We introduce a new objective function that evaluates a
pair of an action and an order based on feature interaction. To extract an
optimal pair, we propose a mixed-integer linear optimization approach with our
objective function. Numerical experiments on real datasets demonstrated the
effectiveness of our OrdCE in comparison with unordered CE methods.Comment: 20 pages, 5 figures, to appear in the 35th AAAI Conference on
Artificial Intelligence (AAAI 2021
Explaining the Black-box Smoothly- A Counterfactual Approach
We propose a BlackBox \emph{Counterfactual Explainer} that is explicitly
developed for medical imaging applications. Classical approaches (e.g. saliency
maps) assessing feature importance do not explain \emph{how} and \emph{why}
variations in a particular anatomical region is relevant to the outcome, which
is crucial for transparent decision making in healthcare application. Our
framework explains the outcome by gradually \emph{exaggerating} the semantic
effect of the given outcome label. Given a query input to a classifier,
Generative Adversarial Networks produce a progressive set of perturbations to
the query image that gradually changes the posterior probability from its
original class to its negation. We design the loss function to ensure that
essential and potentially relevant details, such as support devices, are
preserved in the counterfactually generated images. We provide an extensive
evaluation of different classification tasks on the chest X-Ray images. Our
experiments show that a counterfactually generated visual explanation is
consistent with the disease's clinical relevant measurements, both
quantitatively and qualitatively.Comment: Under review for IEEE-TMI journa
- …