2,020 research outputs found
Interpreting Neural Network Judgments via Minimal, Stable, and Symbolic Corrections
We present a new algorithm to generate minimal, stable, and symbolic
corrections to an input that will cause a neural network with ReLU activations
to change its output. We argue that such a correction is a useful way to
provide feedback to a user when the network's output is different from a
desired output. Our algorithm generates such a correction by solving a series
of linear constraint satisfaction problems. The technique is evaluated on three
neural network models: one predicting whether an applicant will pay a mortgage,
one predicting whether a first-order theorem can be proved efficiently by a
solver using certain heuristics, and the final one judging whether a drawing is
an accurate rendition of a canonical drawing of a cat.Comment: 24 page
Cardinality-Minimal Explanations for Monotonic Neural Networks
In recent years, there has been increasing interest in explanation methods
for neural model predictions that offer precise formal guarantees. These
include abductive (respectively, contrastive) methods, which aim to compute
minimal subsets of input features that are sufficient for a given prediction to
hold (respectively, to change a given prediction). The corresponding decision
problems are, however, known to be intractable. In this paper, we investigate
whether tractability can be regained by focusing on neural models implementing
a monotonic function. Although the relevant decision problems remain
intractable, we can show that they become solvable in polynomial time by means
of greedy algorithms if we additionally assume that the activation functions
are continuous everywhere and differentiable almost everywhere. Our experiments
suggest favourable performance of our algorithms
Synthesizing Action Sequences for Modifying Model Decisions
When a model makes a consequential decision, e.g., denying someone a loan, it
needs to additionally generate actionable, realistic feedback on what the
person can do to favorably change the decision. We cast this problem through
the lens of program synthesis, in which our goal is to synthesize an optimal
(realistically cheapest or simplest) sequence of actions that if a person
executes successfully can change their classification. We present a novel and
general approach that combines search-based program synthesis and test-time
adversarial attacks to construct action sequences over a domain-specific set of
actions. We demonstrate the effectiveness of our approach on a number of deep
neural networks
A Step Towards Explainable Person Re-identification Rankings
More and more video and image data is available to security authorities that can help solve crimes. Since manual analysis is time-consuming, algorithms are needed that support e.g. re-identification of persons. However, person re-identification approaches solely output image rank lists but do not provide an explanation for the results.
In this work, two concepts are proposed to explain person re-identification rankings and a qualitative evaluation is conducted. Both approaches are based on a multi-task convolutional neural network which outputs feature vectors for person re-identification and simultaneously recognizes a person’s semantic attributes. Analyses of the learned weights and the outputs of the attribute classifier are used to generate the explanations.
The results of the conducted experiments indicate that both approaches are suitable to improve the comprehensibility of person re-identification rankings
- …