14,282 research outputs found
Towards better understanding of gradient-based attribution methods for Deep Neural Networks
Understanding the flow of information in Deep Neural Networks (DNNs) is a
challenging problem that has gain increasing attention over the last few years.
While several methods have been proposed to explain network predictions, there
have been only a few attempts to compare them from a theoretical perspective.
What is more, no exhaustive empirical comparison has been performed in the
past. In this work, we analyze four gradient-based attribution methods and
formally prove conditions of equivalence and approximation between them. By
reformulating two of these methods, we construct a unified framework which
enables a direct comparison, as well as an easier implementation. Finally, we
propose a novel evaluation metric, called Sensitivity-n and test the
gradient-based attribution methods alongside with a simple perturbation-based
attribution method on several datasets in the domains of image and text
classification, using various network architectures.Comment: ICLR 201
Interpreting Adversarially Trained Convolutional Neural Networks
We attempt to interpret how adversarially trained convolutional neural
networks (AT-CNNs) recognize objects. We design systematic approaches to
interpret AT-CNNs in both qualitative and quantitative ways and compare them
with normally trained models. Surprisingly, we find that adversarial training
alleviates the texture bias of standard CNNs when trained on object recognition
tasks, and helps CNNs learn a more shape-biased representation. We validate our
hypothesis from two aspects. First, we compare the salience maps of AT-CNNs and
standard CNNs on clean images and images under different transformations. The
comparison could visually show that the prediction of the two types of CNNs is
sensitive to dramatically different types of features. Second, to achieve
quantitative verification, we construct additional test datasets that destroy
either textures or shapes, such as style-transferred version of clean data,
saturated images and patch-shuffled ones, and then evaluate the classification
accuracy of AT-CNNs and normal CNNs on these datasets. Our findings shed some
light on why AT-CNNs are more robust than those normally trained ones and
contribute to a better understanding of adversarial training over CNNs from an
interpretation perspective.Comment: To apper in ICML1
Looking Deeper into Deep Learning Model: Attribution-based Explanations of TextCNN
Layer-wise Relevance Propagation (LRP) and saliency maps have been recently
used to explain the predictions of Deep Learning models, specifically in the
domain of text classification. Given different attribution-based explanations
to highlight relevant words for a predicted class label, experiments based on
word deleting perturbation is a common evaluation method. This word removal
approach, however, disregards any linguistic dependencies that may exist
between words or phrases in a sentence, which could semantically guide a
classifier to a particular prediction. In this paper, we present a
feature-based evaluation framework for comparing the two attribution methods on
customer reviews (public data sets) and Customer Due Diligence (CDD) extracted
reports (corporate data set). Instead of removing words based on the relevance
score, we investigate perturbations based on embedded features removal from
intermediate layers of Convolutional Neural Networks. Our experimental study is
carried out on embedded-word, embedded-document, and embedded-ngrams
explanations. Using the proposed framework, we provide a visualization tool to
assist analysts in reasoning toward the model's final prediction.Comment: NIPS 2018 Workshop on Challenges and Opportunities for AI in
Financial Services: the Impact of Fairness, Explainability, Accuracy, and
Privacy, Montr\'eal, Canad
- …