3,976 research outputs found
Explaining Recurrent Neural Network Predictions in Sentiment Analysis
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown
to deliver insightful explanations in the form of input space relevances for
understanding feed-forward neural network classification decisions. In the
present work, we extend the usage of LRP to recurrent neural networks. We
propose a specific propagation rule applicable to multiplicative connections as
they arise in recurrent network architectures such as LSTMs and GRUs. We apply
our technique to a word-based bi-directional LSTM model on a five-class
sentiment prediction task, and evaluate the resulting LRP relevances both
qualitatively and quantitatively, obtaining better results than a
gradient-based related method which was used in previous work.Comment: 9 pages, 4 figures, accepted for EMNLP'17 Workshop on Computational
Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA
Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
As Deep Neural Networks (DNNs) have demonstrated superhuman performance in a
variety of fields, there is an increasing interest in understanding the complex
internal mechanisms of DNNs. In this paper, we propose Relative Attributing
Propagation (RAP), which decomposes the output predictions of DNNs with a new
perspective of separating the relevant (positive) and irrelevant (negative)
attributions according to the relative influence between the layers. The
relevance of each neuron is identified with respect to its degree of
contribution, separated into positive and negative, while preserving the
conservation rule. Considering the relevance assigned to neurons in terms of
relative priority, RAP allows each neuron to be assigned with a bi-polar
importance score concerning the output: from highly relevant to highly
irrelevant. Therefore, our method makes it possible to interpret DNNs with much
clearer and attentive visualizations of the separated attributions than the
conventional explaining methods. To verify that the attributions propagated by
RAP correctly account for each meaning, we utilize the evaluation metrics: (i)
Outside-inside relevance ratio, (ii) Segmentation mIOU and (iii) Region
perturbation. In all experiments and metrics, we present a sizable gap in
comparison to the existing literature. Our source code is available in
\url{https://github.com/wjNam/Relative_Attributing_Propagation}.Comment: 8 pages, 7 figures, Accepted paper in AAAI Conference on Artificial
Intelligence (AAAI), 202
- …