2,294 research outputs found
The Grammar of Interactive Explanatory Model Analysis
The growing need for in-depth analysis of predictive models leads to a series
of new methods for explaining their local and global properties. Which of these
methods is the best? It turns out that this is an ill-posed question. One
cannot sufficiently explain a black-box machine learning model using a single
method that gives only one perspective. Isolated explanations are prone to
misunderstanding, which inevitably leads to wrong or simplistic reasoning. This
problem is known as the Rashomon effect and refers to diverse, even
contradictory interpretations of the same phenomenon. Surprisingly, the
majority of methods developed for explainable machine learning focus on a
single aspect of the model behavior. In contrast, we showcase the problem of
explainability as an interactive and sequential analysis of a model. This paper
presents how different Explanatory Model Analysis (EMA) methods complement each
other and why it is essential to juxtapose them together. The introduced
process of Interactive EMA (IEMA) derives from the algorithmic side of
explainable machine learning and aims to embrace ideas developed in cognitive
sciences. We formalize the grammar of IEMA to describe potential human-model
dialogues. IEMA is implemented in the human-centered framework that adopts
interactivity, customizability and automation as its main traits. Combined,
these methods enhance the responsible approach to predictive modeling.Comment: 17 pages, 10 figures, 3 table
On Network Science and Mutual Information for Explaining Deep Neural Networks
In this paper, we present a new approach to interpret deep learning models.
By coupling mutual information with network science, we explore how information
flows through feedforward networks. We show that efficiently approximating
mutual information allows us to create an information measure that quantifies
how much information flows between any two neurons of a deep learning model. To
that end, we propose NIF, Neural Information Flow, a technique for codifying
information flow that exposes deep learning model internals and provides
feature attributions.Comment: ICASSP 2020 (shorter version appeared at AAAI-19 Workshop on Network
Interpretability for Deep Learning
- …