2,582 research outputs found
Interpreting CNN Knowledge via an Explanatory Graph
This paper learns a graphical model, namely an explanatory graph, which
reveals the knowledge hierarchy hidden inside a pre-trained CNN. Considering
that each filter in a conv-layer of a pre-trained CNN usually represents a
mixture of object parts, we propose a simple yet efficient method to
automatically disentangles different part patterns from each filter, and
construct an explanatory graph. In the explanatory graph, each node represents
a part pattern, and each edge encodes co-activation relationships and spatial
relationships between patterns. More importantly, we learn the explanatory
graph for a pre-trained CNN in an unsupervised manner, i.e., without a need of
annotating object parts. Experiments show that each graph node consistently
represents the same object part through different images. We transfer part
patterns in the explanatory graph to the task of part localization, and our
method significantly outperforms other approaches.Comment: in AAAI 201
Interactively Transferring CNN Patterns for Part Localization
In the scenario of one/multi-shot learning, conventional end-to-end learning
strategies without sufficient supervision are usually not powerful enough to
learn correct patterns from noisy signals. Thus, given a CNN pre-trained for
object classification, this paper proposes a method that first summarizes the
knowledge hidden inside the CNN into a dictionary of latent activation
patterns, and then builds a new model for part localization by manually
assembling latent patterns related to the target part via human interactions.
We use very few (e.g., three) annotations of a semantic object part to retrieve
certain latent patterns from conv-layers to represent the target part. We then
visualize these latent patterns and ask users to further remove incorrect
patterns, in order to refine part representation. With the guidance of human
interactions, our method exhibited superior performance of part localization in
experiments
Examining CNN Representations with respect to Dataset Bias
Given a pre-trained CNN without any testing samples, this paper proposes a
simple yet effective method to diagnose feature representations of the CNN. We
aim to discover representation flaws caused by potential dataset bias. More
specifically, when the CNN is trained to estimate image attributes, we mine
latent relationships between representations of different attributes inside the
CNN. Then, we compare the mined attribute relationships with ground-truth
attribute relationships to discover the CNN's blind spots and failure modes due
to dataset bias. In fact, representation flaws caused by dataset bias cannot be
examined by conventional evaluation strategies based on testing images, because
testing images may also have a similar bias. Experiments have demonstrated the
effectiveness of our method.Comment: in AAAI 201
Interpretable CNNs for Object Classification
This paper proposes a generic method to learn interpretable convolutional
filters in a deep convolutional neural network (CNN) for object classification,
where each interpretable filter encodes features of a specific object part. Our
method does not require additional annotations of object parts or textures for
supervision. Instead, we use the same training data as traditional CNNs. Our
method automatically assigns each interpretable filter in a high conv-layer
with an object part of a certain category during the learning process. Such
explicit knowledge representations in conv-layers of CNN help people clarify
the logic encoded in the CNN, i.e., answering what patterns the CNN extracts
from an input image and uses for prediction. We have tested our method using
different benchmark CNNs with various structures to demonstrate the broad
applicability of our method. Experiments have shown that our interpretable
filters are much more semantically meaningful than traditional filters
Explanatory Graphs for CNNs
This paper introduces a graphical model, namely an explanatory graph, which
reveals the knowledge hierarchy hidden inside conv-layers of a pre-trained CNN.
Each filter in a conv-layer of a CNN for object classification usually
represents a mixture of object parts. We develop a simple yet effective method
to disentangle object-part pattern components from each filter. We construct an
explanatory graph to organize the mined part patterns, where a node represents
a part pattern, and each edge encodes co-activation relationships and spatial
relationships between patterns. More crucially, given a pre-trained CNN, the
explanatory graph is learned without a need of annotating object parts.
Experiments show that each graph node consistently represented the same object
part through different images, which boosted the transferability of CNN
features. We transferred part patterns in the explanatory graph to the task of
part localization, and our method significantly outperformed other approaches.Comment: arXiv admin note: substantial text overlap with arXiv:1708.0178
Interpretable Convolutional Filters with SincNet
Deep learning is currently playing a crucial role toward higher levels of
artificial intelligence. This paradigm allows neural networks to learn complex
and abstract representations, that are progressively obtained by combining
simpler ones. Nevertheless, the internal "black-box" representations
automatically discovered by current neural architectures often suffer from a
lack of interpretability, making of primary interest the study of explainable
machine learning techniques. This paper summarizes our recent efforts to
develop a more interpretable neural model for directly processing speech from
the raw waveform. In particular, we propose SincNet, a novel Convolutional
Neural Network (CNN) that encourages the first layer to discover more
meaningful filters by exploiting parametrized sinc functions. In contrast to
standard CNNs, which learn all the elements of each filter, only low and high
cutoff frequencies of band-pass filters are directly learned from data. This
inductive bias offers a very compact way to derive a customized filter-bank
front-end, that only depends on some parameters with a clear physical meaning.
Our experiments, conducted on both speaker and speech recognition, show that
the proposed architecture converges faster, performs better, and is more
interpretable than standard CNNs.Comment: In Proceedings of NIPS@IRASL 2018. arXiv admin note: substantial text
overlap with arXiv:1808.0015
Visual Analytics and Human Involvement in Machine Learning
The rapidly developing AI systems and applications still require human
involvement in practically all parts of the analytics process. Human decisions
are largely based on visualizations, providing data scientists details of data
properties and the results of analytical procedures. Different visualizations
are used in the different steps of the Machine Learning (ML) process. The
decision which visualization to use depends on factors, such as the data
domain, the data model and the step in the ML process. In this chapter, we
describe the seven steps in the ML process and review different visualization
techniques that are relevant for the different steps for different types of
data, models and purposes
On monitoring development indicators using high resolution satellite images
We develop a machine learning based tool for accurate prediction of
socio-economic indicators from daytime satellite imagery. The diverse set of
indicators are often not intuitively related to observable features in
satellite images, and are not even always well correlated with each other. Our
predictive tool is more accurate than using night light as a proxy, and can be
used to predict missing data, smooth out noise in surveys, monitor development
progress of a region, and flag potential anomalies. Finally, we use predicted
variables to do robustness analysis of a regression study of high rate of
stunting in India
Unsupervised Learning of Neural Networks to Explain Neural Networks
This paper presents an unsupervised method to learn a neural network, namely
an explainer, to interpret a pre-trained convolutional neural network (CNN),
i.e., explaining knowledge representations hidden in middle conv-layers of the
CNN. Given feature maps of a certain conv-layer of the CNN, the explainer
performs like an auto-encoder, which first disentangles the feature maps into
object-part features and then inverts object-part features back to features of
higher conv-layers of the CNN. More specifically, the explainer contains
interpretable conv-layers, where each filter disentangles the representation of
a specific object part from chaotic input feature maps. As a paraphrase of CNN
features, the disentangled representations of object parts help people
understand the logic inside the CNN. We also learn the explainer to use
object-part features to reconstruct features of higher CNN layers, in order to
minimize loss of information during the feature disentanglement. More
crucially, we learn the explainer via network distillation without using any
annotations of sample labels, object parts, or textures for supervision. We
have applied our method to different types of CNNs for evaluation, and
explainers have significantly boosted the interpretability of CNN features
Entropic Variable Projection for Explainability and Intepretability
In this paper, we present a new explainability formalism designed to explain
how the input variables of the testing set impact the predictions of black-box
decision rules. Hence we propose a group explainability frame for machine
learning algorithms based on the variability of the distribution of the input
variables. Our formalism is based on an information theory framework that
quantifies the influence of all input-output observations when emphasizing the
impact of each input variable, based on entropic projections. This formalism is
thus the first unified and model agnostic framework enabling us to interpret
the dependence between the input variables, their impact on the prediction
errors, and their influence on the output predictions. In addition and most
importantly, we prove that computing an explanation in our framework has a low
algorithmic complexity making it scalable to real-life large datasets. We
illustrate our strategy by explaining complex decision rules learned using
XGBoost, Random Forest or Deep Neural Network classifiers. We finally make
clear its differences with explainability strategies based on single
observations, such as those of LIME or SHAP. A toolbox is proposed at
https://xai-aniti.github.io/ethik/
- …