16 research outputs found
Interpreting CNN Knowledge via an Explanatory Graph
This paper learns a graphical model, namely an explanatory graph, which
reveals the knowledge hierarchy hidden inside a pre-trained CNN. Considering
that each filter in a conv-layer of a pre-trained CNN usually represents a
mixture of object parts, we propose a simple yet efficient method to
automatically disentangles different part patterns from each filter, and
construct an explanatory graph. In the explanatory graph, each node represents
a part pattern, and each edge encodes co-activation relationships and spatial
relationships between patterns. More importantly, we learn the explanatory
graph for a pre-trained CNN in an unsupervised manner, i.e., without a need of
annotating object parts. Experiments show that each graph node consistently
represents the same object part through different images. We transfer part
patterns in the explanatory graph to the task of part localization, and our
method significantly outperforms other approaches.Comment: in AAAI 201
Examining CNN Representations with respect to Dataset Bias
Given a pre-trained CNN without any testing samples, this paper proposes a
simple yet effective method to diagnose feature representations of the CNN. We
aim to discover representation flaws caused by potential dataset bias. More
specifically, when the CNN is trained to estimate image attributes, we mine
latent relationships between representations of different attributes inside the
CNN. Then, we compare the mined attribute relationships with ground-truth
attribute relationships to discover the CNN's blind spots and failure modes due
to dataset bias. In fact, representation flaws caused by dataset bias cannot be
examined by conventional evaluation strategies based on testing images, because
testing images may also have a similar bias. Experiments have demonstrated the
effectiveness of our method.Comment: in AAAI 201
Variational Saccading: Efficient Inference for Large Resolution Images
Image classification with deep neural networks is typically restricted to
images of small dimensionality such as 224 x 244 in Resnet models [24]. This
limitation excludes the 4000 x 3000 dimensional images that are taken by modern
smartphone cameras and smart devices. In this work, we aim to mitigate the
prohibitive inferential and memory costs of operating in such large dimensional
spaces. To sample from the high-resolution original input distribution, we
propose using a smaller proxy distribution to learn the co-ordinates that
correspond to regions of interest in the high-dimensional space. We introduce a
new principled variational lower bound that captures the relationship of the
proxy distribution's posterior and the original image's co-ordinate space in a
way that maximizes the conditional classification likelihood. We empirically
demonstrate on one synthetic benchmark and one real world large resolution DSLR
camera image dataset that our method produces comparable results with ~10x
faster inference and lower memory consumption than a model that utilizes the
entire original input distribution. Finally, we experiment with a more complex
setting using mini-maps from Starcraft II [56] to infer the number of
characters in a complex 3d-rendered scene. Even in such complicated scenes our
model provides strong localization: a feature missing from traditional
classification models.Comment: Published BMVC 2019 & NIPS 2018 Bayesian Deep Learning Worksho
Explainability in Deep Reinforcement Learning
A large set of the explainable Artificial Intelligence (XAI) literature is
emerging on feature relevance techniques to explain a deep neural network (DNN)
output or explaining models that ingest image source data. However, assessing
how XAI techniques can help understand models beyond classification tasks, e.g.
for reinforcement learning (RL), has not been extensively studied. We review
recent works in the direction to attain Explainable Reinforcement Learning
(XRL), a relatively new subfield of Explainable Artificial Intelligence,
intended to be used in general public applications, with diverse audiences,
requiring ethical, responsible and trustable algorithms. In critical situations
where it is essential to justify and explain the agent's behaviour, better
explainability and interpretability of RL models could help gain scientific
insight on the inner workings of what is still considered a black box. We
evaluate mainly studies directly linking explainability to RL, and split these
into two categories according to the way the explanations are generated:
transparent algorithms and post-hoc explainaility. We also review the most
prominent XAI works from the lenses of how they could potentially enlighten the
further deployment of the latest advances in RL, in the demanding present and
future of everyday problems.Comment: Article accepted at Knowledge-Based System