6,984 research outputs found
Understanding Regularization to Visualize Convolutional Neural Networks
Variational methods for revealing visual concepts learned by convolutional
neural networks have gained significant attention during the last years. Being
based on noisy gradients obtained via back-propagation such methods require the
application of regularization strategies. We present a mathematical framework
unifying previously employed regularization methods. Within this framework, we
propose a novel technique based on Sobolev gradients which can be implemented
via convolutions and does not require specialized numerical treatment, such as
total variation regularization. The experiments performed on feature inversion
and activation maximization demonstrate the benefit of a unified approach to
regularization, such as sharper reconstructions via the proposed Sobolev
filters and a better control over reconstructed scales
How convolutional neural network see the world - A survey of convolutional neural network visualization methods
Nowadays, the Convolutional Neural Networks (CNNs) have achieved impressive
performance on many computer vision related tasks, such as object detection,
image recognition, image retrieval, etc. These achievements benefit from the
CNNs outstanding capability to learn the input features with deep layers of
neuron structures and iterative training process. However, these learned
features are hard to identify and interpret from a human vision perspective,
causing a lack of understanding of the CNNs internal working mechanism. To
improve the CNN interpretability, the CNN visualization is well utilized as a
qualitative analysis method, which translates the internal features into
visually perceptible patterns. And many CNN visualization works have been
proposed in the literature to interpret the CNN in perspectives of network
structure, operation, and semantic concept. In this paper, we expect to provide
a comprehensive survey of several representative CNN visualization methods,
including Activation Maximization, Network Inversion, Deconvolutional Neural
Networks (DeconvNet), and Network Dissection based visualization. These methods
are presented in terms of motivations, algorithms, and experiment results.
Based on these visualization methods, we also discuss their practical
applications to demonstrate the significance of the CNN interpretability in
areas of network design, optimization, security enhancement, etc.Comment: 32 pages, 21 figures. Mathematical Foundations of Computin
Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
Image representations, from SIFT and bag of visual words to Convolutional
Neural Networks (CNNs) are a crucial component of almost all computer vision
systems. However, our understanding of them remains limited. In this paper we
study several landmark representations, both shallow and deep, by a number of
complementary visualization techniques. These visualizations are based on the
concept of "natural pre-image", namely a natural-looking image whose
representation has some notable property. We study in particular three such
visualizations: inversion, in which the aim is to reconstruct an image from its
representation, activation maximization, in which we search for patterns that
maximally stimulate a representation component, and caricaturization, in which
the visual patterns that a representation detects in an image are exaggerated.
We pose these as a regularized energy-minimization framework and demonstrate
its generality and effectiveness. In particular, we show that this method can
invert representations such as HOG more accurately than recent alternatives
while being applicable to CNNs too. Among our findings, we show that several
layers in CNNs retain photographically accurate information about the image,
with different degrees of geometric and photometric invariance.Comment: A substantially extended version of
http://www.robots.ox.ac.uk/~vedaldi/assets/pubs/mahendran15understanding.pdf.
arXiv admin note: text overlap with arXiv:1412.003
Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents
As deep reinforcement learning driven by visual perception becomes more
widely used there is a growing need to better understand and probe the learned
agents. Understanding the decision making process and its relationship to
visual inputs can be very valuable to identify problems in learned behavior.
However, this topic has been relatively under-explored in the research
community. In this work we present a method for synthesizing visual inputs of
interest for a trained agent. Such inputs or states could be situations in
which specific actions are necessary. Further, critical states in which a very
high or a very low reward can be achieved are often interesting to understand
the situational awareness of the system as they can correspond to risky states.
To this end, we learn a generative model over the state space of the
environment and use its latent space to optimize a target function for the
state of interest. In our experiments we show that this method can generate
insights for a variety of environments and reinforcement learning methods. We
explore results in the standard Atari benchmark games as well as in an
autonomous driving simulator. Based on the efficiency with which we have been
able to identify behavioural weaknesses with this technique, we believe this
general approach could serve as an important tool for AI safety applications
Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
We can better understand deep neural networks by identifying which features
each of their neurons have learned to detect. To do so, researchers have
created Deep Visualization techniques including activation maximization, which
synthetically generates inputs (e.g. images) that maximally activate each
neuron. A limitation of current techniques is that they assume each neuron
detects only one type of feature, but we know that neurons can be multifaceted,
in that they fire in response to many different types of features: for example,
a grocery store class neuron must activate either for rows of produce or for a
storefront. Previous activation maximization techniques constructed images
without regard for the multiple different facets of a neuron, creating
inappropriate mixes of colors, parts of objects, scales, orientations, etc.
Here, we introduce an algorithm that explicitly uncovers the multiple facets of
each neuron by producing a synthetic visualization of each of the types of
images that activate a neuron. We also introduce regularization methods that
produce state-of-the-art results in terms of the interpretability of images
obtained by activation maximization. By separately synthesizing each type of
image a neuron fires in response to, the visualizations have more appropriate
colors and coherent global structure. Multifaceted feature visualization thus
provides a clearer and more comprehensive description of the role of each
neuron.Comment: 23 pages (including SI), 24 figure
Parsimonious Deep Learning: A Differential Inclusion Approach with Global Convergence
Over-parameterization is ubiquitous nowadays in training neural networks to
benefit both optimization in seeking global optima and generalization in
reducing prediction error. However, compressive networks are desired in many
real world applications and direct training of small networks may be trapped in
local optima. In this paper, instead of pruning or distilling an
over-parameterized model to compressive ones, we propose a parsimonious
learning approach based on differential inclusions of inverse scale spaces,
that generates a family of models from simple to complex ones with a better
efficiency and interpretability than stochastic gradient descent in exploring
the model space. It enjoys a simple discretization, the Split Linearized
Bregman Iterations, with provable global convergence that from any
initializations, algorithmic iterations converge to a critical point of
empirical risks. One may exploit the proposed method to boost the complexity of
neural networks progressively. Numerical experiments with MNIST, Cifar-10/100,
and ImageNet are conducted to show the method is promising in training large
scale models with a favorite interpretability.Comment: 25 pages, 7 figure
Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
Deep neural networks (DNNs) have demonstrated state-of-the-art results on
many pattern recognition tasks, especially vision classification problems.
Understanding the inner workings of such computational brains is both
fascinating basic science that is interesting in its own right - similar to why
we study the human brain - and will enable researchers to further improve DNNs.
One path to understanding how a neural network functions internally is to study
what each of its neurons has learned to detect. One such method is called
activation maximization (AM), which synthesizes an input (e.g. an image) that
highly activates a neuron. Here we dramatically improve the qualitative state
of the art of activation maximization by harnessing a powerful, learned prior:
a deep generator network (DGN). The algorithm (1) generates qualitatively
state-of-the-art synthetic images that look almost real, (2) reveals the
features learned by each neuron in an interpretable way, (3) generalizes well
to new datasets and somewhat well to different network architectures without
requiring the prior to be relearned, and (4) can be considered as a
high-quality generative method (in this case, by generating novel, creative,
interesting, recognizable images).Comment: 29 pages, 35 figures, NIPS camera-read
On the Effect of Low-Rank Weights on Adversarial Robustness of Neural Networks
Recently, there has been an abundance of works on designing Deep Neural
Networks (DNNs) that are robust to adversarial examples. In particular, a
central question is which features of DNNs influence adversarial robustness
and, therefore, can be to used to design robust DNNs. In this work, this
problem is studied through the lens of compression which is captured by the
low-rank structure of weight matrices. It is first shown that adversarial
training tends to promote simultaneously low-rank and sparse structure in the
weight matrices of neural networks. This is measured through the notions of
effective rank and effective sparsity. In the reverse direction, when the low
rank structure is promoted by nuclear norm regularization and combined with
sparsity inducing regularizations, neural networks show significantly improved
adversarial robustness. The effect of nuclear norm regularization on
adversarial robustness is paramount when it is applied to convolutional neural
networks. Although still not competing with adversarial training, this result
contributes to understanding the key properties of robust classifiers
Understanding Graph Isomorphism Network for rs-fMRI Functional Connectivity Analysis
Graph neural networks (GNN) rely on graph operations that include neural
network training for various graph related tasks. Recently, several attempts
have been made to apply the GNNs to functional magnetic resonance image (fMRI)
data. Despite recent progresses, a common limitation is its difficulty to
explain the classification results in a neuroscientifically explainable way.
Here, we develop a framework for analyzing the fMRI data using the Graph
Isomorphism Network (GIN), which was recently proposed as a powerful GNN for
graph classification. One of the important contributions of this paper is the
observation that the GIN is a dual representation of convolutional neural
network (CNN) in the graph space where the shift operation is defined using the
adjacency matrix. This understanding enables us to exploit CNN-based saliency
map techniques for the GNN, which we tailor to the proposed GIN with one-hot
encoding, to visualize the important regions of the brain. We validate our
proposed framework using large-scale resting-state fMRI (rs-fMRI) data for
classifying the sex of the subject based on the graph structure of the brain.
The experiment was consistent with our expectation such that the obtained
saliency map show high correspondence with previous neuroimaging evidences
related to sex differences.Comment: This paper is accepted for Frontiers in Neuroscienc
Understanding Neural Networks via Feature Visualization: A survey
A neuroscience method to understanding the brain is to find and study the
preferred stimuli that highly activate an individual cell or groups of cells.
Recent advances in machine learning enable a family of methods to synthesize
preferred stimuli that cause a neuron in an artificial or biological brain to
fire strongly. Those methods are known as Activation Maximization (AM) or
Feature Visualization via Optimization. In this chapter, we (1) review existing
AM techniques in the literature; (2) discuss a probabilistic interpretation for
AM; and (3) review the applications of AM in debugging and explaining networks.Comment: A book chapter in an Interpretable ML book
(http://www.interpretable-ml.org/book/
- …