1,563 research outputs found
Explaining a black-box using Deep Variational Information Bottleneck Approach
Interpretable machine learning has gained much attention recently. Briefness
and comprehensiveness are necessary in order to provide a large amount of
information concisely when explaining a black-box decision system. However,
existing interpretable machine learning methods fail to consider briefness and
comprehensiveness simultaneously, leading to redundant explanations. We propose
the variational information bottleneck for interpretation, VIBI, a
system-agnostic interpretable method that provides a brief but comprehensive
explanation. VIBI adopts an information theoretic principle, information
bottleneck principle, as a criterion for finding such explanations. For each
instance, VIBI selects key features that are maximally compressed about an
input (briefness), and informative about a decision made by a black-box system
on that input (comprehensive). We evaluate VIBI on three datasets and compare
with state-of-the-art interpretable machine learning methods in terms of both
interpretability and fidelity evaluated by human and quantitative metric
An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction
Decisions of complex language understanding models can be rationalized by
limiting their inputs to a relevant subsequence of the original text. A
rationale should be as concise as possible without significantly degrading task
performance, but this balance can be difficult to achieve in practice. In this
paper, we show that it is possible to better manage this trade-off by
optimizing a bound on the Information Bottleneck (IB) objective. Our fully
unsupervised approach jointly learns an explainer that predicts sparse binary
masks over sentences, and an end-task predictor that considers only the
extracted rationale. Using IB, we derive a learning objective that allows
direct control of mask sparsity levels through a tunable sparse prior.
Experiments on ERASER benchmark tasks demonstrate significant gains over
norm-minimization techniques for both task performance and agreement with human
rationales. Furthermore, we find that in the semi-supervised setting, a modest
amount of gold rationales (25% of training examples) closes the gap with a
model that uses the full input.Comment: EMNLP 2020 main track accepted pape
Advances in Variational Inference
Many modern unsupervised or semi-supervised machine learning algorithms rely
on Bayesian probabilistic models. These models are usually intractable and thus
require approximate inference. Variational inference (VI) lets us approximate a
high-dimensional Bayesian posterior with a simpler variational distribution by
solving an optimization problem. This approach has been successfully used in
various models and large-scale applications. In this review, we give an
overview of recent trends in variational inference. We first introduce standard
mean field variational inference, then review recent advances focusing on the
following aspects: (a) scalable VI, which includes stochastic approximations,
(b) generic VI, which extends the applicability of VI to a large class of
otherwise intractable models, such as non-conjugate models, (c) accurate VI,
which includes variational models beyond the mean field approximation or with
atypical divergences, and (d) amortized VI, which implements the inference over
local latent variables with inference networks. Finally, we provide a summary
of promising future research directions
Information Bottleneck and its Applications in Deep Learning
Information Theory (IT) has been used in Machine Learning (ML) from early
days of this field. In the last decade, advances in Deep Neural Networks (DNNs)
have led to surprising improvements in many applications of ML. The result has
been a paradigm shift in the community toward revisiting previous ideas and
applications in this new framework. Ideas from IT are no exception. One of the
ideas which is being revisited by many researchers in this new era, is
Information Bottleneck (IB); a formulation of information extraction based on
IT. The IB is promising in both analyzing and improving DNNs. The goal of this
survey is to review the IB concept and demonstrate its applications in deep
learning. The information theoretic nature of IB, makes it also a good
candidate in showing the more general concept of how IT can be used in ML. Two
important concepts are highlighted in this narrative on the subject, i) the
concise and universal view that IT provides on seemingly unrelated methods of
ML, demonstrated by explaining how IB relates to minimal sufficient statistics,
stochastic gradient descent, and variational auto-encoders, and ii) the common
technical mistakes and problems caused by applying ideas from IT, which is
discussed by a careful study of some recent methods suffering from them
Guess First to Enable Better Compression and Adversarial Robustness
Machine learning models are generally vulnerable to adversarial examples,
which is in contrast to the robustness of humans. In this paper, we try to
leverage one of the mechanisms in human recognition and propose a bio-inspired
classification framework in which model inference is conditioned on label
hypothesis. We provide a class of training objectives for this framework and an
information bottleneck regularizer which utilizes the advantage that label
information can be discarded during inference. This framework enables better
compression of the mutual information between inputs and latent representations
without loss of learning capacity, at the cost of tractable inference
complexity. Better compression and elimination of label information further
bring better adversarial robustness without loss of natural accuracy, which is
demonstrated in the experiment.Comment: Accepted by NeurIPS 2019 workshop on Information Theory and Machine
Learnin
Understanding Autoencoders with Information Theoretic Concepts
Despite their great success in practical applications, there is still a lack
of theoretical and systematic methods to analyze deep neural networks. In this
paper, we illustrate an advanced information theoretic methodology to
understand the dynamics of learning and the design of autoencoders, a special
type of deep learning architectures that resembles a communication channel. By
generalizing the information plane to any cost function, and inspecting the
roles and dynamics of different layers using layer-wise information quantities,
we emphasize the role that mutual information plays in quantifying learning
from data. We further suggest and also experimentally validate, for mean square
error training, three fundamental properties regarding the layer-wise flow of
information and intrinsic dimensionality of the bottleneck layer, using
respectively the data processing inequality and the identification of a
bifurcation point in the information plane that is controlled by the given
data. Our observations have a direct impact on the optimal design of
autoencoders, the design of alternative feedforward training methods, and even
in the problem of generalization.Comment: Paper accepted by Neural Networks. Code for estimating information
quantities and drawing the information plane is available from
https://drive.google.com/drive/folders/1e5sIywZfmWp4Dn0WEesb6fqQRM0DIGxZ?usp=sharin
Restricting the Flow: Information Bottlenecks for Attribution
Attribution methods provide insights into the decision-making of machine
learning models like artificial neural networks. For a given input sample, they
assign a relevance score to each individual input variable, such as the pixels
of an image. In this work we adapt the information bottleneck concept for
attribution. By adding noise to intermediate feature maps we restrict the flow
of information and can quantify (in bits) how much information image regions
provide. We compare our method against ten baselines using three different
metrics on VGG-16 and ResNet-50, and find that our methods outperform all
baselines in five out of six settings. The method's information-theoretic
foundation provides an absolute frame of reference for attribution values
(bits) and a guarantee that regions scored close to zero are not necessary for
the network's decision. For reviews: https://openreview.net/forum?id=S1xWh1rYwB
For code: https://github.com/BioroboticsLab/IBAComment: 18 pages, 12 figures, accepted at ICLR 2020 (Oral
Adversarial Neural Pruning with Latent Vulnerability Suppression
Despite the remarkable performance of deep neural networks on various
computer vision tasks, they are known to be susceptible to adversarial
perturbations, which makes it challenging to deploy them in real-world
safety-critical applications. In this paper, we conjecture that the leading
cause of adversarial vulnerability is the distortion in the latent feature
space, and provide methods to suppress them effectively. Explicitly, we define
\emph{vulnerability} for each latent feature and then propose a new loss for
adversarial learning, \emph{Vulnerability Suppression (VS)} loss, that aims to
minimize the feature-level vulnerability during training. We further propose a
Bayesian framework to prune features with high vulnerability to reduce both
vulnerability and loss on adversarial samples. We validate our
\emph{Adversarial Neural Pruning with Vulnerability Suppression (ANP-VS)}
method on multiple benchmark datasets, on which it not only obtains
state-of-the-art adversarial robustness but also improves the performance on
clean examples, using only a fraction of the parameters used by the full
network. Further qualitative analysis suggests that the improvements come from
the suppression of feature-level vulnerability.Comment: Accepted to ICML 2020. Code available at
https://github.com/divyam3897/ANP_V
Explainable Machine Learning for Scientific Insights and Discoveries
Machine learning methods have been remarkably successful for a wide range of
application areas in the extraction of essential information from data. An
exciting and relatively recent development is the uptake of machine learning in
the natural sciences, where the major goal is to obtain novel scientific
insights and discoveries from observational or simulated data. A prerequisite
for obtaining a scientific outcome is domain knowledge, which is needed to gain
explainability, but also to enhance scientific consistency. In this article we
review explainable machine learning in view of applications in the natural
sciences and discuss three core elements which we identified as relevant in
this context: transparency, interpretability, and explainability. With respect
to these core elements, we provide a survey of recent scientific works that
incorporate machine learning and the way that explainable machine learning is
used in combination with domain knowledge from the application areas
Excessive Invariance Causes Adversarial Vulnerability
Despite their impressive performance, deep neural networks exhibit striking
failures on out-of-distribution inputs. One core idea of adversarial example
research is to reveal neural network errors under such distribution shifts. We
decompose these errors into two complementary sources: sensitivity and
invariance. We show deep networks are not only too sensitive to task-irrelevant
changes of their input, as is well-known from epsilon-adversarial examples, but
are also too invariant to a wide range of task-relevant changes, thus making
vast regions in input space vulnerable to adversarial attacks. We show such
excessive invariance occurs across various tasks and architecture types. On
MNIST and ImageNet one can manipulate the class-specific content of almost any
image without changing the hidden activations. We identify an insufficiency of
the standard cross-entropy loss as a reason for these failures. Further, we
extend this objective based on an information-theoretic analysis so it
encourages the model to consider all task-dependent features in its decision.
This provides the first approach tailored explicitly to overcome excessive
invariance and resulting vulnerabilities
- …