145 research outputs found
Learning Invariant Representations with Local Transformations
Learning invariant representations is an important problem in machine
learning and pattern recognition. In this paper, we present a novel framework
of transformation-invariant feature learning by incorporating linear
transformations into the feature learning algorithms. For example, we present
the transformation-invariant restricted Boltzmann machine that compactly
represents data by its weights and their transformations, which achieves
invariance of the feature representation via probabilistic max pooling. In
addition, we show that our transformation-invariant feature learning framework
can also be extended to other unsupervised learning methods, such as
autoencoders or sparse coding. We evaluate our method on several image
classification benchmark datasets, such as MNIST variations, CIFAR-10, and
STL-10, and show competitive or superior classification performance when
compared to the state-of-the-art. Furthermore, our method achieves
state-of-the-art performance on phone classification tasks with the TIMIT
dataset, which demonstrates wide applicability of our proposed algorithms to
other domains.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012
Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification
Unsupervised learning and supervised learning are key research topics in deep
learning. However, as high-capacity supervised neural networks trained with a
large amount of labels have achieved remarkable success in many computer vision
tasks, the availability of large-scale labeled images reduced the significance
of unsupervised learning. Inspired by the recent trend toward revisiting the
importance of unsupervised learning, we investigate joint supervised and
unsupervised learning in a large-scale setting by augmenting existing neural
networks with decoding pathways for reconstruction. First, we demonstrate that
the intermediate activations of pretrained large-scale classification networks
preserve almost all the information of input images except a portion of local
spatial details. Then, by end-to-end training of the entire augmented
architecture with the reconstructive objective, we show improvement of the
network performance for supervised tasks. We evaluate several variants of
autoencoders, including the recently proposed "what-where" autoencoder that
uses the encoder pooling switches, to study the importance of the architecture
design. Taking the 16-layer VGGNet trained under the ImageNet ILSVRC 2012
protocol as a strong baseline for image classification, our methods improve the
validation-set accuracy by a noticeable margin.Comment: International Conference on Machine Learning (ICML), 201
Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild
Lifelong learning with deep neural networks is well-known to suffer from
catastrophic forgetting: the performance on previous tasks drastically degrades
when learning a new task. To alleviate this effect, we propose to leverage a
large stream of unlabeled data easily obtainable in the wild. In particular, we
design a novel class-incremental learning scheme with (a) a new distillation
loss, termed global distillation, (b) a learning strategy to avoid overfitting
to the most recent task, and (c) a confidence-based sampling method to
effectively leverage unlabeled external data. Our experimental results on
various datasets, including CIFAR and ImageNet, demonstrate the superiority of
the proposed methods over prior methods, particularly when a stream of
unlabeled data is accessible: our method shows up to 15.8% higher accuracy and
46.5% less forgetting compared to the state-of-the-art method. The code is
available at https://github.com/kibok90/iccv2019-inc.Comment: ICCV 2019; v3 updated Figure
Value Prediction Network
This paper proposes a novel deep reinforcement learning (RL) architecture,
called Value Prediction Network (VPN), which integrates model-free and
model-based RL methods into a single neural network. In contrast to typical
model-based RL methods, VPN learns a dynamics model whose abstract states are
trained to make option-conditional predictions of future values (discounted sum
of rewards) rather than of future observations. Our experimental results show
that VPN has several advantages over both model-free and model-based baselines
in a stochastic environment where careful planning is required but building an
accurate observation-prediction model is difficult. Furthermore, VPN
outperforms Deep Q-Network (DQN) on several Atari games even with
short-lookahead planning, demonstrating its potential as a new way of learning
a good state representation.Comment: NIPS 201
Content preserving text generation with attribute controls
In this work, we address the problem of modifying textual attributes of
sentences. Given an input sentence and a set of attribute labels, we attempt to
generate sentences that are compatible with the conditioning information. To
ensure that the model generates content compatible sentences, we introduce a
reconstruction loss which interpolates between auto-encoding and
back-translation loss components. We propose an adversarial loss to enforce
generated samples to be attribute compatible and realistic. Through
quantitative, qualitative and human evaluations we demonstrate that our model
is capable of generating fluent sentences that better reflect the conditioning
information compared to prior methods. We further demonstrate that the model is
capable of simultaneously controlling multiple attributes.Comment: NIPS 201
Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
The problem of detecting whether a test sample is from in-distribution (i.e.,
training distribution by a classifier) or out-of-distribution sufficiently
different from it arises in many real-world machine learning applications.
However, the state-of-art deep neural networks are known to be highly
overconfident in their predictions, i.e., do not distinguish in- and
out-of-distributions. Recently, to handle this issue, several threshold-based
detectors have been proposed given pre-trained neural classifiers. However, the
performance of prior works highly depends on how to train the classifiers since
they only focus on improving inference procedures. In this paper, we develop a
novel training method for classifiers so that such inference algorithms can
work better. In particular, we suggest two additional terms added to the
original loss (e.g., cross entropy). The first one forces samples from
out-of-distribution less confident by the classifier and the second one is for
(implicitly) generating most effective training samples for the first one. In
essence, our method jointly trains both classification and generative neural
networks for out-of-distribution. We demonstrate its effectiveness using deep
convolutional neural networks on various popular image datasets
Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning
Model-based reinforcement learning (RL) enjoys several benefits, such as
data-efficiency and planning, by learning a model of the environment's
dynamics. However, learning a global model that can generalize across different
dynamics is a challenging task. To tackle this problem, we decompose the task
of learning a global dynamics model into two stages: (a) learning a context
latent vector that captures the local dynamics, then (b) predicting the next
state conditioned on it. In order to encode dynamics-specific information into
the context latent vector, we introduce a novel loss function that encourages
the context latent vector to be useful for predicting both forward and backward
dynamics. The proposed method achieves superior generalization ability across
various simulated robotics and control tasks, compared to existing RL schemes.Comment: Accepted in ICML2020. First two authors contributed equally, website:
https://sites.google.com/view/cadm code: https://github.com/younggyoseo/CaD
Near-Optimal Representation Learning for Hierarchical Reinforcement Learning
We study the problem of representation learning in goal-conditioned
hierarchical reinforcement learning. In such hierarchical structures, a
higher-level controller solves tasks by iteratively communicating goals which a
lower-level policy is trained to reach. Accordingly, the choice of
representation -- the mapping of observation space to goal space -- is crucial.
To study this problem, we develop a notion of sub-optimality of a
representation, defined in terms of expected reward of the optimal hierarchical
policy using this representation. We derive expressions which bound the
sub-optimality and show how these expressions can be translated to
representation learning objectives which may be optimized in practice. Results
on a number of difficult continuous-control tasks show that our approach to
representation learning yields qualitatively better representations as well as
quantitatively better hierarchical policies, compared to existing methods (see
videos at https://sites.google.com/view/representation-hrl).Comment: ICLR 2019 Conference Pape
Similarity of Neural Network Representations Revisited
Recent work has sought to understand the behavior of neural networks by
comparing representations between layers and between different trained models.
We examine methods for comparing neural network representations based on
canonical correlation analysis (CCA). We show that CCA belongs to a family of
statistics for measuring multivariate similarity, but that neither CCA nor any
other statistic that is invariant to invertible linear transformation can
measure meaningful similarities between representations of higher dimension
than the number of data points. We introduce a similarity index that measures
the relationship between representational similarity matrices and does not
suffer from this limitation. This similarity index is equivalent to centered
kernel alignment (CKA) and is also closely connected to CCA. Unlike CCA, CKA
can reliably identify correspondences between representations in networks
trained from different initializations.Comment: ICML 201
Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units
Recently, convolutional neural networks (CNNs) have been used as a powerful
tool to solve many problems of machine learning and computer vision. In this
paper, we aim to provide insight on the property of convolutional neural
networks, as well as a generic method to improve the performance of many CNN
architectures. Specifically, we first examine existing CNN models and observe
an intriguing property that the filters in the lower layers form pairs (i.e.,
filters with opposite phase). Inspired by our observation, we propose a novel,
simple yet effective activation scheme called concatenated ReLU (CRelu) and
theoretically analyze its reconstruction property in CNNs. We integrate CRelu
into several state-of-the-art CNN architectures and demonstrate improvement in
their recognition performance on CIFAR-10/100 and ImageNet datasets with fewer
trainable parameters. Our results suggest that better understanding of the
properties of CNNs can lead to significant performance improvement with a
simple modification.Comment: ICML 201
- …