6,638 research outputs found
Discriminative Recurrent Sparse Auto-Encoders
We present the discriminative recurrent sparse auto-encoder model, comprising
a recurrent encoder of rectified linear units, unrolled for a fixed number of
iterations, and connected to two linear decoders that reconstruct the input and
predict its supervised classification. Training via
backpropagation-through-time initially minimizes an unsupervised sparse
reconstruction error; the loss function is then augmented with a discriminative
term on the supervised classification. The depth implicit in the
temporally-unrolled form allows the system to exhibit all the power of deep
networks, while substantially reducing the number of trainable parameters.
From an initially unstructured network the hidden units differentiate into
categorical-units, each of which represents an input prototype with a
well-defined class; and part-units representing deformations of these
prototypes. The learned organization of the recurrent encoder is hierarchical:
part-units are driven directly by the input, whereas the activity of
categorical-units builds up over time through interactions with the part-units.
Even using a small number of hidden units per layer, discriminative recurrent
sparse auto-encoders achieve excellent performance on MNIST.Comment: Added clarifications suggested by reviewers. 15 pages, 10 figure
Learning Discriminative Features with Class Encoder
Deep neural networks usually benefit from unsupervised pre-training, e.g.
auto-encoders. However, the classifier further needs supervised fine-tuning
methods for good discrimination. Besides, due to the limits of full-connection,
the application of auto-encoders is usually limited to small, well aligned
images. In this paper, we incorporate the supervised information to propose a
novel formulation, namely class-encoder, whose training objective is to
reconstruct a sample from another one of which the labels are identical.
Class-encoder aims to minimize the intra-class variations in the feature space,
and to learn a good discriminative manifolds on a class scale. We impose the
class-encoder as a constraint into the softmax for better supervised training,
and extend the reconstruction on feature-level to tackle the parameter size
issue and translation issue. The experiments show that the class-encoder helps
to improve the performance on benchmarks of classification and face
recognition. This could also be a promising direction for fast training of face
recognition models.Comment: Accepted by CVPR2016 Workshop of Robust Features for Computer Visio
Disentangled Variational Auto-Encoder for Semi-supervised Learning
Semi-supervised learning is attracting increasing attention due to the fact
that datasets of many domains lack enough labeled data. Variational
Auto-Encoder (VAE), in particular, has demonstrated the benefits of
semi-supervised learning. The majority of existing semi-supervised VAEs utilize
a classifier to exploit label information, where the parameters of the
classifier are introduced to the VAE. Given the limited labeled data, learning
the parameters for the classifiers may not be an optimal solution for
exploiting label information. Therefore, in this paper, we develop a novel
approach for semi-supervised VAE without classifier. Specifically, we propose a
new model called Semi-supervised Disentangled VAE (SDVAE), which encodes the
input data into disentangled representation and non-interpretable
representation, then the category information is directly utilized to
regularize the disentangled representation via the equality constraint. To
further enhance the feature learning ability of the proposed VAE, we
incorporate reinforcement learning to relieve the lack of data. The dynamic
framework is capable of dealing with both image and text data with its
corresponding encoder and decoder networks. Extensive experiments on image and
text datasets demonstrate the effectiveness of the proposed framework.Comment: 6 figures, 10 pages, Information Sciences 201
Generating Visual Representations for Zero-Shot Classification
This paper addresses the task of learning an image clas-sifier when some
categories are defined by semantic descriptions only (e.g. visual attributes)
while the others are defined by exemplar images as well. This task is often
referred to as the Zero-Shot classification task (ZSC). Most of the previous
methods rely on learning a common embedding space allowing to compare visual
features of unknown categories with semantic descriptions. This paper argues
that these approaches are limited as i) efficient discrimi-native classifiers
can't be used ii) classification tasks with seen and unseen categories
(Generalized Zero-Shot Classification or GZSC) can't be addressed efficiently.
In contrast , this paper suggests to address ZSC and GZSC by i) learning a
conditional generator using seen classes ii) generate artificial training
examples for the categories without exemplars. ZSC is then turned into a
standard supervised learning problem. Experiments with 4 generative models and
5 datasets experimentally validate the approach, giving state-of-the-art
results on both ZSC and GZSC
- …