4,010 research outputs found
Collaborative Layer-wise Discriminative Learning in Deep Neural Networks
Intermediate features at different layers of a deep neural network are known
to be discriminative for visual patterns of different complexities. However,
most existing works ignore such cross-layer heterogeneities when classifying
samples of different complexities. For example, if a training sample has
already been correctly classified at a specific layer with high confidence, we
argue that it is unnecessary to enforce rest layers to classify this sample
correctly and a better strategy is to encourage those layers to focus on other
samples.
In this paper, we propose a layer-wise discriminative learning method to
enhance the discriminative capability of a deep network by allowing its layers
to work collaboratively for classification. Towards this target, we introduce
multiple classifiers on top of multiple layers. Each classifier not only tries
to correctly classify the features from its input layer, but also coordinates
with other classifiers to jointly maximize the final classification
performance. Guided by the other companion classifiers, each classifier learns
to concentrate on certain training examples and boosts the overall performance.
Allowing for end-to-end training, our method can be conveniently embedded into
state-of-the-art deep networks. Experiments with multiple popular deep
networks, including Network in Network, GoogLeNet and VGGNet, on scale-various
object classification benchmarks, including CIFAR100, MNIST and ImageNet, and
scene classification benchmarks, including MIT67, SUN397 and Places205,
demonstrate the effectiveness of our method. In addition, we also analyze the
relationship between the proposed method and classical conditional random
fields models.Comment: To appear in ECCV 2016. Maybe subject to minor changes before
camera-ready versio
Deep Learning based Recommender System: A Survey and New Perspectives
With the ever-growing volume of online information, recommender systems have
been an effective strategy to overcome such information overload. The utility
of recommender systems cannot be overstated, given its widespread adoption in
many web applications, along with its potential impact to ameliorate many
problems related to over-choice. In recent years, deep learning has garnered
considerable interest in many research fields such as computer vision and
natural language processing, owing not only to stellar performance but also the
attractive property of learning feature representations from scratch. The
influence of deep learning is also pervasive, recently demonstrating its
effectiveness when applied to information retrieval and recommender systems
research. Evidently, the field of deep learning in recommender system is
flourishing. This article aims to provide a comprehensive review of recent
research efforts on deep learning based recommender systems. More concretely,
we provide and devise a taxonomy of deep learning based recommendation models,
along with providing a comprehensive summary of the state-of-the-art. Finally,
we expand on current trends and provide new perspectives pertaining to this new
exciting development of the field.Comment: The paper has been accepted by ACM Computing Surveys.
https://doi.acm.org/10.1145/328502
Collaborative Deep Learning for Speech Enhancement: A Run-Time Model Selection Method Using Autoencoders
We show that a Modular Neural Network (MNN) can combine various speech
enhancement modules, each of which is a Deep Neural Network (DNN) specialized
on a particular enhancement job. Differently from an ordinary ensemble
technique that averages variations in models, the propose MNN selects the best
module for the unseen test signal to produce a greedy ensemble. We see this as
Collaborative Deep Learning (CDL), because it can reuse various already-trained
DNN models without any further refining. In the proposed MNN selecting the best
module during run time is challenging. To this end, we employ a speech
AutoEncoder (AE) as an arbitrator, whose input and output are trained to be as
similar as possible if its input is clean speech. Therefore, the AE can gauge
the quality of the module-specific denoised result by seeing its AE
reconstruction error, e.g. low error means that the module output is similar to
clean speech. We propose an MNN structure with various modules that are
specialized on dealing with a specific noise type, gender, and input
Signal-to-Noise Ratio (SNR) value, and empirically prove that it almost always
works better than an arbitrarily chosen DNN module and sometimes as good as an
oracle result
Domain Conditioned Adaptation Network
Tremendous research efforts have been made to thrive deep domain adaptation
(DA) by seeking domain-invariant features. Most existing deep DA models only
focus on aligning feature representations of task-specific layers across
domains while integrating a totally shared convolutional architecture for
source and target. However, we argue that such strongly-shared convolutional
layers might be harmful for domain-specific feature learning when source and
target data distribution differs to a large extent. In this paper, we relax a
shared-convnets assumption made by previous DA methods and propose a Domain
Conditioned Adaptation Network (DCAN), which aims to excite distinct
convolutional channels with a domain conditioned channel attention mechanism.
As a result, the critical low-level domain-dependent knowledge could be
explored appropriately. As far as we know, this is the first work to explore
the domain-wise convolutional channel activation for deep DA networks.
Moreover, to effectively align high-level feature distributions across two
domains, we further deploy domain conditioned feature correction blocks after
task-specific layers, which will explicitly correct the domain discrepancy.
Extensive experiments on three cross-domain benchmarks demonstrate the proposed
approach outperforms existing methods by a large margin, especially on very
tough cross-domain learning tasks.Comment: Accepted by AAAI 202
State-of-the-art and gaps for deep learning on limited training data in remote sensing
Deep learning usually requires big data, with respect to both volume and
variety. However, most remote sensing applications only have limited training
data, of which a small subset is labeled. Herein, we review three
state-of-the-art approaches in deep learning to combat this challenge. The
first topic is transfer learning, in which some aspects of one domain, e.g.,
features, are transferred to another domain. The next is unsupervised learning,
e.g., autoencoders, which operate on unlabeled data. The last is generative
adversarial networks, which can generate realistic looking data that can fool
the likes of both a deep learning network and human. The aim of this article is
to raise awareness of this dilemma, to direct the reader to existing work and
to highlight current gaps that need solving.Comment: arXiv admin note: text overlap with arXiv:1709.0030
- …