11,597 research outputs found
Effectiveness of Hierarchical Softmax in Large Scale Classification Tasks
Typically, Softmax is used in the final layer of a neural network to get a
probability distribution for output classes. But the main problem with Softmax
is that it is computationally expensive for large scale data sets with large
number of possible outputs. To approximate class probability efficiently on
such large scale data sets we can use Hierarchical Softmax. LSHTC datasets were
used to study the performance of the Hierarchical Softmax. LSHTC datasets have
large number of categories. In this paper we evaluate and report the
performance of normal Softmax Vs Hierarchical Softmax on LSHTC datasets. This
evaluation used macro f1 score as a performance measure. The observation was
that the performance of Hierarchical Softmax degrades as the number of classes
increase
Learning Tree-based Deep Model for Recommender Systems
Model-based methods for recommender systems have been studied extensively in
recent years. In systems with large corpus, however, the calculation cost for
the learnt model to predict all user-item preferences is tremendous, which
makes full corpus retrieval extremely difficult. To overcome the calculation
barriers, models such as matrix factorization resort to inner product form
(i.e., model user-item preference as the inner product of user, item latent
factors) and indexes to facilitate efficient approximate k-nearest neighbor
searches. However, it still remains challenging to incorporate more expressive
interaction forms between user and item features, e.g., interactions through
deep neural networks, because of the calculation cost.
In this paper, we focus on the problem of introducing arbitrary advanced
models to recommender systems with large corpus. We propose a novel tree-based
method which can provide logarithmic complexity w.r.t. corpus size even with
more expressive models such as deep neural networks. Our main idea is to
predict user interests from coarse to fine by traversing tree nodes in a
top-down fashion and making decisions for each user-node pair. We also show
that the tree structure can be jointly learnt towards better compatibility with
users' interest distribution and hence facilitate both training and prediction.
Experimental evaluations with two large-scale real-world datasets show that the
proposed method significantly outperforms traditional methods. Online A/B test
results in Taobao display advertising platform also demonstrate the
effectiveness of the proposed method in production environments.Comment: Accepted by KDD 201
Collaborative Layer-wise Discriminative Learning in Deep Neural Networks
Intermediate features at different layers of a deep neural network are known
to be discriminative for visual patterns of different complexities. However,
most existing works ignore such cross-layer heterogeneities when classifying
samples of different complexities. For example, if a training sample has
already been correctly classified at a specific layer with high confidence, we
argue that it is unnecessary to enforce rest layers to classify this sample
correctly and a better strategy is to encourage those layers to focus on other
samples.
In this paper, we propose a layer-wise discriminative learning method to
enhance the discriminative capability of a deep network by allowing its layers
to work collaboratively for classification. Towards this target, we introduce
multiple classifiers on top of multiple layers. Each classifier not only tries
to correctly classify the features from its input layer, but also coordinates
with other classifiers to jointly maximize the final classification
performance. Guided by the other companion classifiers, each classifier learns
to concentrate on certain training examples and boosts the overall performance.
Allowing for end-to-end training, our method can be conveniently embedded into
state-of-the-art deep networks. Experiments with multiple popular deep
networks, including Network in Network, GoogLeNet and VGGNet, on scale-various
object classification benchmarks, including CIFAR100, MNIST and ImageNet, and
scene classification benchmarks, including MIT67, SUN397 and Places205,
demonstrate the effectiveness of our method. In addition, we also analyze the
relationship between the proposed method and classical conditional random
fields models.Comment: To appear in ECCV 2016. Maybe subject to minor changes before
camera-ready versio
Zero-Shot Learning by Convex Combination of Semantic Embeddings
Several recent publications have proposed methods for mapping images into
continuous semantic embedding spaces. In some cases the embedding space is
trained jointly with the image transformation. In other cases the semantic
embedding space is established by an independent natural language processing
task, and then the image transformation into that space is learned in a second
stage. Proponents of these image embedding systems have stressed their
advantages over the traditional \nway{} classification framing of image
understanding, particularly in terms of the promise for zero-shot learning --
the ability to correctly annotate images of previously unseen object
categories. In this paper, we propose a simple method for constructing an image
embedding system from any existing \nway{} image classifier and a semantic word
embedding model, which contains the \n class labels in its vocabulary. Our
method maps images into the semantic embedding space via convex combination of
the class label embedding vectors, and requires no additional training. We show
that this simple and direct method confers many of the advantages associated
with more complex image embedding schemes, and indeed outperforms state of the
art methods on the ImageNet zero-shot learning task
Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment
Facial action unit (AU) detection and face alignment are two highly
correlated tasks since facial landmarks can provide precise AU locations to
facilitate the extraction of meaningful local features for AU detection. Most
existing AU detection works often treat face alignment as a preprocessing and
handle the two tasks independently. In this paper, we propose a novel
end-to-end deep learning framework for joint AU detection and face alignment,
which has not been explored before. In particular, multi-scale shared features
are learned firstly, and high-level features of face alignment are fed into AU
detection. Moreover, to extract precise local features, we propose an adaptive
attention learning module to refine the attention map of each AU adaptively.
Finally, the assembled local features are integrated with face alignment
features and global features for AU detection. Experiments on BP4D and DISFA
benchmarks demonstrate that our framework significantly outperforms the
state-of-the-art methods for AU detection.Comment: This paper has been accepted by ECCV 201
- …