3,557 research outputs found
Mutual Exclusivity Loss for Semi-Supervised Deep Learning
In this paper we consider the problem of semi-supervised learning with deep
Convolutional Neural Networks (ConvNets). Semi-supervised learning is motivated
on the observation that unlabeled data is cheap and can be used to improve the
accuracy of classifiers. In this paper we propose an unsupervised
regularization term that explicitly forces the classifier's prediction for
multiple classes to be mutually-exclusive and effectively guides the decision
boundary to lie on the low density space between the manifolds corresponding to
different classes of data. Our proposed approach is general and can be used
with any backpropagation-based learning method. We show through different
experiments that our method can improve the object recognition performance of
ConvNets using unlabeled data.Comment: 5 pages, 1 figures, ICIP 201
A Simple Algorithm for Semi-supervised Learning with Improved Generalization Error Bound
In this work, we develop a simple algorithm for semi-supervised regression.
The key idea is to use the top eigenfunctions of integral operator derived from
both labeled and unlabeled examples as the basis functions and learn the
prediction function by a simple linear regression. We show that under
appropriate assumptions about the integral operator, this approach is able to
achieve an improved regression error bound better than existing bounds of
supervised learning. We also verify the effectiveness of the proposed algorithm
by an empirical study.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012
Kernel Analog Forecasting: Multiscale Test Problems
Data-driven prediction is becoming increasingly widespread as the volume of
data available grows and as algorithmic development matches this growth. The
nature of the predictions made, and the manner in which they should be
interpreted, depends crucially on the extent to which the variables chosen for
prediction are Markovian, or approximately Markovian. Multiscale systems
provide a framework in which this issue can be analyzed. In this work kernel
analog forecasting methods are studied from the perspective of data generated
by multiscale dynamical systems. The problems chosen exhibit a variety of
different Markovian closures, using both averaging and homogenization;
furthermore, settings where scale-separation is not present and the predicted
variables are non-Markovian, are also considered. The studies provide guidance
for the interpretation of data-driven prediction methods when used in practice.Comment: 30 pages, 14 figures; clarified several ambiguous parts, added
references, and a comparison with Lorenz' original method (Sec. 4.5
Applicability of semi-supervised learning assumptions for gene ontology terms prediction
Gene Ontology (GO) is one of the most important resources in bioinformatics, aiming to provide a unified framework for the biological annotation of genes and proteins across all species. Predicting GO terms is an essential task for bioinformatics, but the number of available labelled proteins is in several cases insufficient for training reliable machine learning classifiers. Semi-supervised learning methods arise as a powerful solution that explodes the information contained in unlabelled data in order to improve the estimations of traditional supervised approaches. However, semi-supervised learning methods have to make strong assumptions about the nature of the training data and thus, the performance of the predictor is highly dependent on these assumptions. This paper presents an analysis of the applicability of semi-supervised learning assumptions over the specific task of GO terms prediction, focused on providing judgment elements that allow choosing the most suitable tools for specific GO terms. The results show that semi-supervised approaches significantly outperform the traditional supervised methods and that the highest performances are reached when applying the cluster assumption. Besides, it is experimentally demonstrated that cluster and manifold assumptions are complimentary to each other and an analysis of which GO terms can be more prone to be correctly predicted with each assumption, is provided.Postprint (published version
- …