33,101 research outputs found
Casting a BAIT for Offline and Online Source-free Domain Adaptation
We address the source-free domain adaptation (SFDA) problem, where only the
source model is available during adaptation to the target domain. We consider
two settings: the offline setting where all target data can be visited multiple
times (epochs) to arrive at a prediction for each target sample, and the online
setting where the target data needs to be directly classified upon arrival.
Inspired by diverse classifier based domain adaptation methods, in this paper
we introduce a second classifier, but with another classifier head fixed. When
adapting to the target domain, the additional classifier initialized from
source classifier is expected to find misclassified features. Next, when
updating the feature extractor, those features will be pushed towards the right
side of the source decision boundary, thus achieving source-free domain
adaptation. Experimental results show that the proposed method achieves
competitive results for offline SFDA on several benchmark datasets compared
with existing DA and SFDA methods, and our method surpasses by a large margin
other SFDA methods under online source-free domain adaptation setting
Self-Training: A Survey
Semi-supervised algorithms aim to learn prediction functions from a small set
of labeled observations and a large set of unlabeled observations. Because this
framework is relevant in many applications, they have received a lot of
interest in both academia and industry. Among the existing techniques,
self-training methods have undoubtedly attracted greater attention in recent
years. These models are designed to find the decision boundary on low density
regions without making additional assumptions about the data distribution, and
use the unsigned output score of a learned classifier, or its margin, as an
indicator of confidence. The working principle of self-training algorithms is
to learn a classifier iteratively by assigning pseudo-labels to the set of
unlabeled training samples with a margin greater than a certain threshold. The
pseudo-labeled examples are then used to enrich the labeled training data and
to train a new classifier in conjunction with the labeled training set. In this
paper, we present self-training methods for binary and multi-class
classification; as well as their variants and two related approaches, namely
consistency-based approaches and transductive learning. We examine the impact
of significant self-training features on various methods, using different
general and image classification benchmarks, and we discuss our ideas for
future research in self-training. To the best of our knowledge, this is the
first thorough and complete survey on this subject.Comment: 18 pages, 1 figur
Learning Dynamic Feature Selection for Fast Sequential Prediction
We present paired learning and inference algorithms for significantly
reducing computation and increasing speed of the vector dot products in the
classifiers that are at the heart of many NLP components. This is accomplished
by partitioning the features into a sequence of templates which are ordered
such that high confidence can often be reached using only a small fraction of
all features. Parameter estimation is arranged to maximize accuracy and early
confidence in this sequence. Our approach is simpler and better suited to NLP
than other related cascade methods. We present experiments in left-to-right
part-of-speech tagging, named entity recognition, and transition-based
dependency parsing. On the typical benchmarking datasets we can preserve POS
tagging accuracy above 97% and parsing LAS above 88.5% both with over a
five-fold reduction in run-time, and NER F1 above 88 with more than 2x increase
in speed.Comment: Appears in The 53rd Annual Meeting of the Association for
Computational Linguistics, Beijing, China, July 201
Supersparse Linear Integer Models for Optimized Medical Scoring Systems
Scoring systems are linear classification models that only require users to
add, subtract and multiply a few small numbers in order to make a prediction.
These models are in widespread use by the medical community, but are difficult
to learn from data because they need to be accurate and sparse, have coprime
integer coefficients, and satisfy multiple operational constraints. We present
a new method for creating data-driven scoring systems called a Supersparse
Linear Integer Model (SLIM). SLIM scoring systems are built by solving an
integer program that directly encodes measures of accuracy (the 0-1 loss) and
sparsity (the -seminorm) while restricting coefficients to coprime
integers. SLIM can seamlessly incorporate a wide range of operational
constraints related to accuracy and sparsity, and can produce highly tailored
models without parameter tuning. We provide bounds on the testing and training
accuracy of SLIM scoring systems, and present a new data reduction technique
that can improve scalability by eliminating a portion of the training data
beforehand. Our paper includes results from a collaboration with the
Massachusetts General Hospital Sleep Laboratory, where SLIM was used to create
a highly tailored scoring system for sleep apnea screeningComment: This version reflects our findings on SLIM as of January 2016
(arXiv:1306.5860 and arXiv:1405.4047 are out-of-date). The final published
version of this articled is available at http://www.springerlink.co
RandomBoost: Simplified Multi-class Boosting through Randomization
We propose a novel boosting approach to multi-class classification problems,
in which multiple classes are distinguished by a set of random projection
matrices in essence. The approach uses random projections to alleviate the
proliferation of binary classifiers typically required to perform multi-class
classification. The result is a multi-class classifier with a single
vector-valued parameter, irrespective of the number of classes involved. Two
variants of this approach are proposed. The first method randomly projects the
original data into new spaces, while the second method randomly projects the
outputs of learned weak classifiers. These methods are not only conceptually
simple but also effective and easy to implement. A series of experiments on
synthetic, machine learning and visual recognition data sets demonstrate that
our proposed methods compare favorably to existing multi-class boosting
algorithms in terms of both the convergence rate and classification accuracy.Comment: 15 page
- …