50,423 research outputs found
Target Contrastive Pessimistic Discriminant Analysis
Domain-adaptive classifiers learn from a source domain and aim to generalize
to a target domain. If the classifier's assumptions on the relationship between
domains (e.g. covariate shift) are valid, then it will usually outperform a
non-adaptive source classifier. Unfortunately, it can perform substantially
worse when its assumptions are invalid. Validating these assumptions requires
labeled target samples, which are usually not available. We argue that, in
order to make domain-adaptive classifiers more practical, it is necessary to
focus on robust methods; robust in the sense that the model still achieves a
particular level of performance without making strong assumptions on the
relationship between domains. With this objective in mind, we formulate a
conservative parameter estimator that only deviates from the source classifier
when a lower or equal risk is guaranteed for all possible labellings of the
given target samples. We derive the corresponding estimator for a discriminant
analysis model, and show that its risk is actually strictly smaller than that
of the source classifier. Experiments indicate that our classifier outperforms
state-of-the-art classifiers for geographically biased samples.Comment: 9 pages, no figures, 2 tables. arXiv admin note: substantial text
overlap with arXiv:1706.0808
Asymptotic Normality of Support Vector Machine Variants and Other Regularized Kernel Methods
In nonparametric classification and regression problems, regularized kernel
methods, in particular support vector machines, attract much attention in
theoretical and in applied statistics. In an abstract sense, regularized kernel
methods (simply called SVMs here) can be seen as regularized M-estimators for a
parameter in a (typically infinite dimensional) reproducing kernel Hilbert
space. For smooth loss functions, it is shown that the difference between the
estimator, i.e.\ the empirical SVM, and the theoretical SVM is asymptotically
normal with rate . That is, the standardized difference converges
weakly to a Gaussian process in the reproducing kernel Hilbert space. As common
in real applications, the choice of the regularization parameter may depend on
the data. The proof is done by an application of the functional delta-method
and by showing that the SVM-functional is suitably Hadamard-differentiable
Domain Generalization by Marginal Transfer Learning
In the problem of domain generalization (DG), there are labeled training data
sets from several related prediction problems, and the goal is to make accurate
predictions on future unlabeled data sets that are not known to the learner.
This problem arises in several applications where data distributions fluctuate
because of environmental, technical, or other sources of variation. We
introduce a formal framework for DG, and argue that it can be viewed as a kind
of supervised learning problem by augmenting the original feature space with
the marginal distribution of feature vectors. While our framework has several
connections to conventional analysis of supervised learning algorithms, several
unique aspects of DG require new methods of analysis.
This work lays the learning theoretic foundations of domain generalization,
building on our earlier conference paper where the problem of DG was introduced
Blanchard et al., 2011. We present two formal models of data generation,
corresponding notions of risk, and distribution-free generalization error
analysis. By focusing our attention on kernel methods, we also provide more
quantitative results and a universally consistent algorithm. An efficient
implementation is provided for this algorithm, which is experimentally compared
to a pooling strategy on one synthetic and three real-world data sets
- …