1,046 research outputs found
Random projections for Bayesian regression
This article deals with random projections applied as a data reduction
technique for Bayesian regression analysis. We show sufficient conditions under
which the entire -dimensional distribution is approximately preserved under
random projections by reducing the number of data points from to in the case . Under mild
assumptions, we prove that evaluating a Gaussian likelihood function based on
the projected data instead of the original data yields a
-approximation in terms of the Wasserstein
distance. Our main result shows that the posterior distribution of Bayesian
linear regression is approximated up to a small error depending on only an
-fraction of its defining parameters. This holds when using
arbitrary Gaussian priors or the degenerate case of uniform distributions over
for . Our empirical evaluations involve different
simulated settings of Bayesian linear regression. Our experiments underline
that the proposed method is able to recover the regression model up to small
error while considerably reducing the total running time
Optimal Transport for Domain Adaptation
Domain adaptation from one data space (or domain) to another is one of the
most challenging tasks of modern data analytics. If the adaptation is done
correctly, models built on a specific data space become more robust when
confronted to data depicting the same semantic concepts (the classes), but
observed by another observation system with its own specificities. Among the
many strategies proposed to adapt a domain to another, finding a common
representation has shown excellent properties: by finding a common
representation for both domains, a single classifier can be effective in both
and use labelled samples from the source domain to predict the unlabelled
samples of the target domain. In this paper, we propose a regularized
unsupervised optimal transportation model to perform the alignment of the
representations in the source and target domains. We learn a transportation
plan matching both PDFs, which constrains labelled samples in the source domain
to remain close during transport. This way, we exploit at the same time the few
labeled information in the source and the unlabelled distributions observed in
both domains. Experiments in toy and challenging real visual adaptation
examples show the interest of the method, that consistently outperforms state
of the art approaches
Projection Robust Wasserstein Distance and Riemannian Optimization
Projection robust Wasserstein (PRW) distance, or Wasserstein projection
pursuit (WPP), is a robust variant of the Wasserstein distance. Recent work
suggests that this quantity is more robust than the standard Wasserstein
distance, in particular when comparing probability measures in high-dimensions.
However, it is ruled out for practical application because the optimization
model is essentially non-convex and non-smooth which makes the computation
intractable. Our contribution in this paper is to revisit the original
motivation behind WPP/PRW, but take the hard route of showing that, despite its
non-convexity and lack of nonsmoothness, and even despite some hardness results
proved by~\citet{Niles-2019-Estimation} in a minimax sense, the original
formulation for PRW/WPP \textit{can} be efficiently computed in practice using
Riemannian optimization, yielding in relevant cases better behavior than its
convex relaxation. More specifically, we provide three simple algorithms with
solid theoretical guarantee on their complexity bound (one in the appendix),
and demonstrate their effectiveness and efficiency by conducing extensive
experiments on synthetic and real data. This paper provides a first step into a
computational theory of the PRW distance and provides the links between optimal
transport and Riemannian optimization.Comment: Accepted by NeurIPS 2020; The first two authors contributed equally;
fix the confusing parts in the proof and refine the algorithms and complexity
bound
- âŠ