72 research outputs found
Semi-Supervised Learning by Augmented Distribution Alignment
In this work, we propose a simple yet effective semi-supervised learning
approach called Augmented Distribution Alignment. We reveal that an essential
sampling bias exists in semi-supervised learning due to the limited number of
labeled samples, which often leads to a considerable empirical distribution
mismatch between labeled data and unlabeled data. To this end, we propose to
align the empirical distributions of labeled and unlabeled data to alleviate
the bias. On one hand, we adopt an adversarial training strategy to minimize
the distribution distance between labeled and unlabeled data as inspired by
domain adaptation works. On the other hand, to deal with the small sample size
issue of labeled data, we also propose a simple interpolation strategy to
generate pseudo training samples. Those two strategies can be easily
implemented into existing deep neural networks. We demonstrate the
effectiveness of our proposed approach on the benchmark SVHN and CIFAR10
datasets. Our code is available at \url{https://github.com/qinenergy/adanet}.Comment: To appear in ICCV 201
A Kernel Test for Three-Variable Interactions
We introduce kernel nonparametric tests for Lancaster three-variable
interaction and for total independence, using embeddings of signed measures
into a reproducing kernel Hilbert space. The resulting test statistics are
straightforward to compute, and are used in powerful interaction tests, which
are consistent against all alternatives for a large family of reproducing
kernels. We show the Lancaster test to be sensitive to cases where two
independent causes individually have weak influence on a third dependent
variable, but their combined effect has a strong influence. This makes the
Lancaster test especially suited to finding structure in directed graphical
models, where it outperforms competing nonparametric tests in detecting such
V-structures
New normality test in high dimension with kernel methods
A new goodness-of-fit test for normality in high-dimension (and Reproducing
Kernel Hilbert Space) is proposed. It shares common ideas with the Maximum Mean
Discrepancy (MMD) it outperforms both in terms of computation time and
applicability to a wider range of data. Theoretical results are derived for the
Type-I and Type-II errors. They guarantee the control of Type-I error at
prescribed level and an exponentially fast decrease of the Type-II error.
Synthetic and real data also illustrate the practical improvement allowed by
our test compared with other leading approaches in high-dimensional settings
Approximate Bayesian computation via the energy statistic
Approximate Bayesian computation (ABC) has become an essential part of the
Bayesian toolbox for addressing problems in which the likelihood is
prohibitively expensive or entirely unknown, making it intractable. ABC defines
a pseudo-posterior by comparing observed data with simulated data,
traditionally based on some summary statistics, the elicitation of which is
regarded as a key difficulty. Recently, using data discrepancy measures has
been proposed in order to bypass the construction of summary statistics. Here
we propose to use the importance-sampling ABC (IS-ABC) algorithm relying on the
so-called two-sample energy statistic. We establish a new asymptotic result for
the case where both the observed sample size and the simulated data sample size
increase to infinity, which highlights to what extent the data discrepancy
measure impacts the asymptotic pseudo-posterior. The result holds in the broad
setting of IS-ABC methodologies, thus generalizing previous results that have
been established only for rejection ABC algorithms. Furthermore, we propose a
consistent V-statistic estimator of the energy statistic, under which we show
that the large sample result holds, and prove that the rejection ABC algorithm,
based on the energy statistic, generates pseudo-posterior distributions that
achieves convergence to the correct limits, when implemented with rejection
thresholds that converge to zero, in the finite sample setting. Our proposed
energy statistic based ABC algorithm is demonstrated on a variety of models,
including a Gaussian mixture, a moving-average model of order two, a bivariate
beta and a multivariate -and- distribution. We find that our proposed
method compares well with alternative discrepancy measures.Comment: 25 pages, 6 figures, 5 table
Fast Two-Sample Testing with Analytic Representations of Probability Measures
We propose a class of nonparametric two-sample tests with a cost linear in
the sample size. Two tests are given, both based on an ensemble of distances
between analytic functions representing each of the distributions. The first
test uses smoothed empirical characteristic functions to represent the
distributions, the second uses distribution embeddings in a reproducing kernel
Hilbert space. Analyticity implies that differences in the distributions may be
detected almost surely at a finite number of randomly chosen
locations/frequencies. The new tests are consistent against a larger class of
alternatives than the previous linear-time tests based on the (non-smoothed)
empirical characteristic functions, while being much faster than the current
state-of-the-art quadratic-time kernel-based or energy distance-based tests.
Experiments on artificial benchmarks and on challenging real-world testing
problems demonstrate that our tests give a better power/time tradeoff than
competing approaches, and in some cases, better outright power than even the
most expensive quadratic-time tests. This performance advantage is retained
even in high dimensions, and in cases where the difference in distributions is
not observable with low order statistics
Kernel-based Conditional Independence Test and Application in Causal Discovery
Conditional independence testing is an important problem, especially in
Bayesian network learning and causal discovery. Due to the curse of
dimensionality, testing for conditional independence of continuous variables is
particularly challenging. We propose a Kernel-based Conditional Independence
test (KCI-test), by constructing an appropriate test statistic and deriving its
asymptotic distribution under the null hypothesis of conditional independence.
The proposed method is computationally efficient and easy to implement.
Experimental results show that it outperforms other methods, especially when
the conditioning set is large or the sample size is not very large, in which
case other methods encounter difficulties
- …