34 research outputs found
A Kernel Independence Test for Random Processes
A new non parametric approach to the problem of testing the independence of
two random process is developed. The test statistic is the Hilbert Schmidt
Independence Criterion (HSIC), which was used previously in testing
independence for i.i.d pairs of variables. The asymptotic behaviour of HSIC is
established when computed from samples drawn from random processes. It is shown
that earlier bootstrap procedures which worked in the i.i.d. case will fail for
random processes, and an alternative consistent estimate of the p-values is
proposed. Tests on artificial data and real-world Forex data indicate that the
new test procedure discovers dependence which is missed by linear approaches,
while the earlier bootstrap procedure returns an elevated number of false
positives. The code is available online:
https://github.com/kacperChwialkowski/HSIC .Comment: In Proceedings of The 31st International Conference on Machine
Learnin
Topics in kernal hypothesis testing
This thesis investigates some unaddressed problems in kernel nonparametric hypothesis testing. The contributions are grouped around three main themes: Wild Bootstrap for Degenerate Kernel Tests. A wild bootstrap method for nonparametric hypothesis tests based on kernel distribution embeddings is proposed. This bootstrap method is used to construct provably consistent tests that apply to random processes. It applies to a large group of kernel tests based on V-statistics, which are degenerate under the null hypothesis, and non-degenerate elsewhere. In experiments, the wild bootstrap gives strong performance on synthetic examples, on audio data, and in performance benchmarking for the Gibbs sampler. A Kernel Test of Goodness of Fit. A nonparametric statistical test for goodness-of-fit is proposed: given a set of samples, the test determines how likely it is that these were generated from a target density function. The measure of goodness-of-fit is a divergence constructed via Stein's method using functions from a Reproducing Kernel Hilbert Space. Construction of the test is based on the wild bootstrap method. We apply our test to quantifying convergence of approximate Markov Chain Monte Carlo methods, statistical model criticism, and evaluating quality of fit vs model complexity in nonparametric density estimation. Fast Analytic Functions Based Two Sample Test. A class of nonparametric two-sample tests with a cost linear in the sample size is proposed. Two tests are given, both based on an ensemble of distances between analytic functions representing each of the distributions. Experiments on artificial benchmarks and on challenging real-world testing problems demonstrate good power/time tradeoff retained even in high dimensional problems. The main contributions to science are the following. We prove that the kernel tests based on the wild bootstrap method tightly control the type one error on the desired level and are consistent i.e. type two error drops to zero with increasing number of samples. We construct a kernel goodness of fit test that requires only knowledge of the density up to an normalizing constant. We use this test to construct first consistent test for convergence of Markov Chains and use it to quantify properties of approximate MCMC algorithms. Finally, we construct a linear time two-sample test that uses new, finite dimensional feature representation of probability measures
Interpretable Distribution Features with Maximum Testing Power
Two semimetrics on probability distributions are proposed, given as the sum
of differences of expectations of analytic functions evaluated at spatial or
frequency locations (i.e, features). The features are chosen so as to maximize
the distinguishability of the distributions, by optimizing a lower bound on
test power for a statistical test using these features. The result is a
parsimonious and interpretable indication of how and where two distributions
differ locally. An empirical estimate of the test power criterion converges
with increasing sample size, ensuring the quality of the returned features. In
real-world benchmarks on high-dimensional text and image data, linear-time
tests using the proposed semimetrics achieve comparable performance to the
state-of-the-art quadratic-time maximum mean discrepancy test, while returning
human-interpretable features that explain the test results
Fast Two-Sample Testing with Analytic Representations of Probability Measures
We propose a class of nonparametric two-sample tests with a cost linear in
the sample size. Two tests are given, both based on an ensemble of distances
between analytic functions representing each of the distributions. The first
test uses smoothed empirical characteristic functions to represent the
distributions, the second uses distribution embeddings in a reproducing kernel
Hilbert space. Analyticity implies that differences in the distributions may be
detected almost surely at a finite number of randomly chosen
locations/frequencies. The new tests are consistent against a larger class of
alternatives than the previous linear-time tests based on the (non-smoothed)
empirical characteristic functions, while being much faster than the current
state-of-the-art quadratic-time kernel-based or energy distance-based tests.
Experiments on artificial benchmarks and on challenging real-world testing
problems demonstrate that our tests give a better power/time tradeoff than
competing approaches, and in some cases, better outright power than even the
most expensive quadratic-time tests. This performance advantage is retained
even in high dimensions, and in cases where the difference in distributions is
not observable with low order statistics
A Kernel Test of Goodness of Fit
We propose a nonparametric statistical test for goodness-of-fit: given a set of samples, the test determines how likely it is that these were generated from a target density function. The measure of goodness-of-fit is a divergence constructed via Stein's method using functions from a Reproducing Kernel Hilbert Space. Our test statistic is based on an empirical estimate of this divergence, taking the form of a V-statistic in terms of the log gradients of the target density and the kernel. We derive a statistical test, both for i.i.d. and non-i.i.d. samples, where we estimate the null distribution quantiles using a wild bootstrap procedure. We apply our test to quantifying convergence of approximate Markov Chain Monte Carlo methods, statistical model criticism, and evaluating quality of fit vs model complexity in nonparametric density estimation
A Kernel Test for Three-Variable Interactions with Random Processes
We apply a wild bootstrap method to the Lancaster three-variable interaction measure in order to detect factorisation of the joint distribution on three variables forming a stationary random process, for which the existing permutation bootstrap method fails. As in the i.i.d. case, the Lancaster test is found to outperform existing tests in cases for which two independent variables individually have a weak influence on a third, but that when considered jointly the influence is strong. The main contributions of this paper are twofold: first, we prove that the Lancaster statistic satisfies the conditions required to estimate the quantiles of the null distribution using the wild bootstrap; second, the manner in which this is proved is novel, simpler than existing methods, and can further be applied to other statistics
Distinguishing distributions with interpretable features
Two semimetrics on probability distributions are
proposed, based on a difference between features
chosen from each, where these features can be in
either the spatial or Fourier domains. The features are chosen so as to maximize the distinguishability of the distributions, by optimizing
a lower bound of power for a statistical test using these features. The result is a parsimonious
and interpretable indication of how and where
two distributions differ, which can be used even
in high dimensions, and when the difference is
localized in the Fourier domain. A real-world
benchmark image data demonstrates that the returned features provide a meaningful and informative indication as to how the distributions diffe
Recommended from our members
Novel approaches in determining baseline information on annual disposal rates and trace element content of U.S. coal combustion residues : a response to EPA’s June 2010 proposed disposal rule
textAlthough products of coal combustion (PCCs) such as coal ash are currently exempted from classification as a hazardous waste in the United States under the 1976 Resource Conservation and Recovery Act (RCRA), the U.S. Environmental Protection Agency (EPA) is now revising a proposed rule to modify disposal practices for these materials in order to prevent contamination of ground- and surface water sources by leached trace elements.
This paper analyzes several aspects of EPA’s scientific reasoning for instating the rule, with the intent of answering the following questions: 1) Are EPA’s cited values for PCC production and disposal accurate estimates of annual totals?; 2) In what ways can EPA’s leaching risk modeling assessment be improved?; 3) What is the total quantity of trace elements contained within all PCCs disposed annually?; and 4) What would be the potential costs and feasibility of reclassifying PCCs not under RCRA, but under existing NRC regulations as low-level radioactive waste (LLRW)?
Among the results of my calculations, I found that although EPA estimates for annual PCC disposal are 20% larger than industry statistics, these latter values appear to be closer to reality. Second, EPA appears to have significantly underestimated historical PCC disposal: my projections indicate that EPA’s maximum estimate for the quantity of fly ash landfilled within the past 90 years was likely met by production in the last 30 years alone, if not less. Finally, my analysis indicates that while PCCs may potentially meet the criteria for reclassification as low-level radioactive waste by NRC, the cost of such regulation would be many times that of the EPA June proposed disposal rule (1.47 billion per year for the Subtitle C option and $236-587 million for Subtitle D regulatory options).Energy and Earth Resource