KD-ART: Should we intensify or diversify tests to kill mutants?

CONTEXT: Adaptive Random Testing (ART) spreads test cases evenly over the input domain. Yet once a fault is found, decisions must be made to diversify or intensify subsequent inputs. Diversification employs a wide range of tests to increase the chances of finding new faults. Intensification selects test inputs similar to those previously shown to be successful. OBJECTIVE: Explore the trade-off between diversification and intensification to kill mutants. METHOD: We augment Adaptive Random Testing (ART) to estimate the Kernel Density (KD–ART) of input values found to kill mutants. KD–ART was first proposed at the 10th International Workshop on Mutation Analysis. We now extend this work to handle real world non numeric applications. Specifically we incorporate a technique to support programs with input parameters that have composite data types (such as arrays and structs). RESULTS: Intensification is the most effective strategy for the numerical programs (it achieves 8.5% higher mutation score than ART). By contrast, diversification seems more effective for programs with composite inputs. KD–ART kills mutants 15.4 times faster than ART. CONCLUSION: Intensify tests for numerical types, but diversify them for composite types

Adaptive procedures in convolution models with known or partially known noise distribution

In a convolution model, we observe random variables whose distribution is the convolution of some unknown density f and some known or partially known noise density g. In this paper, we focus on statistical procedures, which are adaptive with respect to the smoothness parameter tau of unknown density f, and also (in some cases) to some unknown parameter of the noise density g. In a first part, we assume that g is known and polynomially smooth. We provide goodness-of-fit procedures for the test H_0:f=f_0, where the alternative H_1 is expressed with respect to L_2-norm. Our adaptive (w.r.t tau) procedure behaves differently according to whether f_0 is polynomially or exponentially smooth. A payment for adaptation is noted in both cases and for computing this, we provide a non-uniform Berry-Esseen type theorem for degenerate U-statistics. In the first case we prove that the payment for adaptation is optimal (thus unavoidable). In a second part, we study a wider framework: a semiparametric model, where g is exponentially smooth and stable, and its self-similarity index s is unknown. In order to ensure identifiability, we restrict our attention to polynomially smooth, Sobolev-type densities f. In this context, we provide a consistent estimation procedure for s. This estimator is then plugged-into three different procedures: estimation of the unknown density f, of the functional \int f^2 and test of the hypothesis H_0. These procedures are adaptive with respect to both s and tau and attain the rates which are known optimal for known values of s and tau. As a by-product, when the noise is known and exponentially smooth our testing procedure is adaptive for testing Sobolev-type densities.Comment: 35 pages + annexe de 8 page

Adaptive goodness-of-fit tests based on signed ranks

Within the nonparametric regression model with unknown regression function $l$ and independent, symmetric errors, a new multiscale signed rank statistic is introduced and a conditional multiple test of the simple hypothesis $l=0$ against a nonparametric alternative is proposed. This test is distribution-free and exact for finite samples even in the heteroscedastic case. It adapts in a certain sense to the unknown smoothness of the regression function under the alternative, and it is uniformly consistent against alternatives whose sup-norm tends to zero at the fastest possible rate. The test is shown to be asymptotically optimal in two senses: It is rate-optimal adaptive against H\"{o}lder classes. Furthermore, its relative asymptotic efficiency with respect to an asymptotically minimax optimal test under sup-norm loss is close to 1 in case of homoscedastic Gaussian errors within a broad range of H\"{o}lder classes simultaneously.Comment: Published in at http://dx.doi.org/10.1214/009053607000000992 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Optimal Calibration for Multiple Testing against Local Inhomogeneity in Higher Dimension

Based on two independent samples X_1,...,X_m and X_{m+1},...,X_n drawn from multivariate distributions with unknown Lebesgue densities p and q respectively, we propose an exact multiple test in order to identify simultaneously regions of significant deviations between p and q. The construction is built from randomized nearest-neighbor statistics. It does not require any preliminary information about the multivariate densities such as compact support, strict positivity or smoothness and shape properties. The properly adjusted multiple testing procedure is shown to be sharp-optimal for typical arrangements of the observation values which appear with probability close to one. The proof relies on a new coupling Bernstein type exponential inequality, reflecting the non-subgaussian tail behavior of a combinatorial process. For power investigation of the proposed method a reparametrized minimax set-up is introduced, reducing the composite hypothesis "p=q" to a simple one with the multivariate mixed density (m/n)p+(1-m/n)q as infinite dimensional nuisance parameter. Within this framework, the test is shown to be spatially and sharply asymptotically adaptive with respect to uniform loss on isotropic H\"older classes. The exact minimax risk asymptotics are obtained in terms of solutions of the optimal recovery

A maximum-mean-discrepancy goodness-of-fit test for censored data

We introduce a kernel-based goodness-of-fit test for censored data, where observations may be missing in random time intervals: a common occurrence in clinical trials and industrial life-testing. The test statistic is straightforward to compute, as is the test threshold, and we establish consistency under the null. Unlike earlier approaches such as the Log-rank test, we make no assumptions as to how the data distribution might differ from the null, and our test has power against a very rich class of alternatives. In experiments, our test outperforms competing approaches for periodic and Weibull hazard functions (where risks are time dependent), and does not show the failure modes of tests that rely on user-defined features. Moreover, in cases where classical tests are provably most powerful, our test performs almost as well, while being more general

Combining information from independent sources through confidence distributions

This paper develops new methodology, together with related theories, for combining information from independent studies through confidence distributions. A formal definition of a confidence distribution and its asymptotic counterpart (i.e., asymptotic confidence distribution) are given and illustrated in the context of combining information. Two general combination methods are developed: the first along the lines of combining p-values, with some notable differences in regard to optimality of Bahadur type efficiency; the second by multiplying and normalizing confidence densities. The latter approach is inspired by the common approach of multiplying likelihood functions for combining parametric information. The paper also develops adaptive combining methods, with supporting asymptotic theory which should be of practical interest. The key point of the adaptive development is that the methods attempt to combine only the correct information, downweighting or excluding studies containing little or wrong information about the true parameter of interest. The combination methodologies are illustrated in simulated and real data examples with a variety of applications.Comment: Published at http://dx.doi.org/10.1214/009053604000001084 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org