Testing for equivalence: an intersection-union permutation solution

The notion of testing for equivalence of two treatments is widely used in clinical trials, pharmaceutical experiments,bioequivalence and quality control. It is essentially approached within the intersection-union (IU) principle. According to this principle the null hypothesis is stated as the set of effects lying outside a suitably established interval and the alternative as the set of effects lying inside that interval. The solutions provided in the literature are mostly based on likelihood techniques, which in turn are rather difficult to handle, except for cases lying within the regular exponential family and the invariance principle. The main goal of present paper is to go beyond most of the limitations of likelihood based methods, i.e. to work in a nonparametric setting within the permutation frame. To obtain practical solutions, a new IU permutation test is presented and discussed. A simple simulation study for evaluating its main properties, and three application examples are also presented.Comment: 21 pages, 2 figure

Inferential versus descriptive statistical approach in the analysis of Delphi performance: A case study

This study aims at evaluating, by permutation methods, the performances of Delphi approach in the research to predict the future of the family in NorthEast of Italy in ten years. The usual descriptives indicators are: stability, consensus and convergence speed. In the work we intend to test \u2013 by permutation methods -three equivalent distinct statistical hypotheses: equality, convergence and combination

A comparison of efficient permutation tests for unbalanced ANOVA in two by two designs--and their behavior under heteroscedasticity

We compare different permutation tests and some parametric counterparts that are applicable to unbalanced designs in two by two designs. First the different approaches are shortly summarized. Then we investigate the behavior of the tests in a simulation study. A special focus is on the behavior of the tests under heteroscedastic variances.Comment: 20 pages, 9 figures, Working Paper of the Department of Management And Enigineering of the University of Padov

Double-check: validation of diagnostic statistics for PLS-DA models in metabolomics studies

Partial Least Squares-Discriminant Analysis (PLS-DA) is a PLS regression method with a special binary ‘dummy’ y-variable and it is commonly used for classification purposes and biomarker selection in metabolomics studies. Several statistical approaches are currently in use to validate outcomes of PLS-DA analyses e.g. double cross validation procedures or permutation testing. However, there is a great inconsistency in the optimization and the assessment of performance of PLS-DA models due to many different diagnostic statistics currently employed in metabolomics data analyses. In this paper, properties of four diagnostic statistics of PLS-DA, namely the number of misclassifications (NMC), the Area Under the Receiver Operating Characteristic (AUROC), Q2 and Discriminant Q2 (DQ2) are discussed. All four diagnostic statistics are used in the optimization and the performance assessment of PLS-DA models of three different-size metabolomics data sets obtained with two different types of analytical platforms and with different levels of known differences between two groups: control and case groups. Statistical significance of obtained PLS-DA models was evaluated with permutation testing. PLS-DA models obtained with NMC and AUROC are more powerful in detecting very small differences between groups than models obtained with Q2 and Discriminant Q2 (DQ2). Reproducibility of obtained PLS-DA models outcomes, models complexity and permutation test distributions are also investigated to explain this phenomenon. DQ2 and Q2 (in contrary to NMC and AUROC) prefer PLS-DA models with lower complexity and require higher number of permutation tests and submodels to accurately estimate statistical significance of the model performance. NMC and AUROC seem more efficient and more reliable diagnostic statistics and should be recommended in two group discrimination metabolomic studies