Hypothesis testing for two population means: parametric or non-parametric test?

The parametric Welch $t$-test and the non-parametric Wilcoxon-Mann-Whitney test are the most commonly used two independent sample means tests. More recent testing approaches include the non-parametric, empirical likelihood and exponential empirical likelihood. However, the applicability of these non-parametric likelihood testing procedures is limited partially because of their tendency to inflate the type I error in small sized samples. In order to circumvent the type I error problem, we propose simple calibrations using the $t$ distribution and bootstrapping. The two non-parametric likelihood testing procedures, with and without those calibrations, are then compared against the Wilcoxon-Mann-Whitney test and the Welch $t$-test. The comparisons are implemented via extensive Monte Carlo simulations on the grounds of type I error and power in small/medium sized samples generated from various non-normal populations. The simulation studies clearly demonstrate that a) the $t$ calibration improves the type I error of the empirical likelihood, b) bootstrap calibration improves the type I error of both non-parametric likelihoods, c) the Welch $t$-test with or without bootstrap calibration attains the type I error and produces similar levels of power with the former testing procedures, and d) the Wilcoxon-Mann-Whitney test produces inflated type I error while the computation of an exact p-value is not feasible in the presence of ties with discrete data. Further, an application to real gene expression data illustrates the computational high cost and thus the impracticality of the non parametric likelihoods. Overall, the Welch t-test, which is highly computationally efficient and readily interpretable, is shown to be the best method when testing equality of two population means.Comment: Accepted for publication in the Journal of Statistical Computation and Simulatio

A nonmanipulable test

A test is said to control for type I error if it is unlikely to reject the data-generating process. However, if it is possible to produce stochastic processes at random such that, for all possible future realizations of the data, the selected process is unlikely to be rejected, then the test is said to be manipulable. So, a manipulable test has essentially no capacity to reject a strategic expert. Many tests proposed in the existing literature, including calibration tests, control for type I error but are manipulable. We construct a test that controls for type I error and is nonmanipulable.Comment: Published in at http://dx.doi.org/10.1214/08-AOS597 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

A monte carlo analysis of the type II tobit maximum likelihood estimator when the true model is the type I tobit model

Type I (censored regression) and Type II Tobit (sample selection) models are widely used in the various fields of economics. The Type I Tobit model is a special case of the Type II Tobit model. However, the dimension of the error terms decreases and the distribution of the error terms degenerates in the Type I Tobit Model. Therefore, we cannot use the standard asymptotic theorems for the Type II Tobit Maximum Likelihood Estimator (MLE) when the sample is obtained from the Type I Tobit model. Results of Monte Carlo experiments show strange behavior that has never been reported before for the Type II MLE.

Judicial Errors and Crime Deterrence: Theory and Experimental Evidence

The standard economic theory of crime deterrence predicts that the conviction of an innocent (type-I error) is as detrimental to deterrence as the acquittal of a guilty individual (type-II error). In this paper, we qualify this result theoretically, showing that in the presence of risk aversion, loss-aversion, or differential sensitivity to procedural fairness, type-I errors can have a larger effect on deterrence than type-II errors. We test these predictions with an experiment where participants make a decision on whether to steal from other individuals, being subject to different probabilities of judicial errors. The results indicate that both types of judicial errors have a large and significant impact on deterrence, but these effects are not symmetric. An increase in the probability of type-I errors has a larger negative impact on deterrence than an equivalent increase in the probability of type-II errors. This asymmetry is largely explained by risk aversion and, to a lesser extent, type-I error aversion.Judicial errors, criminal procedure, procedural fairness, experimental economics, law and economics, crime, deterrence