Optimal testing of equivalence hypotheses

In this paper we consider the construction of optimal tests of equivalence hypotheses. Specifically, assume X_1,..., X_n are i.i.d. with distribution P_{\theta}, with \theta \in R^k. Let g(\theta) be some real-valued parameter of interest. The null hypothesis asserts g(\theta)\notin (a,b) versus the alternative g(\theta)\in (a,b). For example, such hypotheses occur in bioequivalence studies where one may wish to show two drugs, a brand name and a proposed generic version, have the same therapeutic effect. Little optimal theory is available for such testing problems, and it is the purpose of this paper to provide an asymptotic optimality theory. Thus, we provide asymptotic upper bounds for what is achievable, as well as asymptotically uniformly most powerful test constructions that attain the bounds. The asymptotic theory is based on Le Cam's notion of asymptotically normal experiments. In order to approximate a general problem by a limiting normal problem, a UMP equivalence test is obtained for testing the mean of a multivariate normal mean.Comment: Published at http://dx.doi.org/10.1214/009053605000000048 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Expectations Hypotheses Tests

We investigate the Expectations Hypotheses of the term structure of interest rates and of the foreign exchange market using vector autoregressive methods for the U.S. dollar, Deutsche mark, and British pound interest rates and exchange rates. In addition to standard Wald tests, we formulate Lagrange Multiplier and Distance Metric tests which require estimation under the non-linear constraints of the null hypotheses. Estimation under the null is achieved by iterating on approximate solutions that require only matrix inversions. We use a bias-corrected, constrained vector autoregression as a data generating process and construct extensive Monte Carlo simulations of the various test statistics under the null hypotheses. Wald tests suffer from severe size distortions and use of the asymptotic critical values results in gross over-rejection of the null. The Lagrange Multiplier tests slightly under-reject the null, and the Distance Metric tests over-reject. Use of the small sample distributions of the different tests leads to a common interpretation of the validity of the Expectations Hypotheses. The evidence against the Expectations Hypotheses for these interest rates and exchange rates is much less strong than under asymptotic inference.

On null hypotheses in survival analysis

The conventional nonparametric tests in survival analysis, such as the log-rank test, assess the null hypothesis that the hazards are equal at all times. However, hazards are hard to interpret causally, and other null hypotheses are more relevant in many scenarios with survival outcomes. To allow for a wider range of null hypotheses, we present a generic approach to define test statistics. This approach utilizes the fact that a wide range of common parameters in survival analysis can be expressed as solutions of differential equations. Thereby we can test hypotheses based on survival parameters that solve differential equations driven by cumulative hazards, and it is easy to implement the tests on a computer. We present simulations, suggesting that our tests perform well for several hypotheses in a range of scenarios. Finally, we use our tests to evaluate the effect of adjuvant chemotherapies in patients with colon cancer, using data from a randomised controlled trial

Assessing evidence and testing appropriate hypotheses

It is crucial to identify the most appropriate hypotheses if one is to apply probabilistic reasoning to evaluate and properly understand the impact of evidence. Subtle changes to the choice of a prosecution hypothesis can result in drastically different posterior probabilities to a defence hypothesis from the same evidence. To illustrate the problem we consider a real case in which probabilistic arguments assumed that the prosecution hypothesis “both babies were murdered” was the appropriate alternative to the defence hypothesis “both babies died of Sudden Infant Death Syndrome (SIDS)”. Since it would have been sufficient for the prosecution to establish just one murder, a more appropriate alternative hypothesis was “at least one baby was murdered”. Based on the same assumptions used by one of the probability experts who examined the case, the prior odds in favour of the defence hypothesis over the double murder hypothesis are 30 to 1. However, the prior odds in favour of the defence hypothesis over the alternative ‘at least one murder’ hypothesis are only 5 to 2. Assuming that the medical and other evidence has a likelihood ratio of 5 in favour of the prosecution hypothesis results in very different conclusions about the posterior probability of the defence hypothesis

Ensemble evaluation of hydrological model hypotheses

It is demonstrated for the first time how model parameter, structural and data uncertainties can be accounted for explicitly and simultaneously within the Generalized Likelihood Uncertainty Estimation (GLUE) methodology. As an example application, 72 variants of a single soil moisture accounting store are tested as simplified hypotheses of runoff generation at six experimental grassland field-scale lysimeters through model rejection and a novel diagnostic scheme. The fields, designed as replicates, exhibit different hydrological behaviors which yield different model performances. For fields with low initial discharge levels at the beginning of events, the conceptual stores considered reach their limit of applicability. Conversely, one of the fields yielding more discharge than the others, but having larger data gaps, allows for greater flexibility in the choice of model structures. As a model learning exercise, the study points to a “leaking” of the fields not evident from previous field experiments. It is discussed how understanding observational uncertainties and incorporating these into model diagnostics can help appreciate the scale of model structural error

Distributed Learning with Infinitely Many Hypotheses

We consider a distributed learning setup where a network of agents sequentially access realizations of a set of random variables with unknown distributions. The network objective is to find a parametrized distribution that best describes their joint observations in the sense of the Kullback-Leibler divergence. Apart from recent efforts in the literature, we analyze the case of countably many hypotheses and the case of a continuum of hypotheses. We provide non-asymptotic bounds for the concentration rate of the agents' beliefs around the correct hypothesis in terms of the number of agents, the network parameters, and the learning abilities of the agents. Additionally, we provide a novel motivation for a general set of distributed Non-Bayesian update rules as instances of the distributed stochastic mirror descent algorithm.Comment: Submitted to CDC201