Statistical significance in high-dimensional linear models

We propose a method for constructing p-values for general hypotheses in a high-dimensional linear model. The hypotheses can be local for testing a single regression parameter or they may be more global involving several up to all parameters. Furthermore, when considering many hypotheses, we show how to adjust for multiple testing taking dependence among the p-values into account. Our technique is based on Ridge estimation with an additional correction term due to a substantial projection bias in high dimensions. We prove strong error control for our p-values and provide sufficient conditions for detection: for the former, we do not make any assumption on the size of the true underlying regression coefficients while regarding the latter, our procedure might not be optimal in terms of power. We demonstrate the method in simulated examples and a real data application.Comment: Published in at http://dx.doi.org/10.3150/12-BEJSP11 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

On Bonferroni-Type Inequalities of the same Degree for the Probability of a Union

1 online resource (PDF, 16 pages

Partial spreads and vector space partitions

Constant-dimension codes with the maximum possible minimum distance have been studied under the name of partial spreads in Finite Geometry for several decades. Not surprisingly, for this subclass typically the sharpest bounds on the maximal code size are known. The seminal works of Beutelspacher and Drake \& Freeman on partial spreads date back to 1975, and 1979, respectively. From then until recently, there was almost no progress besides some computer-based constructions and classifications. It turns out that vector space partitions provide the appropriate theoretical framework and can be used to improve the long-standing bounds in quite a few cases. Here, we provide a historic account on partial spreads and an interpretation of the classical results from a modern perspective. To this end, we introduce all required methods from the theory of vector space partitions and Finite Geometry in a tutorial style. We guide the reader to the current frontiers of research in that field, including a detailed description of the recent improvements.Comment: 30 pages, 1 tabl

Resource dependent branching processes and the envelope of societies

Since its early beginnings, mankind has put to test many different society forms, and this fact raises a complex of interesting questions. The objective of this paper is to present a general population model which takes essential features of any society into account and which gives interesting answers on the basis of only two natural hypotheses. One is that societies want to survive, the second, that individuals in a society would, in general, like to increase their standard of living. We start by presenting a mathematical model, which may be seen as a particular type of a controlled branching process. All conditions of the model are justified and interpreted. After several preliminary results about societies in general we can show that two society forms should attract particular attention, both from a qualitative and a quantitative point of view. These are the so-called weakest-first society and the strongest-first society. In particular we prove then that these two societies stand out since they form an envelope of all possible societies in a sense we will make precise. This result (the envelopment theorem) is seen as significant because it is paralleled with precise survival criteria for the enveloping societies. Moreover, given that one of the "limiting" societies can be seen as an extreme form of communism, and the other one as being close to an extreme version of capitalism, we conclude that, remarkably, humanity is close to having already tested the limits.Comment: Published in at http://dx.doi.org/10.1214/13-AAP998 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Probabilistic Tools for the Analysis of Randomized Optimization Heuristics

This chapter collects several probabilistic tools that proved to be useful in the analysis of randomized search heuristics. This includes classic material like Markov, Chebyshev and Chernoff inequalities, but also lesser known topics like stochastic domination and coupling or Chernoff bounds for geometrically distributed random variables and for negatively correlated random variables. Most of the results presented here have appeared previously, some, however, only in recent conference publications. While the focus is on collecting tools for the analysis of randomized search heuristics, many of these may be useful as well in the analysis of classic randomized algorithms or discrete random structures.Comment: 91 page