Statistical Issues in Searches for New Physics

Given the cost, both financial and even more importantly in terms of human effort, in building High Energy Physics accelerators and detectors and running them, it is important to use good statistical techniques in analysing data. Some of the statistical issues that arise in searches for New Physics are discussed briefly. They include topics such as: Should we insist on the 5 sigma criterion for discovery claims? The probability of A, given B, is not the same as the probability of B, given A. The meaning of p-values. What is Wilks Theorem and when does it not apply? How should we deal with the `Look Elsewhere Effect'? Dealing with systematics such as background parametrisation. Coverage: What is it and does my method have the correct coverage? The use of p0 versus p1 plots.Comment: This is the write-up for the Proceedings of a talk delivered at the LHCP2014 Conference at Columbia University, New York in June 2014no diagrams 5 pages long

Combining parameter values or pp-values

We review the methods to combine several measurements, in the form of parameter values or $p$-values.Comment: 15 pages, 4 figure

Open statistical issues in particle physics

Many statistical issues arise in the analysis of Particle Physics experiments. We give a brief introduction to Particle Physics, before describing the techniques used by Particle Physicists for dealing with statistical problems, and also some of the open statistical questions.Comment: Published in at http://dx.doi.org/10.1214/08-AOAS163 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Louis M. Lyons to Dr. Silver, 14 April 1961

Professional correspondenc

Reproducibility and Replication of Experimental Particle Physics Results

Recently, much attention has been focused on the replicability of scientific results, causing scientists, statisticians, and journal editors to examine closely their methodologies and publishing criteria. Experimental particle physicists have been aware of the precursors of non-replicable research for many decades and have many safeguards to ensure that the published results are as reliable as possible. The experiments require large investments of time and effort to design, construct, and operate. Large collaborations produce and check the results, and many papers are signed by more than three thousand authors. This paper gives an introduction to what experimental particle physics is and to some of the tools that are used to analyze the data. It describes the procedures used to ensure that results can be computationally reproduced, both by collaborators and by non-collaborators. It describes the status of publicly available data sets and analysis tools that aid in reproduction and recasting of experimental results. It also describes methods particle physicists use to maximize the reliability of the results, which increases the probability that they can be replicated by other collaborations or even the same collaborations with more data and new personnel. Examples of results that were later found to be false are given, both with failed replication attempts and one with alarmingly successful replications. While some of the characteristics of particle physics experiments are unique, many of the procedures and techniques can be and are used in other fields.Comment: 50 pages, 6 figure