Statistical significance and its critics: practicing damaging science, or damaging scientific practice?

While the common procedure of statistical significance testing and its accompanying concept of p-values have long been surrounded by controversy, renewed concern has been triggered by the replication crisis in science. Many blame statistical significance tests themselves, and some regard them as sufficiently damaging to scientific practice as to warrant being abandoned. We take a contrary position, arguing that the central criticisms arise from misunderstanding and misusing the statistical tools, and that in fact the purported remedies themselves risk damaging science. We argue that banning the use of p-value thresholds in interpreting data does not diminish but rather exacerbates data-dredging and biasing selection effects. If an account cannot specify outcomes that will not be allowed to count as evidence for a claim—if all thresholds are abandoned—then there is no test of that claim. The contributions of this paper are: To explain the rival statistical philosophies underlying the ongoing controversy; To elucidate and reinterpret statistical significance tests, and explain how this reinterpretation ameliorates common misuses and misinterpretations; To argue why recent recommendations to replace, abandon, or retire statistical significance undermine a central function of statistics in science: to test whether observed patterns in the data are genuine or due to background variabilit

Evidence based medicine as science

Evidence based medicine has claimed to be science on a number of occasions but it is not clear that this status is deserved. Within philosophy of science four main theories about the nature of science are historically recognised: inductivism, falsificationism, Kuhnian paradigms and research programmes. If evidence based medicine is science knowledge claims should be derived using a process that corresponds to one of these theories. This paper analyses whether this is the case. In the first section, different theories about the nature of science are introduced. In the second section, the claim that evidence based medicine is science is reinterpreted as the claim that knowledge claims derived from randomised controlled trails and meta-analyses are science. In the third section the knowledge claims valued within evidence based medicine are considered from the perspective of inductivism, falsificationism, Kuhnian paradigms and research programmes. In the final section possible counter arguments are considered. It is argued that the knowledge claims valued by evidence based medicine are not justified using inductivism, falsificationism, Kuhnian paradigms or research programmes. If these are the main criteria for evaluating if something is science or not, evidence based medicine does not meet these criteria

Statistical inference and the replication crisis

The replication crisis has prompted many to call for statistical reform within the psychological sciences. Here we examine issues within Frequentist statistics that may have led to the replication crisis, and we examine the alternative—Bayesian statistics—that many have suggested as a replacement. The Frequentist approach and the Bayesian approach offer radically different perspectives on evidence and inference with the Frequentist approach prioritising error control and the Bayesian approach offering a formal method for quantifying the relative strength of evidence for hypotheses. We suggest that rather than mere statistical reform, what is needed is a better understanding of the different modes of statistical inference and a better understanding of how statistical inference relates to scientific inference

The fallacy of placing confidence in confidence intervals

Interval estimates – estimates of parameters that include an allowance for sampling uncertainty – have long been touted as a key component of statistical analyses. There are several kinds of interval estimates, but the most popular are confidence intervals (CIs): intervals that contain the true parameter value in some known proportion of repeated samples, on average. The width of confidence intervals is thought to index the precision of an estimate; CIs are thought to be a guide to which parameter values are plausible or reasonable; and the confidence coefficient of the interval (e.g., 95 %) is thought to index the plausibility that the true parameter is included in the interval. We show in a number of examples that CIs do not necessarily have any of these properties, and can lead to unjustified or arbitrary inferences. For this reason, we caution against relying upon confidence interval theory to justify interval estimates, and suggest that other theories of interval estimation should be used instead

A multidisciplinary, multifactorial intervention program reduces postoperative falls and injuries after femoral neck fracture

INTRODUCTION: This study evaluates whether a postoperative multidisciplinary, intervention program, including systematic assessment and treatment of fall risk factors, active prevention, detection, and treatment of postoperative complications, could reduce inpatient falls and fall-related injuries after a femoral neck fracture. METHODS: A randomized, controlled trial at the orthopedic and geriatric departments at Umeå University Hospital, Sweden, included 199 patients with femoral neck fracture, aged  ≥70 years. RESULTS: Twelve patients fell 18 times in the intervention group compared with 26 patients suffering 60 falls in the control group. Only one patient with dementia fell in the intervention group compared with 11 in the control group. The crude postoperative fall incidence rate was 6.29/1,000 days in the intervention group vs 16.28/1,000 days in the control group. The incidence rate ratio was 0.38 [95% confidence interval (CI): 0.20 – 0.76, p = 0.006] for the total sample and 0.07 (95% CI: 0.01–0.57, p=0.013) among patients with dementia. There were no new fractures in the intervention group but four in the control group. CONCLUSION: A team applying comprehensive geriatric assessment and rehabilitation, including prevention, detection, and treatment of fall risk factors, can successfully prevent inpatient falls and injuries, even in patients with dementia

Shifting Attention From Theory to Practice in Philosophy of Biology

Traditional approaches in philosophy of biology focus attention on biological concepts, explanations, and theories, on evidential support and inter-theoretical relations. Newer approaches shift attention from concepts to conceptual practices, from theories to practices of theorizing, and from theoretical reduction to reductive retooling. In this article, I describe the shift from theory-focused to practice-centered philosophy of science and explain how it is leading philosophers to abandon fundamentalist assumptions associated with traditional approaches in philosophy of science and to embrace scientific pluralism. This article comes in three parts, each illustrating the shift from theory-focused to practice-centered epistemology. The first illustration shows how shifting philosophical attention to conceptual practice reveals how molecular biologists succeed in identifying coherent causal strands within systems of bewildering complexity. The second illustration suggests that analyzing how a multiplicity of alternative models function in practice provides an illuminating approach for understanding the nature of theoretical knowledge in evolutionary biology. The third illustration demonstrates how framing reductionism in terms of the reductive retooling of practice offers an informative perspective for understanding why putting DNA at the center of biological research has been incredibly productive throughout much of biology. Each illustration begins by describing how traditional theory-focused philosophical approaches are laden with fundamentalist assumptions and then proceeds to show that shifting attention to practice undermines these assumptions and motivates a philosophy of scientific pluralism