Abandon Statistical Significance

We discuss problems the null hypothesis significance testing (NHST) paradigm poses for replication and more broadly in the biomedical and social sciences as well as how these problems remain unresolved by proposals involving modified p-value thresholds, confidence intervals, and Bayes factors. We then discuss our own proposal, which is to abandon statistical significance. We recommend dropping the NHST paradigm--and the p-value thresholds intrinsic to it--as the default statistical paradigm for research, publication, and discovery in the biomedical and social sciences. Specifically, we propose that the p-value be demoted from its threshold screening role and instead, treated continuously, be considered along with currently subordinate factors (e.g., related prior evidence, plausibility of mechanism, study design and data quality, real world costs and benefits, novelty of finding, and other factors that vary by research domain) as just one among many pieces of evidence. We have no desire to "ban" p-values or other purely statistical measures. Rather, we believe that such measures should not be thresholded and that, thresholded or not, they should not take priority over the currently subordinate factors. We also argue that it seldom makes sense to calibrate evidence as a function of p-values or other purely statistical measures. We offer recommendations for how our proposal can be implemented in the scientific publication process as well as in statistical decision making more broadly

Daubert\u27s Significance

The authors review and note the limited reach of Daubert v. Merrell Dow Pharmaceuticals. They also address its implications for concerned non-lawyers

Self-play: statistical significance

Heinz recently completed a comprehensive experiment in self-play using the FRITZ chess engine to establish the ‘decreasing returns’ hypothesis with specific levels of statistical confidence. This note revisits the results and recalculates the confidence levels of this and other hypotheses. These appear to be better than Heinz’ initial analysis suggests

The significance of SNODENT

SNODENT is a dental diagnostic vocabulary incompletely integrated in SNOMED-CT. Nevertheless, SNODENT could become the de facto standard for dental diagnostic coding. SNODENT's manageable size, the fact that it is administratively self-contained, and relates to a well-understood domain provides valuable opportunities to formulate and test, in controlled experiments, a series of hypothesis concerning diagnostic systems. Of particular interest are questions related to establishing appropriate quality assurance methods for its optimal level of detail in content, its ontological structure, its construction and maintenance. This paper builds on previous–software-based methodologies designed to assess the quality of SNOMED-CT. When applied to SNODENT several deficiencies were uncovered. 9.52% of SNODENT terms point to concepts in SNOMED-CT that have some problem. 18.53% of SNODENT terms point to SNOMED-CT concepts do not have, in SNOMED, the term used by SNODENT. Other findings include the absence of a clear specification of the exact relationship between a term and a termcode in SNODENT and the improper assignment of the same termcode to terms with significantly different meanings. An analysis of the way in which SNODENT is structurally integrated into SNOMED resulted in the generation of 1081 new termcodes reflecting entities not present in the SNOMED tables but required by SNOMED's own description logic based classification principles. Our results show that SNODENT requires considerable enhancements in content, quality of coding, quality of ontological structure and the manner in which it is integrated and aligned with SNOMED. We believe that methods for the analysis of the quality of diagnostic coding systems must be developed and employed if such systems are to be used effectively in both clinical practice and clinical research