An introduction to how chi-square and classical exact tests often wildly misreport significance and how the remedy lies in computers

Abstract

Goodness-of-fit tests based on the Euclidean distance often outperform chi-square and other classical tests (including the standard exact tests) by at least an order of magnitude when the model being tested for goodness-of-fit is a discrete probability distribution that is not close to uniform. The present article discusses numerous examples of this. Goodness-of-fit tests based on the Euclidean metric are now practical and convenient: although the actual values taken by the Euclidean distance and similar goodness-of-fit statistics are seldom humanly interpretable, black-box computer programs can rapidly calculate their precise significance.Comment: 41 pages, 25 figures, 7 tables. arXiv admin note: near complete text overlap with arXiv:1108.412