7,090 research outputs found
Coherent frequentism
By representing the range of fair betting odds according to a pair of
confidence set estimators, dual probability measures on parameter space called
frequentist posteriors secure the coherence of subjective inference without any
prior distribution. The closure of the set of expected losses corresponding to
the dual frequentist posteriors constrains decisions without arbitrarily
forcing optimization under all circumstances. This decision theory reduces to
those that maximize expected utility when the pair of frequentist posteriors is
induced by an exact or approximate confidence set estimator or when an
automatic reduction rule is applied to the pair. In such cases, the resulting
frequentist posterior is coherent in the sense that, as a probability
distribution of the parameter of interest, it satisfies the axioms of the
decision-theoretic and logic-theoretic systems typically cited in support of
the Bayesian posterior. Unlike the p-value, the confidence level of an interval
hypothesis derived from such a measure is suitable as an estimator of the
indicator of hypothesis truth since it converges in sample-space probability to
1 if the hypothesis is true or to 0 otherwise under general conditions.Comment: The confidence-measure theory of inference and decision is explicitly
extended to vector parameters of interest. The derivation of upper and lower
confidence levels from valid and nonconservative set estimators is formalize
Continuous Monitoring of A/B Tests without Pain: Optional Stopping in Bayesian Testing
A/B testing is one of the most successful applications of statistical theory
in modern Internet age. One problem of Null Hypothesis Statistical Testing
(NHST), the backbone of A/B testing methodology, is that experimenters are not
allowed to continuously monitor the result and make decision in real time. Many
people see this restriction as a setback against the trend in the technology
toward real time data analytics. Recently, Bayesian Hypothesis Testing, which
intuitively is more suitable for real time decision making, attracted growing
interest as an alternative to NHST. While corrections of NHST for the
continuous monitoring setting are well established in the existing literature
and known in A/B testing community, the debate over the issue of whether
continuous monitoring is a proper practice in Bayesian testing exists among
both academic researchers and general practitioners. In this paper, we formally
prove the validity of Bayesian testing with continuous monitoring when proper
stopping rules are used, and illustrate the theoretical results with concrete
simulation illustrations. We point out common bad practices where stopping
rules are not proper and also compare our methodology to NHST corrections.
General guidelines for researchers and practitioners are also provided
The Jeffreys-Lindley Paradox and Discovery Criteria in High Energy Physics
The Jeffreys-Lindley paradox displays how the use of a p-value (or number of
standard deviations z) in a frequentist hypothesis test can lead to an
inference that is radically different from that of a Bayesian hypothesis test
in the form advocated by Harold Jeffreys in the 1930s and common today. The
setting is the test of a well-specified null hypothesis (such as the Standard
Model of elementary particle physics, possibly with "nuisance parameters")
versus a composite alternative (such as the Standard Model plus a new force of
nature of unknown strength). The p-value, as well as the ratio of the
likelihood under the null hypothesis to the maximized likelihood under the
alternative, can strongly disfavor the null hypothesis, while the Bayesian
posterior probability for the null hypothesis can be arbitrarily large. The
academic statistics literature contains many impassioned comments on this
paradox, yet there is no consensus either on its relevance to scientific
communication or on its correct resolution. The paradox is quite relevant to
frontier research in high energy physics. This paper is an attempt to explain
the situation to both physicists and statisticians, in the hope that further
progress can be made.Comment: v4: Continued editing for clarity. Figure added. v5: Minor fixes to
biblio. Same as published version except for minor copy-edits, Synthese
(2014). v6: fix typos, and restore garbled sentence at beginning of Sec 4 to
v
Approximate Bayesian Model Selection with the Deviance Statistic
Bayesian model selection poses two main challenges: the specification of
parameter priors for all models, and the computation of the resulting Bayes
factors between models. There is now a large literature on automatic and
objective parameter priors in the linear model. One important class are
-priors, which were recently extended from linear to generalized linear
models (GLMs). We show that the resulting Bayes factors can be approximated by
test-based Bayes factors (Johnson [Scand. J. Stat. 35 (2008) 354-368]) using
the deviance statistics of the models. To estimate the hyperparameter , we
propose empirical and fully Bayes approaches and link the former to minimum
Bayes factors and shrinkage estimates from the literature. Furthermore, we
describe how to approximate the corresponding posterior distribution of the
regression coefficients based on the standard GLM output. We illustrate the
approach with the development of a clinical prediction model for 30-day
survival in the GUSTO-I trial using logistic regression.Comment: Published at http://dx.doi.org/10.1214/14-STS510 in the Statistical
Science (http://www.imstat.org/sts/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Exact asymptotic distribution of change-point mle for change in the mean of Gaussian sequences
We derive exact computable expressions for the asymptotic distribution of the
change-point mle when a change in the mean occurred at an unknown point of a
sequence of time-ordered independent Gaussian random variables. The derivation,
which assumes that nuisance parameters such as the amount of change and
variance are known, is based on ladder heights of Gaussian random walks hitting
the half-line. We then show that the exact distribution easily extends to the
distribution of the change-point mle when a change occurs in the mean vector of
a multivariate Gaussian process. We perform simulations to examine the accuracy
of the derived distribution when nuisance parameters have to be estimated as
well as robustness of the derived distribution to deviations from Gaussianity.
Through simulations, we also compare it with the well-known conditional
distribution of the mle, which may be interpreted as a Bayesian solution to the
change-point problem. Finally, we apply the derived methodology to monthly
averages of water discharges of the Nacetinsky creek, Germany.Comment: Published in at http://dx.doi.org/10.1214/09-AOAS294 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …