Confidence Intervals

PowerPoint slides for Confidence Intervals. Examples are taken from the Medical Literatur

Frequentist confidence intervals for orbits

The problem of efficiently computing the orbital elements of a visual binary while still deriving confidence intervals with frequentist properties is treated. When formulated in terms of the Thiele-Innes elements, the known distribution of probability in Thiele-Innes space allows efficient grid-search plus Monte-Carlo-sampling schemes to be constructed for both the minimum-$\!\chi^{2}$ and Bayesian approaches to parameter estimation. Numerical experiments with $10^{4}$ independent realizations of an observed orbit confirm that the $1-$ and $2\sigma$ confidence and credibility intervals have coverage fractions close to their frequentist values. \keywords{binaries: visual - stars: fundamental parameters - methods:statistical}Comment: 7 pages, 2 figures. Minor changes. Accepted by Astronomy and Astrophysic

The Physical Significance of Confidence Intervals

We define some appropriate statistical quantities that indicate the physical significance (reliability) of confidence intervals in the framework of both Frequentist and Bayesian statistical theories. We consider the expectation value of the upper limit in the absence of a signal (that we propose to call "exclusion potential", instead of "sensitivity" as done by Feldman and Cousins) and its standard deviation, we define the "Pull" of a null result, expressing the reliability of an experimental upper limit, and the "upper and lower detection functions", that give information on the possible outcome of an experiment if there is a signal. We also give a new appropriate definition of "sensitivity", that quantifies the capability of an experiment to reveal the signal that is searched for at the given confidence level.Comment: 16 page

P values, confidence intervals, or confidence levels for hypotheses?

Null hypothesis significance tests and p values are widely used despite very strong arguments against their use in many contexts. Confidence intervals are often recommended as an alternative, but these do not achieve the objective of assessing the credibility of a hypothesis, and the distinction between confidence and probability is an unnecessary confusion. This paper proposes a more straightforward (probabilistic) definition of confidence, and suggests how the idea can be applied to whatever hypotheses are of interest to researchers. The relative merits of the different approaches are discussed using a series of illustrative examples: usually confidence based approaches seem more transparent and useful, but there are some contexts in which p values may be appropriate. I also suggest some methods for converting results from one format to another. (The attractiveness of the idea of confidence is demonstrated by the widespread persistence of the completely incorrect idea that p=5% is equivalent to 95% confidence in the alternative hypothesis. In this paper I show how p values can be used to derive meaningful confidence statements, and the assumptions underlying the derivation.) Key words: Confidence interval, Confidence level, Hypothesis testing, Null hypothesis significance tests, P value, User friendliness.Comment: The essential argument is unchanged from previous versions, but the paper has been largely rewritten, the argument extended, and more examples and background context included. 21 pages, 3 diagrams, 3 table

Markov Chain Monte Carlo confidence intervals

For a reversible and ergodic Markov chain $\{X_n,n\geq0\}$ with invariant distribution $\pi$, we show that a valid confidence interval for $\pi(h)$ can be constructed whenever the asymptotic variance $\sigma^2_P(h)$ is finite and positive. We do not impose any additional condition on the convergence rate of the Markov chain. The confidence interval is derived using the so-called fixed-b lag-window estimator of $\sigma_P^2(h)$. We also derive a result that suggests that the proposed confidence interval procedure converges faster than classical confidence interval procedures based on the Gaussian distribution and standard central limit theorems for Markov chains.Comment: Published at http://dx.doi.org/10.3150/15-BEJ712 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Nonparametric confidence intervals for monotone functions

We study nonparametric isotonic confidence intervals for monotone functions. In Banerjee and Wellner (2001) pointwise confidence intervals, based on likelihood ratio tests for the restricted and unrestricted MLE in the current status model, are introduced. We extend the method to the treatment of other models with monotone functions, and demonstrate our method by a new proof of the results in Banerjee and Wellner (2001) and also by constructing confidence intervals for monotone densities, for which still theory had to be developed. For the latter model we prove that the limit distribution of the LR test under the null hypothesis is the same as in the current status model. We compare the confidence intervals, so obtained, with confidence intervals using the smoothed maximum likelihood estimator (SMLE), using bootstrap methods. The `Lagrange-modified' cusum diagrams, developed here, are an essential tool both for the computation of the restricted MLEs and for the development of the theory for the confidence intervals, based on the LR tests.Comment: 31 pages, 13 figure