21,452 research outputs found
Local Exchangeability
Exchangeability---in which the distribution of an infinite sequence is
invariant to reorderings of its elements---implies the existence of a simple
conditional independence structure that may be leveraged in the design of
probabilistic models, efficient inference algorithms, and randomization-based
testing procedures. In practice, however, this assumption is too strong an
idealization; the distribution typically fails to be exactly invariant to
permutations and de Finetti's representation theory does not apply. Thus there
is the need for a distributional assumption that is both weak enough to hold in
practice, and strong enough to guarantee a useful underlying representation. We
introduce a relaxed notion of local exchangeability---where swapping data
associated with nearby covariates causes a bounded change in the distribution.
We prove that locally exchangeable processes correspond to independent
observations from an underlying measure-valued stochastic process. We thereby
show that de Finetti's theorem is robust to perturbation and provide further
justification for the Bayesian modelling approach. Using this probabilistic
result, we develop three novel statistical procedures for (1) estimating the
underlying process via local empirical measures, (2) testing via local
randomization, and (3) estimating the canonical premetric of local
exchangeability. These three procedures extend the applicability of previous
exchangeability-based methods without sacrificing rigorous statistical
guarantees. The paper concludes with examples of popular statistical models
that exhibit local exchangeability
A Bayesian information criterion for singular models
We consider approximate Bayesian model choice for model selection problems
that involve models whose Fisher-information matrices may fail to be invertible
along other competing submodels. Such singular models do not obey the
regularity conditions underlying the derivation of Schwarz's Bayesian
information criterion (BIC) and the penalty structure in BIC generally does not
reflect the frequentist large-sample behavior of their marginal likelihood.
While large-sample theory for the marginal likelihood of singular models has
been developed recently, the resulting approximations depend on the true
parameter value and lead to a paradox of circular reasoning. Guided by examples
such as determining the number of components of mixture models, the number of
factors in latent factor models or the rank in reduced-rank regression, we
propose a resolution to this paradox and give a practical extension of BIC for
singular model selection problems
Empirical Bayes selection of wavelet thresholds
This paper explores a class of empirical Bayes methods for level-dependent
threshold selection in wavelet shrinkage. The prior considered for each wavelet
coefficient is a mixture of an atom of probability at zero and a heavy-tailed
density. The mixing weight, or sparsity parameter, for each level of the
transform is chosen by marginal maximum likelihood. If estimation is carried
out using the posterior median, this is a random thresholding procedure; the
estimation can also be carried out using other thresholding rules with the same
threshold. Details of the calculations needed for implementing the procedure
are included. In practice, the estimates are quick to compute and there is
software available. Simulations on the standard model functions show excellent
performance, and applications to data drawn from various fields of application
are used to explore the practical performance of the approach. By using a
general result on the risk of the corresponding marginal maximum likelihood
approach for a single sequence, overall bounds on the risk of the method are
found subject to membership of the unknown function in one of a wide range of
Besov classes, covering also the case of f of bounded variation. The rates
obtained are optimal for any value of the parameter p in (0,\infty],
simultaneously for a wide range of loss functions, each dominating the L_q norm
of the \sigmath derivative, with \sigma\ge0 and 0<q\le2.Comment: Published at http://dx.doi.org/10.1214/009053605000000345 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Some Aspects of Measurement Error in Linear Regression of Astronomical Data
I describe a Bayesian method to account for measurement errors in linear
regression of astronomical data. The method allows for heteroscedastic and
possibly correlated measurement errors, and intrinsic scatter in the regression
relationship. The method is based on deriving a likelihood function for the
measured data, and I focus on the case when the intrinsic distribution of the
independent variables can be approximated using a mixture of Gaussians. I
generalize the method to incorporate multiple independent variables,
non-detections, and selection effects (e.g., Malmquist bias). A Gibbs sampler
is described for simulating random draws from the probability distribution of
the parameters, given the observed data. I use simulation to compare the method
with other common estimators. The simulations illustrate that the Gaussian
mixture model outperforms other common estimators and can effectively give
constraints on the regression parameters, even when the measurement errors
dominate the observed scatter, source detection fraction is low, or the
intrinsic distribution of the independent variables is not a mixture of
Gaussians. I conclude by using this method to fit the X-ray spectral slope as a
function of Eddington ratio using a sample of 39 z < 0.8 radio-quiet quasars. I
confirm the correlation seen by other authors between the radio-quiet quasar
X-ray spectral slope and the Eddington ratio, where the X-ray spectral slope
softens as the Eddington ratio increases.Comment: 39 pages, 11 figures, 1 table, accepted by ApJ. IDL routines
(linmix_err.pro) for performing the Markov Chain Monte Carlo are available at
the IDL astronomy user's library, http://idlastro.gsfc.nasa.gov/homepage.htm
- …