2,923 research outputs found
Guessing Revisited: A Large Deviations Approach
The problem of guessing a random string is revisited. A close relation
between guessing and compression is first established. Then it is shown that if
the sequence of distributions of the information spectrum satisfies the large
deviation property with a certain rate function, then the limiting guessing
exponent exists and is a scalar multiple of the Legendre-Fenchel dual of the
rate function. Other sufficient conditions related to certain continuity
properties of the information spectrum are briefly discussed. This approach
highlights the importance of the information spectrum in determining the
limiting guessing exponent. All known prior results are then re-derived as
example applications of our unifying approach.Comment: 16 pages, to appear in IEEE Transaction on Information Theor
Information Geometry Approach to Parameter Estimation in Markov Chains
We consider the parameter estimation of Markov chain when the unknown
transition matrix belongs to an exponential family of transition matrices.
Then, we show that the sample mean of the generator of the exponential family
is an asymptotically efficient estimator. Further, we also define a curved
exponential family of transition matrices. Using a transition matrix version of
the Pythagorean theorem, we give an asymptotically efficient estimator for a
curved exponential family.Comment: Appendix D is adde
On the VC-Dimension of Binary Codes
We investigate the asymptotic rates of length- binary codes with
VC-dimension at most and minimum distance at least . Two upper
bounds are obtained, one as a simple corollary of a result by Haussler and the
other via a shortening approach combining Sauer-Shelah lemma and the linear
programming bound. Two lower bounds are given using Gilbert-Varshamov type
arguments over constant-weight and Markov-type sets
Computational statistics using the Bayesian Inference Engine
This paper introduces the Bayesian Inference Engine (BIE), a general
parallel, optimised software package for parameter inference and model
selection. This package is motivated by the analysis needs of modern
astronomical surveys and the need to organise and reuse expensive derived data.
The BIE is the first platform for computational statistics designed explicitly
to enable Bayesian update and model comparison for astronomical problems.
Bayesian update is based on the representation of high-dimensional posterior
distributions using metric-ball-tree based kernel density estimation. Among its
algorithmic offerings, the BIE emphasises hybrid tempered MCMC schemes that
robustly sample multimodal posterior distributions in high-dimensional
parameter spaces. Moreover, the BIE is implements a full persistence or
serialisation system that stores the full byte-level image of the running
inference and previously characterised posterior distributions for later use.
Two new algorithms to compute the marginal likelihood from the posterior
distribution, developed for and implemented in the BIE, enable model comparison
for complex models and data sets. Finally, the BIE was designed to be a
collaborative platform for applying Bayesian methodology to astronomy. It
includes an extensible object-oriented and easily extended framework that
implements every aspect of the Bayesian inference. By providing a variety of
statistical algorithms for all phases of the inference problem, a scientist may
explore a variety of approaches with a single model and data implementation.
Additional technical details and download details are available from
http://www.astro.umass.edu/bie. The BIE is distributed under the GNU GPL.Comment: Resubmitted version. Additional technical details and download
details are available from http://www.astro.umass.edu/bie. The BIE is
distributed under the GNU GP
- …