15 research outputs found
Consistent estimation of the basic neighborhood of Markov random fields
For Markov random fields on with finite state space, we
address the statistical estimation of the basic neighborhood, the smallest
region that determines the conditional distribution at a site on the condition
that the values at all other sites are given. A modification of the Bayesian
Information Criterion, replacing likelihood by pseudo-likelihood, is proved to
provide strongly consistent estimation from observing a realization of the
field on increasing finite regions: the estimated basic neighborhood equals the
true one eventually almost surely, not assuming any prior bound on the size of
the latter. Stationarity of the Markov field is not required, and phase
transition does not affect the results.Comment: Published at http://dx.doi.org/10.1214/009053605000000912 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Context Tree Selection: A Unifying View
The present paper investigates non-asymptotic properties of two popular
procedures of context tree (or Variable Length Markov Chains) estimation:
Rissanen's algorithm Context and the Penalized Maximum Likelihood criterion.
First showing how they are related, we prove finite horizon bounds for the
probability of over- and under-estimation. Concerning overestimation, no
boundedness or loss-of-memory conditions are required: the proof relies on new
deviation inequalities for empirical probabilities of independent interest. The
underestimation properties rely on loss-of-memory and separation conditions of
the process.
These results improve and generalize the bounds obtained previously. Context
tree models have been introduced by Rissanen as a parsimonious generalization
of Markov models. Since then, they have been widely used in applied probability
and statistics
Maximum entropy estimation of transition probabilities of reversible Markov chains
In this paper, we develop a general theory for the estimation of the
transition probabilities of reversible Markov chains using the maximum entropy
principle. A broad range of physical models can be studied within this
approach. We use one-dimensional classical spin systems to illustrate the
theoretical ideas. The examples studied in this paper are: the Ising model, the
Potts model and the Blume-Emery-Griffiths model
Divergence rates of Markov order estimators and their application to statistical estimation of stationary ergodic processes
Stationary ergodic processes with finite alphabets are estimated by finite
memory processes from a sample, an n-length realization of the process, where
the memory depth of the estimator process is also estimated from the sample
using penalized maximum likelihood (PML). Under some assumptions on the
continuity rate and the assumption of non-nullness, a rate of convergence in
-distance is obtained, with explicit constants. The result requires an
analysis of the divergence of PML Markov order estimators for not necessarily
finite memory processes. This divergence problem is investigated in more
generality for three information criteria: the Bayesian information criterion
with generalized penalty term yielding the PML, and the normalized maximum
likelihood and the Krichevsky-Trofimov code lengths. Lower and upper bounds on
the estimated order are obtained. The notion of consistent Markov order
estimation is generalized for infinite memory processes using the concept of
oracle order estimates, and generalized consistency of the PML Markov order
estimator is presented.Comment: Published in at http://dx.doi.org/10.3150/12-BEJ468 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
On estimating the memory for finitarily markovian processes
Finitarily Markovian processes are those processes
for which there is a finite () such that the conditional distribution of given
the entire past is equal to the conditional distribution of given only
. The least such value of is called the memory length.
We give a rather complete analysis of the problems of universally estimating
the least such value of , both in the backward sense that we have just
described and in the forward sense, where one observes successive values of
for and asks for the least value such that the
conditional distribution of given is the same
as the conditional distribution of given . We
allow for finite or countably infinite alphabet size
On the minimal penalty for Markov order estimation
We show that large-scale typicality of Markov sample paths implies that the
likelihood ratio statistic satisfies a law of iterated logarithm uniformly to
the same scale. As a consequence, the penalized likelihood Markov order
estimator is strongly consistent for penalties growing as slowly as log log n
when an upper bound is imposed on the order which may grow as rapidly as log n.
Our method of proof, using techniques from empirical process theory, does not
rely on the explicit expression for the maximum likelihood estimator in the
Markov case and could therefore be applicable in other settings.Comment: 29 page