7,680 research outputs found
Performance Bounds for Parameter Estimation under Misspecified Models: Fundamental findings and applications
Inferring information from a set of acquired data is the main objective of
any signal processing (SP) method. In particular, the common problem of
estimating the value of a vector of parameters from a set of noisy measurements
is at the core of a plethora of scientific and technological advances in the
last decades; for example, wireless communications, radar and sonar,
biomedicine, image processing, and seismology, just to name a few. Developing
an estimation algorithm often begins by assuming a statistical model for the
measured data, i.e. a probability density function (pdf) which if correct,
fully characterizes the behaviour of the collected data/measurements.
Experience with real data, however, often exposes the limitations of any
assumed data model since modelling errors at some level are always present.
Consequently, the true data model and the model assumed to derive the
estimation algorithm could differ. When this happens, the model is said to be
mismatched or misspecified. Therefore, understanding the possible performance
loss or regret that an estimation algorithm could experience under model
misspecification is of crucial importance for any SP practitioner. Further,
understanding the limits on the performance of any estimator subject to model
misspecification is of practical interest. Motivated by the widespread and
practical need to assess the performance of a mismatched estimator, the goal of
this paper is to help to bring attention to the main theoretical findings on
estimation theory, and in particular on lower bounds under model
misspecification, that have been published in the statistical and econometrical
literature in the last fifty years. Secondly, some applications are discussed
to illustrate the broad range of areas and problems to which this framework
extends, and consequently the numerous opportunities available for SP
researchers.Comment: To appear in the IEEE Signal Processing Magazin
Computing the Cramer-Rao bound of Markov random field parameters: Application to the Ising and the Potts models
This report considers the problem of computing the Cramer-Rao bound for the
parameters of a Markov random field. Computation of the exact bound is not
feasible for most fields of interest because their likelihoods are intractable
and have intractable derivatives. We show here how it is possible to formulate
the computation of the bound as a statistical inference problem that can be
solve approximately, but with arbitrarily high accuracy, by using a Monte Carlo
method. The proposed methodology is successfully applied on the Ising and the
Potts models.% where it is used to assess the performance of three state-of-the
art estimators of the parameter of these Markov random fields
Variational Bayes with Intractable Likelihood
Variational Bayes (VB) is rapidly becoming a popular tool for Bayesian
inference in statistical modeling. However, the existing VB algorithms are
restricted to cases where the likelihood is tractable, which precludes the use
of VB in many interesting situations such as in state space models and in
approximate Bayesian computation (ABC), where application of VB methods was
previously impossible. This paper extends the scope of application of VB to
cases where the likelihood is intractable, but can be estimated unbiasedly. The
proposed VB method therefore makes it possible to carry out Bayesian inference
in many statistical applications, including state space models and ABC. The
method is generic in the sense that it can be applied to almost all statistical
models without requiring too much model-based derivation, which is a drawback
of many existing VB algorithms. We also show how the proposed method can be
used to obtain highly accurate VB approximations of marginal posterior
distributions.Comment: 40 pages, 6 figure
Statistical inference in compound functional models
We consider a general nonparametric regression model called the compound
model. It includes, as special cases, sparse additive regression and
nonparametric (or linear) regression with many covariates but possibly a small
number of relevant covariates. The compound model is characterized by three
main parameters: the structure parameter describing the "macroscopic" form of
the compound function, the "microscopic" sparsity parameter indicating the
maximal number of relevant covariates in each component and the usual
smoothness parameter corresponding to the complexity of the members of the
compound. We find non-asymptotic minimax rate of convergence of estimators in
such a model as a function of these three parameters. We also show that this
rate can be attained in an adaptive way
- …