1,889 research outputs found
Revisiting maximum-a-posteriori estimation in log-concave models
Maximum-a-posteriori (MAP) estimation is the main Bayesian estimation
methodology in imaging sciences, where high dimensionality is often addressed
by using Bayesian models that are log-concave and whose posterior mode can be
computed efficiently by convex optimisation. Despite its success and wide
adoption, MAP estimation is not theoretically well understood yet. The
prevalent view in the community is that MAP estimation is not proper Bayesian
estimation in a decision-theoretic sense because it does not minimise a
meaningful expected loss function (unlike the minimum mean squared error (MMSE)
estimator that minimises the mean squared loss). This paper addresses this
theoretical gap by presenting a decision-theoretic derivation of MAP estimation
in Bayesian models that are log-concave. A main novelty is that our analysis is
based on differential geometry, and proceeds as follows. First, we use the
underlying convex geometry of the Bayesian model to induce a Riemannian
geometry on the parameter space. We then use differential geometry to identify
the so-called natural or canonical loss function to perform Bayesian point
estimation in that Riemannian manifold. For log-concave models, this canonical
loss is the Bregman divergence associated with the negative log posterior
density. We then show that the MAP estimator is the only Bayesian estimator
that minimises the expected canonical loss, and that the posterior mean or MMSE
estimator minimises the dual canonical loss. We also study the question of MAP
and MSSE estimation performance in large scales and establish a universal bound
on the expected canonical error as a function of dimension, offering new
insights into the good performance observed in convex problems. These results
provide a new understanding of MAP and MMSE estimation in log-concave settings,
and of the multiple roles that convex geometry plays in imaging problems.Comment: Accepted for publication in SIAM Imaging Science
On MMSE and MAP Denoising Under Sparse Representation Modeling Over a Unitary Dictionary
Among the many ways to model signals, a recent approach that draws
considerable attention is sparse representation modeling. In this model, the
signal is assumed to be generated as a random linear combination of a few atoms
from a pre-specified dictionary. In this work we analyze two Bayesian denoising
algorithms -- the Maximum-Aposteriori Probability (MAP) and the
Minimum-Mean-Squared-Error (MMSE) estimators, under the assumption that the
dictionary is unitary. It is well known that both these estimators lead to a
scalar shrinkage on the transformed coefficients, albeit with a different
response curve. In this work we start by deriving closed-form expressions for
these shrinkage curves and then analyze their performance. Upper bounds on the
MAP and the MMSE estimation errors are derived. We tie these to the error
obtained by a so-called oracle estimator, where the support is given,
establishing a worst-case gain-factor between the MAP/MMSE estimation errors
and the oracle's performance. These denoising algorithms are demonstrated on
synthetic signals and on true data (images).Comment: 29 pages, 10 figure
- …