2 research outputs found
On the Posterior Distribution in Denoising: Application to Uncertainty Quantification
Denoisers play a central role in many applications, from noise suppression in
low-grade imaging sensors, to empowering score-based generative models. The
latter category of methods makes use of Tweedie's formula, which links the
posterior mean in Gaussian denoising (i.e., the minimum MSE denoiser) with the
score of the data distribution. Here, we derive a fundamental relation between
the higher-order central moments of the posterior distribution, and the
higher-order derivatives of the posterior mean. We harness this result for
uncertainty quantification of pre-trained denoisers. Particularly, we show how
to efficiently compute the principal components of the posterior distribution
for any desired region of an image, as well as to approximate the full marginal
distribution along those (or any other) one-dimensional directions. Our method
is fast and memory efficient, as it does not explicitly compute or store the
high-order moment tensors and it requires no training or fine tuning of the
denoiser. Code and examples are available on the project's webpage in
https://hilamanor.github.io/GaussianDenoisingPosterior/Comment: Code and examples are available on the project's webpage in
https://hilamanor.github.io/GaussianDenoisingPosterior
From Posterior Sampling to Meaningful Diversity in Image Restoration
Image restoration problems are typically ill-posed in the sense that each
degraded image can be restored in infinitely many valid ways. To accommodate
this, many works generate a diverse set of outputs by attempting to randomly
sample from the posterior distribution of natural images given the degraded
input. Here we argue that this strategy is commonly of limited practical value
because of the heavy tail of the posterior distribution. Consider for example
inpainting a missing region of the sky in an image. Since there is a high
probability that the missing region contains no object but clouds, any set of
samples from the posterior would be entirely dominated by (practically
identical) completions of sky. However, arguably, presenting users with only
one clear sky completion, along with several alternative solutions such as
airships, birds, and balloons, would better outline the set of possibilities.
In this paper, we initiate the study of meaningfully diverse image restoration.
We explore several post-processing approaches that can be combined with any
diverse image restoration method to yield semantically meaningful diversity.
Moreover, we propose a practical approach for allowing diffusion based image
restoration methods to generate meaningfully diverse outputs, while incurring
only negligent computational overhead. We conduct extensive user studies to
analyze the proposed techniques, and find the strategy of reducing similarity
between outputs to be significantly favorable over posterior sampling. Code and
examples are available in https://noa-cohen.github.io/MeaningfulDiversityInIRComment: Code and examples are available in
https://noa-cohen.github.io/MeaningfulDiversityInI