6 research outputs found
Conditional Generative Models are Provably Robust: Pointwise Guarantees for Bayesian Inverse Problems
Conditional generative models became a very powerful tool to sample from
Bayesian inverse problem posteriors. It is well-known in classical Bayesian
literature that posterior measures are quite robust with respect to
perturbations of both the prior measure and the negative log-likelihood, which
includes perturbations of the observations. However, to the best of our
knowledge, the robustness of conditional generative models with respect to
perturbations of the observations has not been investigated yet. In this paper,
we prove for the first time that appropriately learned conditional generative
models provide robust results for single observations
Generative Sliced MMD Flows with Riesz Kernels
Maximum mean discrepancy (MMD) flows suffer from high computational costs in
large scale computations. In this paper, we show that MMD flows with Riesz
kernels , have exceptional properties which
allow for their efficient computation. First, the MMD of Riesz kernels
coincides with the MMD of their sliced version. As a consequence, the
computation of gradients of MMDs can be performed in the one-dimensional
setting. Here, for , a simple sorting algorithm can be applied to reduce
the complexity from to for two empirical
measures with and support points. For the implementations we
approximate the gradient of the sliced MMD by using only a finite number of
slices. We show that the resulting error has complexity , where
is the data dimension. These results enable us to train generative models
by approximating MMD gradient flows by neural networks even for large scale
applications. We demonstrate the efficiency of our model by image generation on
MNIST, FashionMNIST and CIFAR10
PatchNR: Learning from Very Few Images by Patch Normalizing Flow Regularization
Learning neural networks using only few available information is an important
ongoing research topic with tremendous potential for applications. In this
paper, we introduce a powerful regularizer for the variational modeling of
inverse problems in imaging. Our regularizer, called patch normalizing flow
regularizer (patchNR), involves a normalizing flow learned on small patches of
very few images. In particular, the training is independent of the considered
inverse problem such that the same regularizer can be applied for different
forward operators acting on the same class of images. By investigating the
distribution of patches versus those of the whole image class, we prove that
our model is indeed a MAP approach. Numerical examples for low-dose and
limited-angle computed tomography (CT) as well as superresolution of material
images demonstrate that our method provides very high quality results. The
training set consists of just six images for CT and one image for
superresolution. Finally, we combine our patchNR with ideas from internal
learning for performing superresolution of natural images directly from the
low-resolution observation without knowledge of any high-resolution image
Learning Regularization Parameter-Maps for Variational Image Reconstruction using Deep Neural Networks and Algorithm Unrolling
We introduce a method for fast estimation of data-adapted, spatio-temporally dependent regularization parameter-maps for variational image reconstruction, focusing on total variation (TV)-minimization. Our approach is inspired by recent developments in algorithm unrolling using deep neural networks (NNs), and relies on two distinct sub-networks. The first sub-network estimates the regularization parameter-map from the input data. The second sub-network unrolls T iterations of an iterative algorithm which approximately solves the corresponding TV-minimization problem incorporating the previously estimated regularization parameter-map. The overall network is trained end-to-end in a supervised learning fashion using pairs of clean-corrupted data but crucially without the need of having access to labels for the optimal regularization parameter-maps. We prove consistency of the unrolled scheme by showing that the unrolled energy functional used for the supervised learning Γ-converges as T tends to infinity, to the corresponding functional that incorporates the exact solution map of the TV-minimization problem. We apply and evaluate our method on a variety of large scale and dynamic imaging problems in which the automatic computation of such parameters has been so far challenging: 2D dynamic cardiac MRI reconstruction, quantitative brain MRI reconstruction, low-dose CT and dynamic image denoising. The proposed method consistently improves the TV-reconstructions using scalar parameters and the obtained parameter-maps adapt well to each imaging problem and data by leading to the preservation of detailed features. Although the choice of the regularization parameter-maps is data-driven and based on NNs, the proposed algorithm is entirely interpretable since it inherits the properties of the respective iterative reconstruction method from which the network is implicitly defined
Unrolled three-operator splitting for parameter-map learning in low dose X-ray CT reconstruction
We propose a method for fast and automatic estimation of spatially dependent regularization maps for total variation-based (TV) tomography reconstruction. The estimation is based on two distinct sub-networks, with the first sub-network estimating the regularization parameter-map from the input data while the second one unrolling T iterations of the Primal-Dual Three-Operator Splitting (PD3O) algorithm. The latter approximately solves the corresponding TV-minimization problem incorporating the previously estimated regularization parameter-map. The overall network is then trained end-to-end in a supervised learning fashion using pairs of clean-corrupted data but crucially without the need of having access to labels for the optimal regularization parameter-maps
WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution
Exploiting image patches instead of whole images have proved to be a powerful
approach to tackle various problems in image processing. Recently, Wasserstein
patch priors (WPP), which are based on the comparison of the patch
distributions of the unknown image and a reference image, were successfully
used as data-driven regularizers in the variational formulation of
superresolution. However, for each input image, this approach requires the
solution of a non-convex minimization problem which is computationally costly.
In this paper, we propose to learn two kinds of neural networks in an
unsupervised way based on WPP loss functions. First, we show how convolutional
neural networks (CNNs) can be incorporated. Once the network, called WPPNet, is
learned, it can very efficiently applied to any input image. Second, we
incorporate conditional normalizing flows to provide a tool for uncertainty
quantification. Numerical examples demonstrate the very good performance of
WPPNets for superresolution in various image classes even if the forward
operator is known only approximately