A Variational Model for Joint Motion Estimation and Image Reconstruction

The aim of this paper is to derive and analyze a variational model for the joint estimation of motion and reconstruction of image sequences, which is based on a time-continuous Eulerian motion model. The model can be set up in terms of the continuity equation or the brightness constancy equation. The analysis in this paper focuses on the latter for robust motion estimation on sequences of twodimensional images. We rigorously prove the existence of a minimizer in a suitable function space setting. Moreover, we discuss the numerical solution of the model based on primal-dual algorithms and investigate several examples. Finally, the benefits of our model compared to existing techniques, such as sequential image reconstruction and motion estimation, are shown.The work of the first author was also supported by the German Science Foundation DFG via EXC 1003 Cells in Motion Cluster of Excellence, M¨unster, German

A Variational Reconstruction Method for Undersampled Dynamic X-ray Tomography based on Physical Motion Models

In this paper we study the reconstruction of moving object densities from undersampled dynamic x-ray tomography in two dimensions. A particular motivation of this study is to use realistic measurement protocols for practical applications, i.e. we do not assume to have a full Radon transform in each time step, but only projections in few angular directions. This restriction enforces a space-time reconstruction, which we perform by incorporating physical motion models and regularization of motion vectors in a variational framework. The methodology of optical flow, which is one of the most common methods to estimate motion between two images, is utilized to formulate a joint variational model for reconstruction and motion estimation. We provide a basic mathematical analysis of the forward model and the variational model for the image reconstruction. Moreover, we discuss the efficient numerical minimization based on alternating minimizations between images and motion vectors. A variety of results are presented for simulated and real measurement data with different sampling strategy. A key observation is that random sampling combined with our model allows reconstructions of similar amount of measurements and quality as a single static reconstruction.Peer reviewe

Joint Image Reconstruction and Motion Estimation for Spatiotemporal Imaging

International audienceWe propose a variational model for joint image reconstruction and motion estimation applicable to spatiotemporal imaging. This model consists of two parts, one that conducts image reconstruction in a static setting and another that estimates the motion by solving a sequence of coupled indirect image registration problems, each formulated within the large deformation diffeomorphic metric mapping framework. The proposed model is compared against alternative approaches (optical flow based model and diffeomorphic motion models). Next, we derive efficient algorithms for a time-discretized setting and show that the optimal solution of the time-discretized formulation is consistent with that of the time-continuous one. The complexity of the algorithm is characterized and we conclude by giving some numerical examples in 2D space + time tomography with very sparse and/or highly noisy dat

Variational methods for joint motion estimation and image reconstruction

In dieser Dissertation werden verschiedene Techniken zur Bewegungsschätzung aus Bildsequenzen vorgestellt und mit Methoden zur Bildrekonstruktion verbunden. Die Arbeit gliedert sich in zwei Modellierungsteile und einen Anwendungsteil. Im ersten Modellierungsteil werden Variationsmethoden für die Bewegungsschätzung hergeleitet. Wir illustrieren hier sowohl Modellierung als auch numerische Implementierung basierend auf einem primal-dualen Modell. Alle vorgestellten Modelle werden numerisch evaluiert. Darauf aufbauend werden im zweiten Teil der Arbeit Modelle zur gleichzeitigen Bewegungsschätzung und Bildrekonstruktion entwickelt. Neben einem Beweis für die Existenz eines Minimierers gehen wir ausführlich auf die numerische Implementierung ein. Der Anwendungsteil der Arbeit teilt sich in ein Kapitel über Bildsegmentierung und ein weiteres zur Anwendung der neu erarbeiteten Modelle.In this thesis we present different techniques for motion estimation from image sequences and combine them with image reconstruction. The main body of this work is divided in two modeling and one application part. The first model is a variational approach for motion estimation from image sequences. Mathematical background as well as different models for motion estimation are presented. We illustrate the numerical realization based on a primal-dual framework and evaluate our model towards different types of motion. In the second main part we connect the field of motion estimation to the task of image reconstruction. We deduce variational models for joint motion estimation and image reconstruction, prove existence of minimizers and present primal-dual schemes for the numerical implementation. The application part divides into chapters about image segmentation and application of our joint models to denoise image sequences and estimate their underlying motion simultaneously

Joint Image Reconstruction and Motion Estimation for Spatiotemporal Imaging

International audienceWe propose a variational model for joint image reconstruction and motion estimation applicable to spatiotemporal imaging. This model consists of two parts, one that conducts image reconstruction in a static setting and another that estimates the motion by solving a sequence of coupled indirect image registration problems, each formulated within the large deformation diffeomorphic metric mapping framework. The proposed model is compared against alternative approaches (optical flow based model and diffeomorphic motion models). Next, we derive efficient algorithms for a time-discretized setting and show that the optimal solution of the time-discretized formulation is consistent with that of the time-continuous one. The complexity of the algorithm is characterized and we conclude by giving some numerical examples in 2D space + time tomography with very sparse and/or highly noisy dat

Enhancing Compressed Sensing 4D Photoacoustic Tomography by Simultaneous Motion Estimation

A crucial limitation of current high-resolution 3D photoacoustic tomography (PAT) devices that employ sequential scanning is their long acquisition time. In previous work, we demonstrated how to use compressed sensing techniques to improve upon this: images with good spatial resolution and contrast can be obtained from suitably sub-sampled PAT data acquired by novel acoustic scanning systems if sparsity-constrained image reconstruction techniques such as total variation regularization are used. Now, we show how a further increase of image quality can be achieved for imaging dynamic processes in living tissue (4D PAT). The key idea is to exploit the additional temporal redundancy of the data by coupling the previously used spatial image reconstruction models with sparsity-constrained motion estimation models. While simulated data from a two-dimensional numerical phantom will be used to illustrate the main properties of this recently developed joint-image-reconstruction-and-motion-estimation framework, measured data from a dynamic experimental phantom will also be used to demonstrate their potential for challenging, large-scale, real-world, three-dimensional scenarios. The latter only becomes feasible if a carefully designed combination of tailored optimization schemes is employed, which we describe and examine in more detail

Variational semi-blind sparse deconvolution with orthogonal kernel bases and its application to MRFM

We present a variational Bayesian method of joint image reconstruction and point spread function (PSF) estimation when the PSF of the imaging device is only partially known. To solve this semi-blind deconvolution problem, prior distributions are specified for the PSF and the 3D image. Joint image reconstruction and PSF estimation is then performed within a Bayesian framework, using a variational algorithm to estimate the posterior distribution. The image prior distribution imposes an explicit atomic measure that corresponds to image sparsity. Importantly, the proposed Bayesian deconvolution algorithm does not require hand tuning. Simulation results clearly demonstrate that the semi-blind deconvolution algorithm compares favorably with previous Markov chain Monte Carlo (MCMC) version of myopic sparse reconstruction. It significantly outperforms mismatched non-blind algorithms that rely on the assumption of the perfect knowledge of the PSF. The algorithm is illustrated on real data from magnetic resonance force microscopy (MRFM)