Integrated Registration, Segmentation, and Interpolation for 3D/4D Sparse Data

We address the problem of object modelling from 3D and 4D sparse data acquired as different sequences which are misaligned with respect to each other. Such data may result from various imaging modalities and can therefore present very diverse spatial configurations and appearances. We focus on medical tomographic data, made up of sets of 2D slices having arbitrary positions and orientations, and which may have different gains and contrasts even within the same dataset. The analysis of such tomographic data is essential for establishing a diagnosis or planning surgery.Modelling from sparse and misaligned data requires solving the three inherently related problems of registration, segmentation, and interpolation. We propose a new method to integrate these stages in a level set framework. Registration is particularly challenging by the limited number of intersections present in a sparse dataset, and interpolation has to handle images that may have very different appearances. Hence, registration and interpolation exploit segmentation information, rather than pixel intensities, for increased robustness and accuracy. We achieve this by first introducing a new level set scheme based on the interpolation of the level set function by radial basis functions. This new scheme can inherently handle sparse data, and is more numerically stable and robust to noise than the classical level set. We also present a new registration algorithm based on the level set method, which is robust to local minima and can handle sparse data that have only a limited number of intersections. Then, we integrate these two methods into the same level set framework.The proposed method is validated quantitatively and subjectively on artificial data and MRI and CT scans. It is compared against a state-of-the-art, sequential method comprising traditional mutual information based registration, image interpolation, and 3D or 4D segmentation of the registered and interpolated volume. In our experiments, the proposed framework yields similar segmentation results to the sequential approach, but provides a more robust and accurate registration and interpolation. In particular, the registration is more robust to limited intersections in the data and to local minima. The interpolation is more satisfactory in cases of large gaps, due to the method taking into account the global shape of the object, and it recovers better topologies at the extremities of the shapes where the objects disappear from the image slices. As a result, the complete integrated framework provides more satisfactory shape reconstructions than the sequential approach

Sub-pixel registration for low cost, low dosage, x-ray phase contrast imaging : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Electronic and Computer Engineering at Massey University, Palmerston North, New Zealand

Figure 1.2 (=David et al., 2007 Fig 1a) was removed for copyright reasons. Figure 1.5 is re-used under a Creative Commons Attribution 4.0 International licence (CC by 4.0). Figures 4.3 & 4.4 are ©2007 IEEE and re-used with permission.X-ray phase contrast imaging is an imaging modality that measures the phase shift of the X-ray wavefronts as they travel through different materials. This gives a higher contrast between regions of similar X-ray attenuation, in a medical sense this corresponds to a higher contrast of soft tissues. A new area of research for X-ray phase contrast imaging is to use shadow-based intensity modulation to generate these images. This thesis explores a range of different registration techniques, and their suitability for phase contrast imaging using shadow based intensity modulation. Image registration is a key step in generating the phase contrast images as it is related to the x and y differential phase. These are then integrated to generate the phase contrast image. Therefor a high accuracy sub-pixel registration technique will provide high quality phase contrast images. The registration techniques explored are 1D curve 2-D surface fitting to a correlation map, phase registration, Newton-Raphson method, and optimal interpolation filtering. These registration techniques were tested with images that are noise free, as well as images corrupted by Poisson noise. The Newton-Raphson, and the optimal interpolation filters show the most promise due to low errors in the noise free environment. In the presence of noise, the Newton-Raphson method performs poorly, and hence requires a good denoising method, while the optimal interpolation filters do not get any improvement from any denoising techniques. Currently the Newton-Raphson based method are used widely in digital image correlation, however the optimal interpolation filtering has the benefit of not being limited by the choice of interpolation technique, and it removes the iterative process, and depending on the size of the optimal interpolation filter it performs better than, or only marginally worse than the Newton-Raphson method

Local interpolation schemes for landmark-based image registration: a comparison

In this paper we focus, from a mathematical point of view, on properties and performances of some local interpolation schemes for landmark-based image registration. Precisely, we consider modified Shepard's interpolants, Wendland's functions, and Lobachevsky splines. They are quite unlike each other, but all of them are compactly supported and enjoy interesting theoretical and computational properties. In particular, we point out some unusual forms of the considered functions. Finally, detailed numerical comparisons are given, considering also Gaussians and thin plate splines, which are really globally supported but widely used in applications

Distributed-memory large deformation diffeomorphic 3D image registration

We present a parallel distributed-memory algorithm for large deformation diffeomorphic registration of volumetric images that produces large isochoric deformations (locally volume preserving). Image registration is a key technology in medical image analysis. Our algorithm uses a partial differential equation constrained optimal control formulation. Finding the optimal deformation map requires the solution of a highly nonlinear problem that involves pseudo-differential operators, biharmonic operators, and pure advection operators both forward and back- ward in time. A key issue is the time to solution, which poses the demand for efficient optimization methods as well as an effective utilization of high performance computing resources. To address this problem we use a preconditioned, inexact, Gauss-Newton- Krylov solver. Our algorithm integrates several components: a spectral discretization in space, a semi-Lagrangian formulation in time, analytic adjoints, different regularization functionals (including volume-preserving ones), a spectral preconditioner, a highly optimized distributed Fast Fourier Transform, and a cubic interpolation scheme for the semi-Lagrangian time-stepping. We demonstrate the scalability of our algorithm on images with resolution of up to $1024^3$ on the "Maverick" and "Stampede" systems at the Texas Advanced Computing Center (TACC). The critical problem in the medical imaging application domain is strong scaling, that is, solving registration problems of a moderate size of $256^3$---a typical resolution for medical images. We are able to solve the registration problem for images of this size in less than five seconds on 64 x86 nodes of TACC's "Maverick" system.Comment: accepted for publication at SC16 in Salt Lake City, Utah, USA; November 201

Temporal Interpolation via Motion Field Prediction

Navigated 2D multi-slice dynamic Magnetic Resonance (MR) imaging enables high contrast 4D MR imaging during free breathing and provides in-vivo observations for treatment planning and guidance. Navigator slices are vital for retrospective stacking of 2D data slices in this method. However, they also prolong the acquisition sessions. Temporal interpolation of navigator slices an be used to reduce the number of navigator acquisitions without degrading specificity in stacking. In this work, we propose a convolutional neural network (CNN) based method for temporal interpolation via motion field prediction. The proposed formulation incorporates the prior knowledge that a motion field underlies changes in the image intensities over time. Previous approaches that interpolate directly in the intensity space are prone to produce blurry images or even remove structures in the images. Our method avoids such problems and faithfully preserves the information in the image. Further, an important advantage of our formulation is that it provides an unsupervised estimation of bi-directional motion fields. We show that these motion fields can be used to halve the number of registrations required during 4D reconstruction, thus substantially reducing the reconstruction time.Comment: Submitted to 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherland