3D shape matching and registration : a probabilistic perspective

Dense correspondence is a key area in computer vision and medical image analysis. It has applications in registration and shape analysis. In this thesis, we develop a technique to recover dense correspondences between the surfaces of neuroanatomical objects over heterogeneous populations of individuals. We recover dense correspondences based on 3D shape matching. In this thesis, the 3D shape matching problem is formulated under the framework of Markov Random Fields (MRFs). We represent the surfaces of neuroanatomical objects as genus zero voxel-based meshes. The surface meshes are projected into a Markov random field space. The projection carries both geometric and topological information in terms of Gaussian curvature and mesh neighbourhood from the original space to the random field space. Gaussian curvature is projected to the nodes of the MRF, and the mesh neighbourhood structure is projected to the edges. 3D shape matching between two surface meshes is then performed by solving an energy function minimisation problem formulated with MRFs. The outcome of the 3D shape matching is dense point-to-point correspondences. However, the minimisation of the energy function is NP hard. In this thesis, we use belief propagation to perform the probabilistic inference for 3D shape matching. A sparse update loopy belief propagation algorithm adapted to the 3D shape matching is proposed to obtain an approximate global solution for the 3D shape matching problem. The sparse update loopy belief propagation algorithm demonstrates significant efficiency gain compared to standard belief propagation. The computational complexity and convergence property analysis for the sparse update loopy belief propagation algorithm are also conducted in the thesis. We also investigate randomised algorithms to minimise the energy function. In order to enhance the shape matching rate and increase the inlier support set, we propose a novel clamping technique. The clamping technique is realized by combining the loopy belief propagation message updating rule with the feedback from 3D rigid body registration. By using this clamping technique, the correct shape matching rate is increased significantly. Finally, we investigate 3D shape registration techniques based on the 3D shape matching result. Based on the point-to-point dense correspondences obtained from the 3D shape matching, a three-point based transformation estimation technique is combined with the RANdom SAmple Consensus (RANSAC) algorithm to obtain the inlier support set. The global registration approach is purely dependent on point-wise correspondences between two meshed surfaces. It has the advantage that the need for orientation initialisation is eliminated and that all shapes of spherical topology. The comparison of our MRF based 3D registration approach with a state-of-the-art registration algorithm, the first order ellipsoid template, is conducted in the experiments. These show dense correspondence for pairs of hippocampi from two different data sets, each of around 20 60+ year old healthy individuals

A Dynamic Programming Solution to Bounded Dejittering Problems

We propose a dynamic programming solution to image dejittering problems with bounded displacements and obtain efficient algorithms for the removal of line jitter, line pixel jitter, and pixel jitter.Comment: The final publication is available at link.springer.co

Optical flow estimation using steered-L1 norm

Motion is a very important part of understanding the visual picture of the surrounding environment. In image processing it involves the estimation of displacements for image points in an image sequence. In this context dense optical flow estimation is concerned with the computation of pixel displacements in a sequence of images, therefore it has been used widely in the field of image processing and computer vision. A lot of research was dedicated to enable an accurate and fast motion computation in image sequences. Despite the recent advances in the computation of optical flow, there is still room for improvements and optical flow algorithms still suffer from several issues, such as motion discontinuities, occlusion handling, and robustness to illumination changes. This thesis includes an investigation for the topic of optical flow and its applications. It addresses several issues in the computation of dense optical flow and proposes solutions. Specifically, this thesis is divided into two main parts dedicated to address two main areas of interest in optical flow. In the first part, image registration using optical flow is investigated. Both local and global image registration has been used for image registration. An image registration based on an improved version of the combined Local-global method of optical flow computation is proposed. A bi-lateral filter was used in this optical flow method to improve the edge preserving performance. It is shown that image registration via this method gives more robust results compared to the local and the global optical flow methods previously investigated. The second part of this thesis encompasses the main contribution of this research which is an improved total variation L1 norm. A smoothness term is used in the optical flow energy function to regularise this function. The L1 is a plausible choice for such a term because of its performance in preserving edges, however this term is known to be isotropic and hence decreases the penalisation near motion boundaries in all directions. The proposed improved L1 (termed here as the steered-L1 norm) smoothness term demonstrates similar performance across motion boundaries but improves the penalisation performance along such boundaries

Dense Motion Estimation for Smoke

Motion estimation for highly dynamic phenomena such as smoke is an open challenge for Computer Vision. Traditional dense motion estimation algorithms have difficulties with non-rigid and large motions, both of which are frequently observed in smoke motion. We propose an algorithm for dense motion estimation of smoke. Our algorithm is robust, fast, and has better performance over different types of smoke compared to other dense motion estimation algorithms, including state of the art and neural network approaches. The key to our contribution is to use skeletal flow, without explicit point matching, to provide a sparse flow. This sparse flow is upgraded to a dense flow. In this paper we describe our algorithm in greater detail, and provide experimental evidence to support our claims.Comment: ACCV201

Improved depth recovery in consumer depth cameras via disparity space fusion within cross-spectral stereo.

We address the issue of improving depth coverage in consumer depth cameras based on the combined use of cross-spectral stereo and near infra-red structured light sensing. Specifically we show that fusion of disparity over these modalities, within the disparity space image, prior to disparity optimization facilitates the recovery of scene depth information in regions where structured light sensing fails. We show that this joint approach, leveraging disparity information from both structured light and cross-spectral sensing, facilitates the joint recovery of global scene depth comprising both texture-less object depth, where conventional stereo otherwise fails, and highly reflective object depth, where structured light (and similar) active sensing commonly fails. The proposed solution is illustrated using dense gradient feature matching and shown to outperform prior approaches that use late-stage fused cross-spectral stereo depth as a facet of improved sensing for consumer depth cameras

Saliency-guided integration of multiple scans

we present a novel method..

Review of the mathematical foundations of data fusion techniques in surface metrology

The recent proliferation of engineered surfaces, including freeform and structured surfaces, is challenging current metrology techniques. Measurement using multiple sensors has been proposed to achieve enhanced benefits, mainly in terms of spatial frequency bandwidth, which a single sensor cannot provide. When using data from different sensors, a process of data fusion is required and there is much active research in this area. In this paper, current data fusion methods and applications are reviewed, with a focus on the mathematical foundations of the subject. Common research questions in the fusion of surface metrology data are raised and potential fusion algorithms are discussed

Discontinuity preserving image registration for breathing induced sliding organ motion

Image registration is a powerful tool in medical image analysis and facilitates the clinical routine in several aspects. It became an indispensable device for many medical applications including image-guided therapy systems. The basic goal of image registration is to spatially align two images that show a similar region of interest. More speci�cally, a displacement �eld respectively a transformation is estimated, that relates the positions of the pixels or feature points in one image to the corresponding positions in the other one. The so gained alignment of the images assists the doctor in comparing and diagnosing them. There exist di�erent kinds of image registration methods, those which are capable to estimate a rigid transformation or more generally an a�ne transformation between the images and those which are able to capture a more complex motion by estimating a non-rigid transformation. There are many well established non-rigid registration methods, but those which are able to preserve discontinuities in the displacement �eld are rather rare. These discontinuities appear in particular at organ boundaries during the breathing induced organ motion. In this thesis, we make use of the idea to combine motion segmentation with registration to tackle the problem of preserving the discontinuities in the resulting displacement �eld. We introduce a binary function to represent the motion segmentation and the proposed discontinuity preserving non-rigid registration method is then formulated in a variational framework. Thus, an energy functional is de�ned and its minimisation with respect to the displacement �eld and the motion segmentation will lead to the desired result. In theory, one can prove that for the motion segmentation a global minimiser of the energy functional can be found, if the displacement �eld is given. The overall minimisation problem, however, is non-convex and a suitable optimisation strategy has to be considered. Furthermore, depending on whether we use the pure L1-norm or an approximation of it in the formulation of the energy functional, we use di�erent numerical methods to solve the minimisation problem. More speci�cally, when using an approximation of the L1-norm, the minimisation of the energy functional with respect to the displacement �eld is performed through Brox et al.'s �xed point iteration scheme, and the minimisation with respect to the motion segmentation with the dual algorithm of Chambolle. On the other hand, when we make use of the pure L1-norm in the energy functional, the primal-dual algorithm of Chambolle and Pock is used for both, the minimisation with respect to the displacement �eld and the motion segmentation. This approach is clearly faster compared to the one using the approximation of the L1-norm and also theoretically more appealing. Finally, to support the registration method during the minimisation process, we incorporate additionally in a later approach the information of certain landmark positions into the formulation of the energy functional, that makes use of the pure L1-norm. Similarly as before, the primal-dual algorithm of Chambolle and Pock is then used for both, the minimisation with respect to the displacement �eld and the motion segmentation. All the proposed non-rigid discontinuity preserving registration methods delivered promising results for experiments with synthetic images and real MR images of breathing induced liver motion

Image processing for plastic surgery planning

This thesis presents some image processing tools for plastic surgery planning. In particular, it presents a novel method that combines local and global context in a probabilistic relaxation framework to identify cephalometric landmarks used in Maxillofacial plastic surgery. It also uses a method that utilises global and local symmetry to identify abnormalities in CT frontal images of the human body. The proposed methodologies are evaluated with the help of several clinical data supplied by collaborating plastic surgeons