2,036 research outputs found
Optimisation for image processing
The main purpose of optimisation in image processing is to compensate for missing, corrupted image data, or to find good correspondences between input images. We note that image data essentially has infinite dimensionality that needs to be discretised at certain levels of resolution. Most image processing methods find a suboptimal solution, given the characteristics of the problem. While the general optimisation literature is vast, there does not seem to be an accepted universal method for all image problems. In this thesis, we consider three interrelated optimisation approaches to exploit problem structures of various relaxations to three common image processing problems:
1. The first approach to the image registration problem is based on the nonlinear programming model. Image registration is an ill-posed problem and suffers from many undesired local optima. In order to remove these unwanted solutions, certain regularisers or constraints are needed. In this thesis, prior knowledge of rigid structures of the images is included in the problem using linear and bilinear constraints. The aim is to match two images while maintaining the rigid structure of certain parts of the images. A sequential quadratic programming algorithm is used, employing dimensional reduction, to solve the resulting discretised constrained optimisation problem. We show that pre-processing of the constraints can reduce problem dimensionality. Experimental results demonstrate better performance of our proposed algorithm compare to the current methods.
2. The second approach is based on discrete Markov Random Fields (MRF). MRF has been successfully used in machine learning, artificial intelligence, image processing, including the image registration problem. In the discrete MRF model, the domain of the image problem is fixed (relaxed) to a certain range. Therefore, the optimal solution to the relaxed problem could be found in the predefined domain. The original discrete MRF is NP hard and relaxations are needed to obtain a suboptimal solution in polynomial time. One popular approach is the linear programming (LP) relaxation. However, the LP relaxation of MRF (LP-MRF) is excessively high dimensional and contains sophisticated constraints. Therefore, even one iteration of a standard LP solver (e.g. interior-point algorithm), may take too long to terminate. Dual decomposition technique has been used to formulate a convex-nondifferentiable dual LP-MRF that has geometrical advantages. This has led to the development of first order methods that take into account the MRF structure. The methods considered in this thesis for solving the dual LP-MRF are the projected subgradient and mirror descent using nonlinear weighted distance functions. An analysis of the convergence properties of the method is provided, along with improved convergence rate estimates. The experiments on synthetic data and an image segmentation problem show promising results.
3. The third approach employs a hierarchy of problem's models for computing the search directions. The first two approaches are specialised methods for image problems at a certain level of discretisation. As input images are infinite-dimensional, all computational methods require their discretisation at some levels. Clearly, high resolution images carry more information but they lead to very large scale and ill-posed optimisation problems. By contrast, although low level discretisation suffers from the loss of information, it benefits from low computational cost. In addition, a coarser representation of a fine image problem could be treated as a relaxation to the problem, i.e. the coarse problem is less ill-conditioned. Therefore, propagating a solution of a good coarse approximation to the fine problem could potentially improve the fine level. With the aim of utilising low level information within the high level process, we propose a multilevel optimisation method to solve the convex composite optimisation problem. This problem consists of the minimisation of the sum of a smooth convex function and a simple non-smooth convex function. The method iterates between fine and coarse levels of discretisation in the sense that the search direction is computed using information from either the gradient or a solution of the coarse model. We show that the proposed algorithm is a contraction on the optimal solution and demonstrate excellent performance on experiments with image restoration problems.Open Acces
Dense Vision in Image-guided Surgery
Image-guided surgery needs an efficient and effective camera tracking system in order to perform augmented reality for overlaying preoperative models or label cancerous tissues on the 2D video images of the surgical scene. Tracking in endoscopic/laparoscopic scenes however is an extremely difficult task primarily due to tissue deformation, instrument invasion into the surgical scene and the presence of specular highlights. State of the art feature-based SLAM systems such as PTAM fail in tracking such scenes since the number of good features to track is very limited. When the scene is smoky and when there are instrument motions, it will cause feature-based tracking to fail immediately.
The work of this thesis provides a systematic approach to this problem using dense vision. We initially attempted to register a 3D preoperative model with multiple 2D endoscopic/laparoscopic images using a dense method but this approach did not perform well. We subsequently proposed stereo reconstruction to directly obtain the 3D structure of the scene. By using the dense reconstructed model together with robust estimation, we demonstrate that dense stereo tracking can be incredibly robust even within extremely challenging endoscopic/laparoscopic scenes.
Several validation experiments have been conducted in this thesis. The proposed stereo reconstruction algorithm has turned out to be the state of the art method for several publicly available ground truth datasets. Furthermore, the proposed robust dense stereo tracking algorithm has been proved highly accurate in synthetic environment (< 0.1 mm RMSE) and qualitatively extremely robust when being applied to real scenes in RALP prostatectomy surgery. This is an important step toward achieving accurate image-guided laparoscopic surgery.Open Acces
To Learn or Not to Learn Features for Deformable Registration?
Feature-based registration has been popular with a variety of features
ranging from voxel intensity to Self-Similarity Context (SSC). In this paper,
we examine the question on how features learnt using various Deep Learning (DL)
frameworks can be used for deformable registration and whether this feature
learning is necessary or not. We investigate the use of features learned by
different DL methods in the current state-of-the-art discrete registration
framework and analyze its performance on 2 publicly available datasets. We draw
insights into the type of DL framework useful for feature learning and the
impact, if any, of the complexity of different DL models and brain parcellation
methods on the performance of discrete registration. Our results indicate that
the registration performance with DL features and SSC are comparable and stable
across datasets whereas this does not hold for low level features.Comment: 9 pages, 4 figure
Numerical methods for coupled reconstruction and registration in digital breast tomosynthesis.
Digital Breast Tomosynthesis (DBT) provides an insight into the fine details of normal fibroglandular tissues and abnormal lesions by reconstructing a pseudo-3D image of the breast. In this respect, DBT overcomes a major limitation of conventional X-ray mam- mography by reducing the confounding effects caused by the superposition of breast tissue. In a breast cancer screening or diagnostic context, a radiologist is interested in detecting change, which might be indicative of malignant disease. To help automate this task image registration is required to establish spatial correspondence between time points. Typically, images, such as MRI or CT, are first reconstructed and then registered. This approach can be effective if reconstructing using a complete set of data. However, for ill-posed, limited-angle problems such as DBT, estimating the deformation is com- plicated by the significant artefacts associated with the reconstruction, leading to severe inaccuracies in the registration. This paper presents a mathematical framework, which couples the two tasks and jointly estimates both image intensities and the parameters of a transformation. Under this framework, we compare an iterative method and a simultaneous method, both of which tackle the problem of comparing DBT data by combining reconstruction of a pair of temporal volumes with their registration. We evaluate our methods using various computational digital phantoms, uncom- pressed breast MR images, and in-vivo DBT simulations. Firstly, we compare both iter- ative and simultaneous methods to the conventional, sequential method using an affine transformation model. We show that jointly estimating image intensities and parametric transformations gives superior results with respect to reconstruction fidelity and regis- tration accuracy. Also, we incorporate a non-rigid B-spline transformation model into our simultaneous method. The results demonstrate a visually plausible recovery of the deformation with preservation of the reconstruction fidelity
Discontinuity preserving image registration for breathing induced sliding organ motion
Image registration is a powerful tool in medical image analysis and facilitates
the clinical routine in several aspects. It became an indispensable device for
many medical applications including image-guided therapy systems. The
basic goal of image registration is to spatially align two images that show a
similar region of interest. More speci�cally, a displacement �eld respectively
a transformation is estimated, that relates the positions of the pixels or
feature points in one image to the corresponding positions in the other one.
The so gained alignment of the images assists the doctor in comparing and
diagnosing them. There exist di�erent kinds of image registration methods,
those which are capable to estimate a rigid transformation or more generally
an a�ne transformation between the images and those which are able to
capture a more complex motion by estimating a non-rigid transformation.
There are many well established non-rigid registration methods, but those
which are able to preserve discontinuities in the displacement �eld are rather
rare. These discontinuities appear in particular at organ boundaries during
the breathing induced organ motion.
In this thesis, we make use of the idea to combine motion segmentation
with registration to tackle the problem of preserving the discontinuities in
the resulting displacement �eld. We introduce a binary function to represent
the motion segmentation and the proposed discontinuity preserving
non-rigid registration method is then formulated in a variational framework.
Thus, an energy functional is de�ned and its minimisation with respect to
the displacement �eld and the motion segmentation will lead to the desired
result. In theory, one can prove that for the motion segmentation a global
minimiser of the energy functional can be found, if the displacement �eld
is given. The overall minimisation problem, however, is non-convex and a
suitable optimisation strategy has to be considered. Furthermore, depending
on whether we use the pure L1-norm or an approximation of it in the formulation
of the energy functional, we use di�erent numerical methods to solve
the minimisation problem. More speci�cally, when using an approximation
of the L1-norm, the minimisation of the energy functional with respect to the displacement �eld is performed through Brox et al.'s �xed point iteration
scheme, and the minimisation with respect to the motion segmentation
with the dual algorithm of Chambolle. On the other hand, when we make
use of the pure L1-norm in the energy functional, the primal-dual algorithm
of Chambolle and Pock is used for both, the minimisation with respect to
the displacement �eld and the motion segmentation. This approach is clearly
faster compared to the one using the approximation of the L1-norm and also
theoretically more appealing. Finally, to support the registration method
during the minimisation process, we incorporate additionally in a later approach
the information of certain landmark positions into the formulation of
the energy functional, that makes use of the pure L1-norm. Similarly as before,
the primal-dual algorithm of Chambolle and Pock is then used for both,
the minimisation with respect to the displacement �eld and the motion segmentation.
All the proposed non-rigid discontinuity preserving registration
methods delivered promising results for experiments with synthetic images
and real MR images of breathing induced liver motion
Robust and Optimal Methods for Geometric Sensor Data Alignment
Geometric sensor data alignment - the problem of finding the
rigid transformation that correctly aligns two sets of sensor
data without prior knowledge of how the data correspond - is a
fundamental task in computer vision and robotics. It is
inconvenient then that outliers and non-convexity are inherent to
the problem and present significant challenges for alignment
algorithms. Outliers are highly prevalent in sets of sensor data,
particularly when the sets overlap incompletely. Despite this,
many alignment objective functions are not robust to outliers,
leading to erroneous alignments. In addition, alignment problems
are highly non-convex, a property arising from the objective
function and the transformation. While finding a local optimum
may not be difficult, finding the global optimum is a hard
optimisation problem. These key challenges have not been fully
and jointly resolved in the existing literature, and so there is
a need for robust and optimal solutions to alignment problems.
Hence the objective of this thesis is to develop tractable
algorithms for geometric sensor data alignment that are robust to
outliers and not susceptible to spurious local optima.
This thesis makes several significant contributions to the
geometric alignment literature, founded on new insights into
robust alignment and the geometry of transformations. Firstly, a
novel discriminative sensor data representation is proposed that
has better viewpoint invariance than generative models and is
time and memory efficient without sacrificing model fidelity.
Secondly, a novel local optimisation algorithm is developed for
nD-nD geometric alignment under a robust distance measure. It
manifests a wider region of convergence and a greater robustness
to outliers and sampling artefacts than other local optimisation
algorithms. Thirdly, the first optimal solution for 3D-3D
geometric alignment with an inherently robust objective function
is proposed. It outperforms other geometric alignment algorithms
on challenging datasets due to its guaranteed optimality and
outlier robustness, and has an efficient parallel implementation.
Fourthly, the first optimal solution for 2D-3D geometric
alignment with an inherently robust objective function is
proposed. It outperforms existing approaches on challenging
datasets, reliably finding the global optimum, and has an
efficient parallel implementation. Finally, another optimal
solution is developed for 2D-3D geometric alignment, using a
robust surface alignment measure.
Ultimately, robust and optimal methods, such as those in this
thesis, are necessary to reliably find accurate solutions to
geometric sensor data alignment problems
GOGMA: Globally-Optimal Gaussian Mixture Alignment
Gaussian mixture alignment is a family of approaches that are frequently used
for robustly solving the point-set registration problem. However, since they
use local optimisation, they are susceptible to local minima and can only
guarantee local optimality. Consequently, their accuracy is strongly dependent
on the quality of the initialisation. This paper presents the first
globally-optimal solution to the 3D rigid Gaussian mixture alignment problem
under the L2 distance between mixtures. The algorithm, named GOGMA, employs a
branch-and-bound approach to search the space of 3D rigid motions SE(3),
guaranteeing global optimality regardless of the initialisation. The geometry
of SE(3) was used to find novel upper and lower bounds for the objective
function and local optimisation was integrated into the scheme to accelerate
convergence without voiding the optimality guarantee. The evaluation
empirically supported the optimality proof and showed that the method performed
much more robustly on two challenging datasets than an existing
globally-optimal registration solution.Comment: Manuscript in press 2016 IEEE Conference on Computer Vision and
Pattern Recognitio
Retrieval and Registration of Long-Range Overlapping Frames for Scalable Mosaicking of In Vivo Fetoscopy
Purpose: The standard clinical treatment of Twin-to-Twin Transfusion Syndrome
consists in the photo-coagulation of undesired anastomoses located on the
placenta which are responsible to a blood transfer between the two twins. While
being the standard of care procedure, fetoscopy suffers from a limited
field-of-view of the placenta resulting in missed anastomoses. To facilitate
the task of the clinician, building a global map of the placenta providing a
larger overview of the vascular network is highly desired. Methods: To overcome
the challenging visual conditions inherent to in vivo sequences (low contrast,
obstructions or presence of artifacts, among others), we propose the following
contributions: (i) robust pairwise registration is achieved by aligning the
orientation of the image gradients, and (ii) difficulties regarding long-range
consistency (e.g. due to the presence of outliers) is tackled via a bag-of-word
strategy, which identifies overlapping frames of the sequence to be registered
regardless of their respective location in time. Results: In addition to visual
difficulties, in vivo sequences are characterised by the intrinsic absence of
gold standard. We present mosaics motivating qualitatively our methodological
choices and demonstrating their promising aspect. We also demonstrate
semi-quantitatively, via visual inspection of registration results, the
efficacy of our registration approach in comparison to two standard baselines.
Conclusion: This paper proposes the first approach for the construction of
mosaics of placenta in in vivo fetoscopy sequences. Robustness to visual
challenges during registration and long-range temporal consistency are
proposed, offering first positive results on in vivo data for which standard
mosaicking techniques are not applicable.Comment: Accepted for publication in International Journal of Computer
Assisted Radiology and Surgery (IJCARS
Review of the mathematical foundations of data fusion techniques in surface metrology
The recent proliferation of engineered surfaces, including freeform and structured surfaces, is challenging current metrology techniques. Measurement using multiple sensors has been proposed to achieve enhanced benefits, mainly in terms of spatial frequency bandwidth, which a single sensor cannot provide. When using data from different sensors, a process of data fusion is required and there is much active research in this area. In this paper, current data fusion methods and applications are reviewed, with a focus on the mathematical foundations of the subject. Common research questions in the fusion of surface metrology data are raised and potential fusion algorithms are discussed
- …