Retrieval and Registration of Long-Range Overlapping Frames for Scalable Mosaicking of In Vivo Fetoscopy

Purpose: The standard clinical treatment of Twin-to-Twin Transfusion Syndrome consists in the photo-coagulation of undesired anastomoses located on the placenta which are responsible to a blood transfer between the two twins. While being the standard of care procedure, fetoscopy suffers from a limited field-of-view of the placenta resulting in missed anastomoses. To facilitate the task of the clinician, building a global map of the placenta providing a larger overview of the vascular network is highly desired. Methods: To overcome the challenging visual conditions inherent to in vivo sequences (low contrast, obstructions or presence of artifacts, among others), we propose the following contributions: (i) robust pairwise registration is achieved by aligning the orientation of the image gradients, and (ii) difficulties regarding long-range consistency (e.g. due to the presence of outliers) is tackled via a bag-of-word strategy, which identifies overlapping frames of the sequence to be registered regardless of their respective location in time. Results: In addition to visual difficulties, in vivo sequences are characterised by the intrinsic absence of gold standard. We present mosaics motivating qualitatively our methodological choices and demonstrating their promising aspect. We also demonstrate semi-quantitatively, via visual inspection of registration results, the efficacy of our registration approach in comparison to two standard baselines. Conclusion: This paper proposes the first approach for the construction of mosaics of placenta in in vivo fetoscopy sequences. Robustness to visual challenges during registration and long-range temporal consistency are proposed, offering first positive results on in vivo data for which standard mosaicking techniques are not applicable.Comment: Accepted for publication in International Journal of Computer Assisted Radiology and Surgery (IJCARS

Computerized Analysis of Magnetic Resonance Images to Study Cerebral Anatomy in Developing Neonates

The study of cerebral anatomy in developing neonates is of great importance for the understanding of brain development during the early period of life. This dissertation therefore focuses on three challenges in the modelling of cerebral anatomy in neonates during brain development. The methods that have been developed all use Magnetic Resonance Images (MRI) as source data. To facilitate study of vascular development in the neonatal period, a set of image analysis algorithms are developed to automatically extract and model cerebral vessel trees. The whole process consists of cerebral vessel tracking from automatically placed seed points, vessel tree generation, and vasculature registration and matching. These algorithms have been tested on clinical Time-of- Flight (TOF) MR angiographic datasets. To facilitate study of the neonatal cortex a complete cerebral cortex segmentation and reconstruction pipeline has been developed. Segmentation of the neonatal cortex is not effectively done by existing algorithms designed for the adult brain because the contrast between grey and white matter is reversed. This causes pixels containing tissue mixtures to be incorrectly labelled by conventional methods. The neonatal cortical segmentation method that has been developed is based on a novel expectation-maximization (EM) method with explicit correction for mislabelled partial volume voxels. Based on the resulting cortical segmentation, an implicit surface evolution technique is adopted for the reconstruction of the cortex in neonates. The performance of the method is investigated by performing a detailed landmark study. To facilitate study of cortical development, a cortical surface registration algorithm for aligning the cortical surface is developed. The method first inflates extracted cortical surfaces and then performs a non-rigid surface registration using free-form deformations (FFDs) to remove residual alignment. Validation experiments using data labelled by an expert observer demonstrate that the method can capture local changes and follow the growth of specific sulcus

Registration using Graphics Processor Unit

Data point set registration is an important operation in coordinate metrology. Registration is the operation by which sampled point clouds are aligned with a CAD model by a 4X4 homogeneous transformation (e.g., rotation and translation). This alignment permits validation of the produced artifact\u27s geometry. State-of-the-art metrology systems are now capable of generating thousands, if not millions, of data points during an inspection operation, resulting in increased computational power to fully utilize these larger data sets. The registration process is an iterative nonlinear optimization operation having an execution time directly related to the number of points processed and CAD model complexity. The objective function to be minimized by this optimization is the sum of the square distances between each point in the point cloud and the closest surface in the CAD model. A brute force approach to registration, which is often used, is to compute the minimum distance between each point and each surface in the CAD model. As point cloud sizes and CAD model complexity increase, this approach becomes intractable and inefficient. Highly efficient numerical and analytical gradient based algorithms exist and their goal is to convergence to an optimal solution in minimum time. This thesis presents a new approach to efficiently perform the registration process by employing readily available computer hardware, the graphical processor unit (GPU). The data point set registration time for the GPU shows a significant improvement (around 15-20 times) over typical CPU performance. Efficient GPU programming decreases the complexity of the steps and improves the rate of convergence of the existing algorithms. The experimental setup reveals the exponential increasing nature of the CPU and the linear performance of the GPU in various aspects of an algorithm. The importance of CPU in the GPU programming is highlighted. The future implementations disclose the possible extensions of a GPU for higher order and complex coordinate metrology algorithms

Development and characterization of methodology and technology for the alignment of fMRI time series

This dissertation has developed, implemented and tested a novel computer based system (AUTOALIGN) that incorporates an algorithm for the alignment of functional Magnetic Resonance Image (fMRI) time series. The algorithm assumes the human brain to be a rigid body and computes a head coordinate system on the basis of three reference points that lie on the directions correspondent to two of the eigenvectors of inertia of the volume, at the intersections with the head boundary. The eigenvectors are found weighting the inertia components with the voxel\u27s intensity values assumed as mass. The three reference points are found in the same position, relative to the origin of the head coordinate system, in both test and reference brain images. Intensity correction is performed at sub-voxel accuracy by tri-linear interpolation. A test fMR brain volume in which controlled simulations of rigid-body transformations have been introduced has preliminarily assessed system performance. Further experimentation has been conducted with real fMRI time series. Rigid-body transformations have been retrieved automatically and the values of the motion parameters compared to those obtained by the Statistical Parametric Mapping (SPM99), and the Automatic Image Registration (AIR 3.08). Results indicated that AUTOALIGN offers subvoxel accuracy in correcting both misalignment and intensity among time points in fMR images time series, and also that its performance is comparable to that of SPM99 and AIR3.08

Diffeomorphic image registration with applications to deformation modelling between multiple data sets

Over last years, the diffeomorphic image registration algorithms have been successfully introduced into the field of the medical image analysis. At the same time, the particular usability of these techniques, in majority derived from the solid mathematical background, has been only quantitatively explored for the limited applications such as longitudinal studies on treatment quality, or diseases progression. The thesis considers the deformable image registration algorithms, seeking out those that maintain the medical correctness of the estimated dense deformation fields in terms of the preservation of the object and its neighbourhood topology, offer the reasonable computational complexity to satisfy time restrictions coming from the potential applications, and are able to cope with low quality data typically encountered in Adaptive Radiotherapy (ART). The research has led to the main emphasis being laid on the diffeomorphic image registration to achieve one-to-one mapping between images. This involves introduction of the log-domain parameterisation of the deformation field by its approximation via a stationary velocity field. A quantitative and qualitative examination of existing and newly proposed algorithms for pairwise deformable image registration presented in this thesis, shows that the log-Euclidean parameterisation can be successfully utilised in the biomedical applications. Although algorithms utilising the log-domain parameterisation have theoretical justification for maintaining diffeomorphism, in general, the deformation fields produced by them have similar properties as these estimated by classical methods. Having this in mind, the best compromise in terms of the quality of the deformation fields has been found for the consistent image registration framework. The experimental results suggest also that the image registration with the symmetrical warping of the input images outperforms the classical approaches, and simultaneously can be easily introduced to most known algorithms. Furthermore, the log-domain implicit group-wise image registration is proposed. By linking the various sets of images related to the different subjects, the proposed image registration approach establishes a common subject space and between-subject correspondences therein. Although the correspondences between groups of images can be found by performing the classic image registration, the reference image selection (not required in the proposed implementation), may lead to a biased mean image being estimated and the corresponding common subject space not adequate to represent the general properties of the data sets. The approaches to diffeomorphic image registration have been also utilised as the principal elements for estimating the movements of the organs in the pelvic area based on the dense deformation field prediction system driven by the partial information coming from the specific type of the measurements parameterised using the implicit surface representation, and recognising facial expressions where the stationary velocity fields are used as the facial expression descriptors. Both applications have been extensively evaluated based on the real representative data sets of three-dimensional volumes and two-dimensional images, and the obtained results indicate the practical usability of the proposed techniques

Automation of the Registration of Range Plant Images Using Geomagic Studio

As plant science advances, the quantitative measurement of 3D plant growth has become an essential tool to evaluate the growth performance of genetically modified seeds or plants. In the past, “wet volumes” had been used to quantitatively measure plant growth which result in the destruction of the plants. Other traditional methods like interferometry methods or mechanical methods are also inefficient because they are either invasive or unable to provide sufficient growth information of the plant body. On the other hand, computer vision methods, especially optical and range flow techniques, are non-contact, more accurate, and relatively less expensive. But they require a fixed light source for the experimental environment and also, higher acquisition rate for the data. In this thesis, unlike optical or range flow methods, we generate a number of 3D range scans of the Arabidopsis thaliana plant from different viewpoints to reconstruct 3D polygonal meshes of it. We use ShapeGrabber range sensor to acquire the scanned data and Geomagic Studio 12 CAD software to register these scans to produce 3D polygonal meshes. Then, the canopy surface area and 3D stem volume of the plant are computed from these meshes to determine its growth over a time cycle. To perform the registration process in Geomagic, 6 ping pong balls are used as reference spheres.However, because the laser scanner can only see the parts of an object that are visible from its viewpoint, the balls are imaged as semi-spheres in the original range images.But, Geomagic is unable to register multiple images with incomplete semi-sphere data and requires full spheres as target objects. Zhao (MSc UWO 2010) and Yang (MSc UWO 2009) manually replaced these semi-spheres with artificially generated spheres using Geomagic which is both tedious and error prone. Moreover, they manually rotated the images to find out the pair of spheres that matches in adjacent range scans and aligned them manually before performing the registration. Because of the manual nature of this processing, automation of the registration process is very desirable. Among the major contributions of this research are not only to automatically generate synthetic sphere data but also detect and localize the semi-spheres in the original range images using parameter estimation techniques and parametric equation of spheres. Additionally, we reconstruct the original range images by replacing the incomplete semi-sphere data with “perfect” full sphere data to aid Geomagic during the automatic detection of target objects. After this pre-processing, the modified images are automatically registered using Geomagic macros so that the 3D polygonal meshes found from the registration process can be used to measure plant growth quantitatively. We believe that the automation of the registration process is a good first step towards a fully automated system of 3D plant growth measurement

Assisting digital volume correlation with mechanical image-based modeling: application to the measurement of kinematic fields at the architecture scale in cellular materials

La mesure de champs de déplacement et de déformation aux petites échelles dans des microstructures complexes représente encore un défi majeur dans le monde de la mécanique expérimentale. Ceci est en partie dû aux acquisitions d'images et à la pauvreté de la texture à ces échelles. C'est notamment le cas pour les matériaux cellulaires lorsqu'ils sont imagés avec des micro-tomographes conventionnels et qu'ils peuvent être sujets à des mécanismes de déformation complexes. Comme la validation de modèles numériques et l'identification des propriétés mécaniques de matériaux se base sur des mesures précises de déplacements et de déformations, la conception et l'implémentation d'algorithmes robustes et fiables de corrélation d'images semble nécessaire. Lorsque l'on s'intéresse à l'utilisation de la corrélation d'images volumiques (DVC) pour les matériaux cellulaires, on est confronté à un paradoxe: l'absence de texture à l'échelle du constituant conduit à considérer l'architecture comme marqueur pour la corrélation. Ceci conduit à l'échec des techniques ordinaires de DVC à mesurer des cinématiques aux échelles subcellulaires en lien avec des comportements mécaniques locaux complexes tels que la flexion ou le flambement de travées. L'objectif de cette thèse est la conception d'une technique de DVC pour la mesure de champs de déplacement dans des matériaux cellulaires à l'échelle de leurs architectures. Cette technique assiste la corrélation d'images par une régularisation élastique faible en utilisant un modèle mécanique généré automatiquement et basé sur les images. La méthode suggérée introduit une séparation d'échelles au dessus desquelles la DVC est dominante et en dessous desquelles elle est assistée par le modèle mécanique basé sur l'image. Une première étude numérique consistant à comparer différentes techniques de construction de modèles mécaniques basés sur les images est conduite. L'accent est mis sur deux méthodes de calcul particulières: la méthode des éléments finis (FEM) et la méthode des cellules finies (FCM) qui consiste à immerger la géométrie complexe dans une grille régulière de haut ordre sans utiliser de mailleurs. Si la FCM évite une première phase délicate de discrétisation, plusieurs paramètres restent néanmoins délicats à fixer. Dans ce travail, ces paramètres sont ajustés afin d'obtenir (a) la meilleure précision (bornée par les erreurs de pixellisation) tout en (b) assurant une complexité minimale. Pour l'aspect mesure par corrélation d'images régularisée, plusieurs expérimentations virtuelles à partir de différentes simulations numériques (en élasticité, en plasticité et en non-linéarité géométrique) sont d'abord réalisées afin d'analyser l'influence des paramètres de régularisation introduits. Les erreurs de mesures peuvent dans ce cas être quantifiées à l'aide des solutions de référence éléments finis. La capacité de la méthode à mesurer des cinématiques complexes en absence de texture est démontrée pour des régimes non-linéaires tels que le flambement. Finalement, le travail proposé est généralisé à la corrélation volumique des différents états de déformation du matériau et à la construction automatique de la micro-architecture cellulaire en utilisant soit une grille B-spline d'ordre arbitraire (FCM) soit un maillage éléments finis (FEM). Une mise en évidence expérimentale de l'efficacité et de la justesse de l'approche proposée est effectuée à travers de la mesure de cinématiques complexes dans une mousse polyuréthane sollicitée en compression lors d'un essai in situ.Measuring displacement and strain fields at low observable scales in complex microstructures still remains a challenge in experimental mechanics often because of the combination of low definition images with poor texture at this scale. The problem is particularly acute in the case of cellular materials, when imaged by conventional micro-tomographs, for which complex highly non-linear local phenomena can occur. As the validation of numerical models and the identification of mechanical properties of materials must rely on accurate measurements of displacement and strain fields, the design and implementation of robust and faithful image correlation algorithms must be conducted. With cellular materials, the use of digital volume correlation (DVC) faces a paradox: in the absence of markings of exploitable texture on/or in the struts or cell walls, the available speckle will be formed by the material architecture itself. This leads to the inability of classical DVC codes to measure kinematics at the cellular and a fortiori sub-cellular scales, precisely because the interpolation basis of the displacement field cannot account for the complexity of the underlying kinematics, especially when bending or buckling of beams or walls occurs. The objective of the thesis is to develop a DVC technique for the measurement of displacement fields in cellular materials at the scale of their architecture. The proposed solution consists in assisting DVC by a weak elastic regularization using an automatic image-based mechanical model. The proposed method introduces a separation of scales above which DVC is dominant and below which it is assisted by image-based modeling. First, a numerical investigation and comparison of different techniques for building automatically a geometric and mechanical model from tomographic images is conducted. Two particular methods are considered: the finite element method (FEM) and the finite-cell method (FCM). The FCM is a fictitious domain method that consists in immersing the complex geometry in a high order structured grid and does not require meshing. In this context, various discretization parameters appear delicate to choose. In this work, these parameters are adjusted to obtain (a) the best possible accuracy (bounded by pixelation errors) while (b) ensuring minimal complexity. Concerning the ability of the mechanical image-based models to regularize DIC, several virtual experimentations are performed in two-dimensions in order to finely analyze the influence of the introduced regularization lengths for different input mechanical behaviors (elastic, elasto-plastic and geometrically non-linear) and in comparison with ground truth. We show that the method can estimate complex local displacement and strain fields with speckle-free low definition images, even in non-linear regimes such as local buckling. Finally a three-dimensional generalization is performed through the development of a DVC framework. It takes as an input the reconstructed volumes at the different deformation states of the material and constructs automatically the cellular micro-architeture geometry. It considers either an immersed structured B-spline grid of arbitrary order or a finite-element mesh. An experimental evidence is performed by measuring the complex kinematics of a polyurethane foam under compression during an in situ test

A fast and robust patient specific Finite Element mesh registration technique: application to 60 clinical cases

Finite Element mesh generation remains an important issue for patient specific biomechanical modeling. While some techniques make automatic mesh generation possible, in most cases, manual mesh generation is preferred for better control over the sub-domain representation, element type, layout and refinement that it provides. Yet, this option is time consuming and not suited for intraoperative situations where model generation and computation time is critical. To overcome this problem we propose a fast and automatic mesh generation technique based on the elastic registration of a generic mesh to the specific target organ in conjunction with element regularity and quality correction. This Mesh-Match-and-Repair (MMRep) approach combines control over the mesh structure along with fast and robust meshing capabilities, even in situations where only partial organ geometry is available. The technique was successfully tested on a database of 5 pre-operatively acquired complete femora CT scans, 5 femoral heads partially digitized at intraoperative stage, and 50 CT volumes of patients' heads. The MMRep algorithm succeeded in all 60 cases, yielding for each patient a hex-dominant, Atlas based, Finite Element mesh with submillimetric surface representation accuracy, directly exploitable within a commercial FE software

Computer-Aided Detection and diagnosis for prostate cancer based on mono and multi-parametric MRI: A review

International audienceProstate cancer is the second most diagnosed cancer of men all over the world. In the last decades, new imaging techniques based on Magnetic Resonance Imaging (MRI) have been developed improving diagnosis.In practise, diagnosis can be affected by multiple factors such as observer variability and visibility and complexity of the lesions. In this regard, computer-aided detection and computer-aided diagnosis systemshave been designed to help radiologists in their clinical practice. Research on computer-aided systems specifically focused for prostate cancer is a young technology and has been part of a dynamic field ofresearch for the last ten years. This survey aims to provide a comprehensive review of the state of the art in this lapse of time, focusing on the different stages composing the work-flow of a computer-aidedsystem. We also provide a comparison between studies and a discussion about the potential avenues for future research. In addition, this paper presents a new public online dataset which is made available to theresearch community with the aim of providing a common evaluation framework to overcome some of the current limitations identified in this survey