Sequential non-rigid structure from motion using physical priors

© 20xx IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.We propose a new approach to simultaneously recover camera pose and 3D shape of non-rigid and potentially extensible surfaces from a monocular image sequence. For this purpose, we make use of the Extended Kalman Filter based Simultaneous Localization And Mapping (EKF-SLAM) formulation, a Bayesian optimization framework traditionally used in mobile robotics for estimating camera pose and reconstructing rigid scenarios. In order to extend the problem to a deformable domain we represent the object's surface mechanics by means of Navier's equations, which are solved using a Finite Element Method (FEM). With these main ingredients, we can further model the material's stretching, allowing us to go a step further than most of current techniques, typically constrained to surfaces undergoing isometric deformations. We extensively validate our approach in both real and synthetic experiments, and demonstrate its advantages with respect to competing methods. More specifically, we show that besides simultaneously retrieving camera pose and non-rigid shape, our approach is adequate for both isometric and extensible surfaces, does not require neither batch processing all the frames nor tracking points over the whole sequence and runs at several frames per second.Peer ReviewedPostprint (author's final draft

Assembling models of embryo development: Image analysis and the construction of digital atlases

Digital atlases of animal development provide a quantitative description of morphogenesis, opening the path toward processes modeling. Prototypic atlases offer a data integration framework where to gather information from cohorts of individuals with phenotypic variability. Relevant information for further theoretical reconstruction includes measurements in time and space for cell behaviors and gene expression. The latter as well as data integration in a prototypic model, rely on image processing strategies. Developing the tools to integrate and analyze biological multidimensional data are highly relevant for assessing chemical toxicity or performing drugs preclinical testing. This article surveys some of the most prominent efforts to assemble these prototypes, categorizes them according to salient criteria and discusses the key questions in the field and the future challenges toward the reconstruction of multiscale dynamics in model organisms

Deformable surface registration for breast tumors tracking: A phantom study

A phantom study for breast tumor registration based on the deformation of the external surface is proposed. This study aims at the integration into an image guided system for breast cancer biopsy or ablation. To compensate potentially large breast displacements, due to different positions of the breast during biopsy or ablation compared with preoperative data, where the diagnosis was made, an initial linear alignment using visible landmarks is involved, followed by thin-plate spline (TPS) registration of the linearly aligned surfaces. Subsequently, the TPS deformation will be applied to the tumors. The results were validated using a multi modal phantom of the breast, while the tumors and the surface were segmented on four different positions of the phantom: prone, supine, vertical and on a side. The use of computed tomography (CT) dataset allowed us to obtain a very precise segmentation of the external surface, of the tumors and the landmarks. Despite large variation among the different positions of the phantom due to the gravitational force, the accuracy of the method at the target point was under 5 millimeters. These results allow us to conclude that, using our prototype image registration system, we are able to align acquisition of the breast in different positions with clinically relevant accuracy

Retrieval and Registration of Long-Range Overlapping Frames for Scalable Mosaicking of In Vivo Fetoscopy

Purpose: The standard clinical treatment of Twin-to-Twin Transfusion Syndrome consists in the photo-coagulation of undesired anastomoses located on the placenta which are responsible to a blood transfer between the two twins. While being the standard of care procedure, fetoscopy suffers from a limited field-of-view of the placenta resulting in missed anastomoses. To facilitate the task of the clinician, building a global map of the placenta providing a larger overview of the vascular network is highly desired. Methods: To overcome the challenging visual conditions inherent to in vivo sequences (low contrast, obstructions or presence of artifacts, among others), we propose the following contributions: (i) robust pairwise registration is achieved by aligning the orientation of the image gradients, and (ii) difficulties regarding long-range consistency (e.g. due to the presence of outliers) is tackled via a bag-of-word strategy, which identifies overlapping frames of the sequence to be registered regardless of their respective location in time. Results: In addition to visual difficulties, in vivo sequences are characterised by the intrinsic absence of gold standard. We present mosaics motivating qualitatively our methodological choices and demonstrating their promising aspect. We also demonstrate semi-quantitatively, via visual inspection of registration results, the efficacy of our registration approach in comparison to two standard baselines. Conclusion: This paper proposes the first approach for the construction of mosaics of placenta in in vivo fetoscopy sequences. Robustness to visual challenges during registration and long-range temporal consistency are proposed, offering first positive results on in vivo data for which standard mosaicking techniques are not applicable.Comment: Accepted for publication in International Journal of Computer Assisted Radiology and Surgery (IJCARS

Evaluating and Improving Cochlear Length Measurements on Clinical Computed Tomography Images

Cochlear implants provide the sensation of sound to deaf individuals. An accurate estimate of cochlear duct length (CDL) is required for pre-operative implant electrode selection and can be obtained from clinical computed tomography (CT) by measuring the “A-value”. The objectives of this work were to estimate the accuracy and variability in manual A-value measurements, and to automate measurements. Four specialists repeatedly measured the A-value on clinical CT images from which the inter- and intra-observer variability were calculated. Accuracy was assessed by comparison to measurements on higher resolution micro-CT images. Motivated by this study, software was developed to automate the A-value measurement by registering an annotated atlas to unlabelled images. There was significant variability in manual A-value measurements made using either standard clinical or multi-planar reformatted views with the latter exhibiting higher variability but better accuracy. The automated approach eliminated variability and improved accuracy, enabling the correct selection of electrode length

Modified mass-spring system for physically based deformation modeling

Mass-spring systems are considered the simplest and most intuitive of all deformable models. They are computationally efficient, and can handle large deformations with ease. But they suffer several intrinsic limitations. In this book a modified mass-spring system for physically based deformation modeling that addresses the limitations and solves them elegantly is presented. Several implementations in modeling breast mechanics, heart mechanics and for elastic images registration are presented

Practicable assessment of cochlear size and shape from clinical CT images

There is considerable interpersonal variation in the size and shape of the human cochlea, with evident consequences for cochlear implantation. The ability to characterize a specific cochlea, from preoperative computed tomography (CT) images, would allow the clinician to personalize the choice of electrode, surgical approach and postoperative programming. In this study, we present a fast, practicable and freely available method for estimating cochlear size and shape from clinical CT. The approach taken is to fit a template surface to the CT data, using either a statistical shape model or a locally affine deformation (LAD). After fitting, we measure cochlear size, duct length and a novel measure of basal turn non-planarity, which we suggest might correlate with the risk of insertion trauma. Gold-standard measurements from a convenience sample of 18 micro-CT scans are compared with the same quantities estimated from low-resolution, noisy, pseudo-clinical data synthesized from the same micro-CT scans. The best results were obtained using the LAD method, with an expected error of 8-17% of the gold-standard sample range for non-planarity, cochlear size and duct length.Evelyn Trust, MRC Confidence in Concept Fund Cambridge Hearing Trust

Modified mass-spring system for physically based deformation modeling

Mass-spring systems are considered the simplest and most intuitive of all deformable models. They are computationally efficient, and can handle large deformations with ease. But they suffer several intrinsic limitations. In this book a modified mass-spring system for physically based deformation modeling that addresses the limitations and solves them elegantly is presented. Several implementations in modeling breast mechanics, heart mechanics and for elastic images registration are presented

Doctor of Philosophy

dissertationIn this thesis I present novel findings of microstructural remodeling that occurs during dyssynchronous heart failure (DHF) and the ability for cardiac resynchronization therapy (CRT) to reverse this remodeling. DHF is an advanced disease state that occurs in a large portion of patients suffering from heart failure. Mechanical dyssynchrony between the left and right ventricles of the heart, the hallmark of DHF, results in significantly increased heterogeneity of stress in the cardiac wall. DHF severely limits cardiac performance, decreasing quality of life and increasing mortality. The main therapy for treating DHF is CRT, a therapy in which mechanical synchrony is restored to the ventricles via electrical pacing. The success of CRT varies widely. Scientific knowledge surrounding DHF and CRT is surprisingly sparse for how widespread the disease and therapy are. A better understanding of the subcellular structure and function altered during DHF will improve our understanding of the disease and potentially help develop novel therapies and even lead to development of assays capable of better predicting success of current therapies. Here we use confocal microscopy to explore protein distributions within isolated cardiomyocytes and intact tissue, Ca2+ handling during activation and relaxation of stimulated cardiomyocytes, and to develop a method for quantifying strain in 2D image sequences of contracting cardiomyocytes at an unprecedented spatiotemporal resolution. Specifically I will demonstrate that ?-actinin, the protein comprising the majority of the sarcomeric Z-disk, is significantly altered during DHF and that CRT is able to partially reverse this remodeling. I will then present findings on remodeling of the transverse tubular system and associated ryanodine receptor clusters, both crucial components of excitation-contraction coupling. In particular, I will show that these structures exhibit subcellular heterogeneity during DHF, affecting excitation-contraction coupling. This heterogeneity is reduced after CRT, indicating previously unknown capabilities of restoration. Finally, I will present a novel method to characterize strain within contracting cardiomyocytes. This method expands on previous methods by providing a regional 2D strain tensor at unprecedented spatiotemporal resolution, allowing more accurate description of the mechanical properties of the cell. Together, this work makes a significant contribution to the understanding of DHF and CRT