Real-time 3D reconstruction of non-rigid shapes with a single moving camera

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small mapsPeer ReviewedPostprint (author's final draft

Segmentation of RT3D Ultrasound with Implicit Deformable Models Without Gradients

This paper presents the implementation and validation of a new 3D deformable model method, based on the Mumford-Shah functional for segmentation of three-dimensional real-time ultrasound. An experiment on 10 patients with primary hypertension was carried out to compare three segmentation methods for quantification of right and left ventricular ejection fraction: (1) manual tracing by an expert cardiologist, (2) 2D parametric deformable model, and (3) 3D implicit deformable model implemented with a level set framework. Deformable model segmentations were performed on denoised data using a (3D+Time) brushlet expansion. The clinical study showed superior performance of the deformable model in assessing ejection fraction when compared to MRI measures. It also showed that the three-dimensional deformable model improved EF measures, which is explained by a more accurate segmentation of small and convoluted ventricular shapes when integrating the third spatial dimension

Doctor of Philosophy

dissertationImage-based biomechanics, particularly numerical modeling using subject-specific data obtained via imaging, has proven useful for elucidating several biomechanical processes, such as prediction of deformation due to external loads, applicable to both normal function and pathophysiology of various organs. As the field evolves towards applications that stretch the limits of imaging hardware and acquisition time, the information traditionally expected as input for numerical routines often becomes incomplete or ambiguous, and requires specific acquisition and processing strategies to ensure physical accuracy and compatibility with predictive mathematical modeling. These strategies, often derivatives or specializations of traditional mechanics, effectively extend the nominal capability of medical imaging hardware providing subject-specific information coupled with the option of using the results for predictive numerical simulations. This research deals with the development of tools for extracting mechanical measurements from a finite set of imaging data and finite element analysis in the context of constructing structural atlases of the heart, understanding the biomechanics of the venous vasculature, and right ventricular failure. The tools include: (1) application of Hyperelastic Warping image registration to displacement-encoded MRI for reconstructing absolute displacement fields, (2) combination of imaging and a material parameter identification approach to measure morphology, deformation, and mechanical properties of vascular tissue, and (3) extrapolation of diffusion tensor MRI acquired at a single time point for the prediction the structural changes across the cardiac cycle with mechanical simulations. Selected tools were then applied to evaluate structural changes in a reversible animal model for right ventricular failure due to pressure overload

Fluid-driven deformation of a soft granular material

Compressing a porous, fluid-filled material will drive the interstitial fluid out of the pore space, as when squeezing water out of a kitchen sponge. Inversely, injecting fluid into a porous material can deform the solid structure, as when fracturing a shale for natural gas recovery. These poromechanical interactions play an important role in geological and biological systems across a wide range of scales, from the propagation of magma through the Earth's mantle to the transport of fluid through living cells and tissues. The theory of poroelasticity has been largely successful in modeling poromechanical behavior in relatively simple systems, but this continuum theory is fundamentally limited by our understanding of the pore-scale interactions between the fluid and the solid, and these problems are notoriously difficult to study in a laboratory setting. Here, we present a high-resolution measurement of injection-driven poromechanical deformation in a system with granular microsctructure: We inject fluid into a dense, confined monolayer of soft particles and use particle tracking to reveal the dynamics of the multi-scale deformation field. We find that a continuum model based on poroelasticity theory captures certain macroscopic features of the deformation, but the particle-scale deformation field exhibits dramatic departures from smooth, continuum behavior. We observe particle-scale rearrangement and hysteresis, as well as petal-like mesoscale structures that are connected to material failure through spiral shear banding