Finite Element Based Tracking of Deforming Surfaces

We present an approach to robustly track the geometry of an object that deforms over time from a set of input point clouds captured from a single viewpoint. The deformations we consider are caused by applying forces to known locations on the object's surface. Our method combines the use of prior information on the geometry of the object modeled by a smooth template and the use of a linear finite element method to predict the deformation. This allows the accurate reconstruction of both the observed and the unobserved sides of the object. We present tracking results for noisy low-quality point clouds acquired by either a stereo camera or a depth camera, and simulations with point clouds corrupted by different error terms. We show that our method is also applicable to large non-linear deformations.Comment: additional experiment

Optimized imaging using non-rigid registration

The extraordinary improvements of modern imaging devices offer access to data with unprecedented information content. However, widely used image processing methodologies fall far short of exploiting the full breadth of information offered by numerous types of scanning probe, optical, and electron microscopies. In many applications, it is necessary to keep measurement intensities below a desired threshold. We propose a methodology for extracting an increased level of information by processing a series of data sets suffering, in particular, from high degree of spatial uncertainty caused by complex multiscale motion during the acquisition process. An important role is played by a nonrigid pixel-wise registration method that can cope with low signal-to-noise ratios. This is accompanied by formulating objective quality measures which replace human intervention and visual inspection in the processing chain. Scanning transmission electron microscopy of siliceous zeolite material exhibits the above-mentioned obstructions and therefore serves as orientation and a test of our procedures

MORPH-DSLAM: Model Order Reduction for PHysics-based Deformable SLAM

We propose a new methodology to estimate the 3D displacement field of deformable objects from video sequences using standard monocular cameras. We solve in real time the complete (possibly visco-)hyperelasticity problem to properly describe the strain and stress fields that are consistent with the displacements captured by the images, constrained by real physics. We do not impose any ad-hoc prior or energy minimization in the external surface, since the real and complete mechanics problem is solved. This means that we can also estimate the internal state of the objects, even in occluded areas, just by observing the external surface and the knowledge of material properties and geometry. Solving this problem in real time using a realistic constitutive law, usually non-linear, is out of reach for current systems. To overcome this difficulty, we solve off-line a parametrized problem that considers each source of variability in the problem as a new parameter and, consequently, as a new dimension in the formulation. Model Order Reduction methods allow us to reduce the dimensionality of the problem, and therefore, its computational cost, while preserving the visualization of the solution in the high-dimensionality space. This allows an accurate estimation of the object deformations, improving also the robustness in the 3D points estimation

Single View Augmentation of 3D Elastic Objects

International audienceThis paper proposes an efficient method to capture and augment highly elastic objects from a single view. 3D shape recovery from a monocular video sequence is an underconstrained problem and many approaches have been proposed to enforce constraints and resolve the ambiguities. State-of-the art solutions enforce smoothness or geometric constraints, consider specific deformation properties such as inextensibility or ressort to shading constraints. However, few of them can handle properly large elastic deformations. We propose in this paper a real-time method which makes use of a me chanical model and is able to handle highly elastic objects. Our method is formulated as a energy minimization problem accounting for a non-linear elastic model constrained by external image points acquired from a monocular camera. This method prevents us from formulating restrictive assumptions and specific constraint terms in the minimization. The only parameter involved in the method is the Young's modulus where we show in experiments that a rough estimate of its value is sufficient to obtain a good reconstruction. Our method is compared to existing techniques with experiments conducted on computer-generated and real data that show the effectiveness of our approach. Experiments in the context of minimally invasive liver surgery are also provided

Finite element modeling of soft tissue deformation.

Computer-aided minimally invasive surgery (MIS) has progressed significantly in the last decade and it has great potential in surgical planning and operations. To limit the damage to nearby healthy tissue, accurate modeling is required of the mechanical behavior of a target soft tissue subject to surgical manipulations. Therefore, the study of soft tissue deformations is important for computer-aided (MIS) in surgical planning and operation, or in developing surgical simulation tools or systems. The image acquisition facilities are also important for prediction accuracy. This dissertation addresses partial differential and integral equations (PDIE) based biomechanical modeling of soft tissue deformations incorporating the specific material properties to characterize the soft tissue responses for certain human interface behaviors. To achieve accurate simulation of real tissue deformations, several biomechanical finite element (FE) models are proposed to characterize liver tissue. The contribution of this work is in theoretical and practical aspects of tissue modeling. High resolution imaging techniques of Micro Computed Tomography (Micro-CT) and Cone Beam Computed Tomography (CBCT) imaging are first proposed to study soft tissue deformation in this dissertation. These high resolution imaging techniques can detect the tissue deformation details in the contact region between the tissue and the probe for small force loads which would be applied to a surgical probe used. Traditional imaging techniques in clinics can only achieve low image resolutions. Very small force loads seen in these procedures can only yield tissue deformation on the few millimeters to submillimeter scale. Small variations are hardly to detect. Furthermore, if a model is validated using high resolution images, it implies that the model is true in using the same model for low resolution imaging facilities. The reverse cannot be true since the small variations at the sub-millimeter level cannot be detected. In this dissertation, liver tissue deformations, surface morphological changes, and volume variations are explored and compared from simulations and experiments. The contributions of the dissertation are as follows. For liver tissue, for small force loads (5 grams to tens of grams), the linear elastic model and the neo-Hooke\u27s hyperelastic model are applied and shown to yield some discrepancies among them in simulations and discrepancies between simulations and experiments. The proposed finite element models are verified for liver tissue. A general FE modeling validation system is proposed to verify the applicability of FE models to the soft tissue deformation study. The validation of some FE models is performed visually and quantitatively in several ways in comparison with the actual experimental results. Comparisons among these models are also performed to show their advantages and disadvantages. The method or verification system can be applied for other soft tissues for the finite element analysis of the soft tissue deformation. For brain tissue, an elasticity based model was proposed previously employing local elasticity and Poisson\u27s ratio. It is validated by intraoperative images to show more accurate prediction of brain deformation than the linear elastic model. FE analysis of brain ventricle shape changes was also performed to capture the dynamic variation of the ventricles in author\u27s other works. There, for the safety reasons, the images for brain deformation modeling were from Magnetic Resonance Imaging (MRI) scanning which have been used for brain scanning. The measurement process of material properties involves the tissue desiccation, machine limits, human operation errors, and time factors. The acquired material parameters from measurement devices may have some difference from the tissue used in real state of experiments. Therefore, an experimental and simulation based method to inversely evaluate the material parameters is proposed and compare

Neurosurgery and brain shift: review of the state of the art and main contributions of robotics

Este artículo presenta una revisión acerca de la neurocirugía, los asistentes robóticos en este tipo de procedimiento, y el tratamiento que se le da al problema del desplazamiento que sufre el tejido cerebral, incluyendo las técnicas para la obtención de imágenes médicas. Se abarca de manera especial el fenómeno del desplazamiento cerebral, comúnmente conocido como brain shift, el cual causa pérdida de referencia entre las imágenes preoperatorias y los volúmenes a tratar durante la cirugía guiada por imágenes médicas. Hipotéticamente, con la predicción y corrección del brain shift sobre el sistema de neuronavegación, se podrían planear y seguir trayectorias de mínima invasión, lo que conllevaría a minimizar el daño a los tejidos funcionales y posiblemente a reducir la morbilidad y mortalidad en estos delicados y exigentes procedimientos médicos, como por ejemplo, en la extirpación de un tumor cerebral. Se mencionan también otros inconvenientes asociados a la neurocirugía y se muestra cómo los sistemas robotizados han ayudado a solventar esta problemática. Finalmente se ponen en relieve las perspectivas futuras de esta rama de la medicina, la cual desde muchas disciplinas busca tratar las dolencias del principal órgano del ser humano.This paper presents a review about neurosurgery, robotic assistants in this type of procedure, and the approach to the problem of brain tissue displacement, including techniques for obtaining medical images. It is especially focused on the phenomenon of brain displacement, commonly known as brain shift, which causes a loss of reference between the preoperative images and the volumes to be treated during image-guided surgery. Hypothetically, with brain shift prediction and correction for the neuronavigation system, minimal invasion trajectories could be planned and shortened. This would reduce damage to functional tissues and possibly lower the morbidity and mortality in delicate and demanding medical procedures such as the removal of a brain tumor. This paper also mentions other issues associated with neurosurgery and shows the way robotized systems have helped solve these problems. Finally, it highlights the future perspectives of neurosurgery, a branch of medicine that seeks to treat the ailments of the main organ of the human body from the perspective of many disciplines

Physical Constraint Finite Element Model for Medical Image Registration

Due to being derived from linear assumption, most elastic body based non-rigid image registration algorithms are facing challenges for soft tissues with complex nonlinear behavior and with large deformations. To take into account the geometric nonlinearity of soft tissues, we propose a registration algorithm on the basis of Newtonian differential equation. The material behavior of soft tissues is modeled as St. Venant-Kirchhoff elasticity, and the nonlinearity of the continuum represents the quadratic term of the deformation gradient under the Green- St.Venant strain. In our algorithm, the elastic force is formulated as the derivative of the deformation energy with respect to the nodal displacement vectors of the finite element; the external force is determined by the registration similarity gradient flow which drives the floating image deforming to the equilibrium condition. We compared our approach to three other models: 1) the conventional linear elastic finite element model (FEM); 2) the dynamic elastic FEM; 3) the robust block matching (RBM) method. The registration accuracy was measured using three similarities: MSD (Mean Square Difference), NC (Normalized Correlation) and NMI (Normalized Mutual Information), and was also measured using the mean and max distance between the ground seeds and corresponding ones after registration. We validated our method on 60 image pairs including 30 medical image pairs with artificial deformation and 30 clinical image pairs for both the chest chemotherapy treatment in different periods and brain MRI normalization. Our method achieved a distance error of 0.320±0.138 mm in x direction and 0.326±0.111 mm in y direction, MSD of 41.96±13.74, NC of 0.9958±0.0019, NMI of 1.2962±0.0114 for images with large artificial deformations; and average NC of 0.9622±0.008 and NMI of 1.2764±0.0089 for the real clinical cases. Student's t-test demonstrated that our model statistically outperformed the other methods in comparison (p-values <0.05)

Comparing Regularized Kelvinlet Functions and the Finite Element Method for Registration of Medical Images to Sparse Organ Data

Image-guided surgery collocates patient-specific data with the physical environment to facilitate surgical decision making in real-time. Unfortunately, these guidance systems commonly become compromised by intraoperative soft-tissue deformations. Nonrigid image-to-physical registration methods have been proposed to compensate for these deformations, but intraoperative clinical utility requires compatibility of these techniques with data sparsity and temporal constraints in the operating room. While linear elastic finite element models are effective in sparse data scenarios, the computation time for finite element simulation remains a limitation to widespread deployment. This paper proposes a registration algorithm that uses regularized Kelvinlets, which are analytical solutions to linear elasticity in an infinite domain, to overcome these barriers. This algorithm is demonstrated and compared to finite element-based registration on two datasets: a phantom dataset representing liver deformations and an in vivo dataset representing breast deformations. The regularized Kelvinlets algorithm resulted in a significant reduction in computation time compared to the finite element method. Accuracy as evaluated by target registration error was comparable between both methods. Average target registration errors were 4.6 +/- 1.0 and 3.2 +/- 0.8 mm on the liver dataset and 5.4 +/- 1.4 and 6.4 +/- 1.5 mm on the breast dataset for the regularized Kelvinlets and finite element method models, respectively. This work demonstrates the generalizability of using a regularized Kelvinlets registration algorithm on multiple soft tissue elastic organs. This method may improve and accelerate registration for image-guided surgery applications, and it shows the potential of using regularized Kelvinlets solutions on medical imaging data.Comment: 17 pages, 9 figure

Recent Milestones in Unraveling the Full-Field Structure of Dynamic Shear Cracks and Fault Ruptures in Real-Time: From Photoelasticity to Ultrahigh-Speed Digital Image Correlation

The last few decades have seen great achievements in dynamic fracture mechanics. Yet, it was not possible to experimentally quantify the full-field behavior of dynamic fractures, until very recently. Here, we review our recent work on the full-field quantification of the temporal evolution of dynamic shear ruptures. Our newly developed approach based on digital image correlation combined with ultrahigh-speed photography has revolutionized the capabilities of measuring highly transient phenomena and enabled addressing key ques- tions of rupture dynamics. Recent milestones include the visualization of the complete displacement, particle velocity, strain, stress and strain rate fields near growing ruptures, capturing the evolution of dynamic friction during individual rupture growth, and the detailed study of rupture speed limits. For example, dynamic friction has been the big- gest unknown controlling how frictional ruptures develop but it has been impossible, until now, to measure dynamic friction during spontaneous rupture propagation and to understand its dependence on other quantities. Our recent measurements allow, by simul- taneously tracking tractions and sliding speeds on the rupturing interface, to disentangle its complex dependence on the slip, slip velocity, and on their history. In another application, we have uncovered new phenomena that could not be detected with previous methods, such as the formation of pressure shock fronts associated with “supersonic” propagation of shear ruptures in viscoelastic materials where the wave speeds are shown to depend strongly on the strain rate