Image-guided Simulation of Heterogeneous Tissue Deformation For Augmented Reality during Hepatic Surgery

International audienceThis paper presents a method for real-time augmentation of vas- cular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Com- pared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and ex vivo data and illustrate the benefits of such an approach for minimally invasive surgery

Image-guided Simulation of Heterogeneous Tissue Deformation For Augmented Reality during Hepatic Surgery

International audienceThis paper presents a method for real-time augmentation of vas- cular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Com- pared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and ex vivo data and illustrate the benefits of such an approach for minimally invasive surgery

Simulation of hyperelastic materials in real-time using Deep Learning

The finite element method (FEM) is among the most commonly used numerical methods for solving engineering problems. Due to its computational cost, various ideas have been introduced to reduce computation times, such as domain decomposition, parallel computing, adaptive meshing, and model order reduction. In this paper we present U-Mesh: a data-driven method based on a U-Net architecture that approximates the non-linear relation between a contact force and the displacement field computed by a FEM algorithm. We show that deep learning, one of the latest machine learning methods based on artificial neural networks, can enhance computational mechanics through its ability to encode highly non-linear models in a compact form. Our method is applied to two benchmark examples: a cantilever beam and an L-shape subject to moving punctual loads. A comparison between our method and proper orthogonal decomposition (POD) is done through the paper. The results show that U-Mesh can perform very fast simulations on various geometries, mesh resolutions and number of input forces with very small errors

Determining the Biomechanical Behavior of the Liver Using Medical Image Analysis and Evolutionary Computation

Modeling the liver deformation forms the basis for the development of new clinical applications that improve the diagnosis, planning and guidance in liver surgery. However, the patient-specific modeling of this organ and its validation are still a challenge in Biomechanics. The reason is the difficulty to measure the mechanical response of the in vivo liver tissue. The current approach consist of performing minimally invasive or open surgery aimed at estimating the elastic constant of the proposed biomechanical models. This dissertation presents how the use of medical image analysis and evolutionary computation allows the characterization of the biomechanical behavior of the liver, avoiding the use of these minimally invasive techniques. In particular, the use of similarity coefficients commonly used in medical image analysis has permitted, on one hand, to estimate the patient-specific biomechanical model of the liver avoiding the invasive measurement of its mechanical response. On the other hand, these coefficients have also permitted to validate the proposed biomechanical models. Jaccard coefficient and Hausdorff distance have been used to validate the models proposed to simulate the behavior of ex vivo lamb livers, calculating the error between the volume of the experimentally deformed samples of the livers and the volume from biomechanical simulations of these deformations. These coefficients has provided information, such as the shape of the samples and the error distribution along their volume. For this reason, both coefficients have also been used to formulate a novel function, the Geometric Similarity Function (GSF). This function has permitted to establish a methodology to estimate the elastic constants of the models proposed for the human liver using evolutionary computation. Several optimization strategies, using GSF as cost function, have been developed aimed at estimating the patient-specific elastic constants of the biomechanical models proposed for the human liver. Finally, this methodology has been used to define and validate a biomechanical model proposed for an in vitro human liver.Martínez Martínez, F. (2014). Determining the Biomechanical Behavior of the Liver Using Medical Image Analysis and Evolutionary Computation [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/39337TESI

A physically based trunk soft tissue modeling for scoliosis surgery planning systems

One of the major concerns of scoliotic patients undergoing spinal correction surgery is the trunk's external appearance after the surgery. This paper presents a novel incremental approach for simulating postoperative trunk shape in scoliosis surgery. Preoperative and postoperative trunk shapes data were obtained using three-dimensional medical imaging techniques for seven patients with adolescent idiopathic scoliosis. Results of qualitative and quantitative evaluations, based on the comparison of the simulated and actual postoperative trunk surfaces, showed an adequate accuracy of the method. Our approach provides a candidate simulation tool to be used in a clinical environment for the surgery planning process.IRSC / CIH

Real-time 3D reconstruction of non-rigid shapes with a single moving camera

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small mapsPeer ReviewedPostprint (author's final draft

Towards an Accurate Tracking of Liver Tumors for Augmented Reality in Robotic Assisted Surgery

International audienceThis article introduces a method for tracking the internal structures of the liver during robot-assisted procedures. Vascular network, tumors and cut planes, computed from pre-operative data, can be overlaid onto the laparoscopic view for image-guidance, even in the case of large motion or deformation of the organ. Compared to current methods, our method is able to precisely propagate surface motion to the internal structures. This is made possible by relying on a fast yet accurate biomechanical model of the liver combined with a robust visual tracking approach designed to properly constrain the model. Augmentation results are demonstrated on in-vivo sequences of a human liver during robotic surgery, while quantitative validation is performed on an ex-vivo porcine liver experimentation. Validation results show that our approach gives an accurate surface registration with an error of less than 6mm on the position of the tumor

Automatic Alignment of pre and intraoperative Data using Anatomical Landmarks for Augmented Laparoscopic Liver Surgery

International audienceEach year in Europe 50,000 new liver cancer cases are diagnosed for which hepatic surgery combined to chemotherapy is the most common treatment. In particular the number of laparoscopic liver surgeries has increased significantly over the past years. This type of minimally invasive procedure which presents many benefits for the patient is challenging for the surgeons due to the limited field of view. Recently new augmented reality techniques which merge preoperative data and intraoperative images and permit to visualize internal structures have been proposed to help surgeons during this type of surgery. One of the difficulties is to align preoperative data with the intraoperative images. We propose in this paper a semi-automatic approach for solving the ill-posed problem of initial alignment for Augmented Reality systems during liver surgery. Our registration method relies on anatomical landmarks extracted from both the laparoscopic images and three-dimensional model, using an image-based soft-tissue reconstruction technique and an atlas-based approach, respectively. The registration evolves automatically from a quasi-rigid to a non-rigid registration. Furthermore, the surface-driven deformation is induced in the volume via a patient specific biomechanical model. The experiments conducted on both synthetic and in vivo data show promising results with a registration error of 2 mm when dealing with a visible surface of 30% of the whole liver