Meshless Mechanics and Point-Based Visualization Methods for Surgical Simulations

Computer-based modeling and simulation practices have become an integral part of the medical education field. For surgical simulation applications, realistic constitutive modeling of soft tissue is considered to be one of the most challenging aspects of the problem, because biomechanical soft-tissue models need to reflect the correct elastic response, have to be efficient in order to run at interactive simulation rates, and be able to support operations such as cuts and sutures. Mesh-based solutions, where the connections between the individual degrees of freedom (DoF) are defined explicitly, have been the traditional choice to approach these problems. However, when the problem under investigation contains a discontinuity that disrupts the connectivity between the DoFs, the underlying mesh structure has to be reconfigured in order to handle the newly introduced discontinuity correctly. This reconfiguration for mesh-based techniques is typically called dynamic remeshing, and most of the time it causes the performance bottleneck in the simulation. In this dissertation, the efficiency of point-based meshless methods is investigated for both constitutive modeling of elastic soft tissues and visualization of simulation objects, where arbitrary discontinuities/cuts are applied to the objects in the context of surgical simulation. The point-based deformable object modeling problem is examined in three functional aspects: modeling continuous elastic deformations with, handling discontinuities in, and visualizing a point-based object. Algorithmic and implementation details of the presented techniques are discussed in the dissertation. The presented point-based techniques are implemented as separate components and integrated into the open-source software framework SOFA. The presented meshless continuum mechanics model of elastic tissue were verified by comparing it to the Hertzian non-adhesive frictionless contact theory. Virtual experiments were setup with a point-based deformable block and a rigid indenter, and force-displacement curves obtained from the virtual experiments were compared to the theoretical solutions. The meshless mechanics model of soft tissue and the integrated novel discontinuity treatment technique discussed in this dissertation allows handling cuts of arbitrary shape. The implemented enrichment technique not only modifies the internal mechanics of the soft tissue model, but also updates the point-based visual representation in an efficient way preventing the use of costly dynamic remeshing operations

Trajectory planning with task constraints in densely filled environments

In this paper the problem of computing a rigid object trajectory in an environment populated with deformable objects is addressed. The problem arises in Minimally Invasive Robotic Surgery (MIRS) from the needs of reaching a point of interest inside the anatomy with rigid laparoscopic instruments. We address the case of abdominal surgery. The abdomen is a densely populated soft environment and it is not possible to apply classical techniques for obstacle avoidance because a collision free solution is, most of the time, not feasible. In order to have a convergent algorithm with, at least, one possible solution we have to relax the constraints and allow collision under a specific contact threshold to avoid tissue damaging. In this work a new approach for trajectory planning under these peculiar conditions is implemented. The method computes offline the path which is then tested in a surgical simulator as part of a pre-operative surgical plan

Real-time Error Control for Surgical Simulation

Objective: To present the first real-time a posteriori error-driven adaptive finite element approach for real-time simulation and to demonstrate the method on a needle insertion problem. Methods: We use corotational elasticity and a frictional needle/tissue interaction model. The problem is solved using finite elements within SOFA. The refinement strategy relies upon a hexahedron-based finite element method, combined with a posteriori error estimation driven local $h$-refinement, for simulating soft tissue deformation. Results: We control the local and global error level in the mechanical fields (e.g. displacement or stresses) during the simulation. We show the convergence of the algorithm on academic examples, and demonstrate its practical usability on a percutaneous procedure involving needle insertion in a liver. For the latter case, we compare the force displacement curves obtained from the proposed adaptive algorithm with that obtained from a uniform refinement approach. Conclusions: Error control guarantees that a tolerable error level is not exceeded during the simulations. Local mesh refinement accelerates simulations. Significance: Our work provides a first step to discriminate between discretization error and modeling error by providing a robust quantification of discretization error during simulations.Comment: 12 pages, 16 figures, change of the title, submitted to IEEE TBM

Real-time measurement corrected prediction of soft tissue response for medical simulations

Medical simulators, such as in palpation and disease diagnosis, require an efficient model of the biological soft tissue deformation. Hence, a computationally fast and accurate algorithm is required to support and enhance user interactions in near real-time simulations. The visual accuracy of such simulators is dependent on the user¿s reaction time. Static visual images that update at a rate of 25 Hz are perceived as real-time moving images. Hence, visualizing software requires fast algorithms to compute the deformation of soft tissue to facilitate a meaningful simulation. Furthermore, soft tissue behaviour should be modelled accurately while compatible with real-time computation. This work proposes a fast solver for the linearized finite element method (FEM) and validates the proposed algorithm with experimental results. The novelty of the method lies in the utilization of real-time force/displacement measurements that are embedded in the solution via the Kalman filter. A novel computational algorithm that utilizes the strength of the FEM in terms of accuracy and employs direct measurements from the manipulated tissue to overcome the slow computational process of the FEM is proposed in the first part of the thesis. As the behaviour of the mechanically loaded tissue can be regarded as linearly responding at each time step, a constant acceleration temporal discretization method, i.e., the Newmark-ß is employed. In real-time applications, the accuracy of the target variable highly depends on the accuracy of the inputs while differentiating noise from the signal is hardly ever possible. To address this problem, a Kalman filter-based method is developed. The proposed algorithm not only filters the noise from the measurements but also adapts the filter gain to the estimates of the target variable, i.e., the resulting tissue deformation. For a simulated tension test of a cubic model, the proposed algorithm achieves the update frequency of 63.3 Hz. This rate is a significant improvement in computational speed compared to the 5.8 Hz update rate by the classic FEM. Besides, this novel combination of the KF and the FEM makes it possible to expand the displacement estimates in the spatial domain when the measurements are only partially available at certain points. The performance of the above method is validated experimentally through a comparison with indentation tests on artificial human tissue-like material and with the FEM result under identical simulation conditions. The test is repeated on several samples, and the displacement variation from the FEM outcome is considered as the model error. Simulation results show that the proposed method achieves the deformation update frequency of 145.7 Hz compared to the 2.7 Hz from the reference FEM. The proposed method shows the same predictive ability, only 0.47% difference from FEM on average. Experimental validation of the proposed KF-FEM confirms that by consideration of both the measurement noise and the model error, the proposed method is capable of achieving high-frequency response without sacrificing the accuracy. Further to this, the experiments confirmed the linearized model response is reliable within the applied displacement range and therefore proving that KF can be employed. The developed KF-FEM was modified in the next study to address the problem resulting from inaccurate external loads measurements by the force sensors. In the modified version, both the external force, i.e., driving variable, and the displacement, i.e., driven variable, are taken as system states. It is considered that the uncertainty of the model input influences the accuracy of the system estimates. The modified model is calibrated to differentiate the system noise from the input noise. Numerical simulations were conducted on a liver shape geometrical model, and the simulation results demonstrate that more than 90% of the measurement noise is removed. The computational speed is also increased, delivering up to 89 Hz update rate. While the uncertainty of the external load is replicated in the displacements in an FEM solution, the developed algorithm can differentiate the measurement noise, including the displacement and external forces, from the system error, i.e., the FE model error. In the last study, the proposed model was developed to reflect the nonlinear behaviour of the manipulated tissue. The Central Difference time discretization method was used to model large deformations. A novel feature is that the Equation of motion is formulated within the element level rather than in the global spatial domain. This approach helped to improve the computational speed. Indentation with strains of slightly over 10% was simulated to assess the performance of the proposed model. The developed algorithm achieved the 33.85 Hz update frequency on a standard-issue PC and confirmed its suitability for real-time applications. Also, the proposed model achieved estimates with a maximum 5.75% mean absolute error (MAE) concerning the measurements while the classic FEM showed 6.20% MAE under identical simulation condition. Results confirm that deformation estimates for noisy boundary loads of the FEM can be improved with the help of direct measurements and yet be realistic in terms of real-time visual update. This study proposed a novel computational algorithm that achieved update frequencies of higher than 25 Hz to be perceived as real-time in human eyes. The developed KF-FEM model has also shown the potential of improving the FEM accuracy with the help of direct measurements. The proposed algorithm used partially available measurements and expanded its estimates in the spatial domain. The method was experimentally validated, and the model input uncertainty, as well as the nonlinear behaviour of the soft tissue, were assessed and verified

Research on real-time physics-based deformation for haptic-enabled medical simulation

This study developed a multiple effective visuo-haptic surgical engine to handle a variety of surgical manipulations in real-time. Soft tissue models are based on biomechanical experiment and continuum mechanics for greater accuracy. Such models will increase the realism of future training systems and the VR/AR/MR implementations for the operating room

Connective Tissues Simulation on GPU

International audienceRecent work in the field of medical simulation have led to real advances in the mechanical simulation of organs. However, it is important to notice that, despite the major role they may have in the interaction between organs, the connective tissues are often left out of these simulations. In this paper, we propose a model which can rely on either a mesh based or a meshless methods. To provide a realistic simulation of these tissues, our work is based on the weak form of continuum mechanics equations for hyperelastic soft materials. Furthermore, the stability of deformable objects simulation is ensured by an implicit temporal integration scheme. Our method allows to model these tissues without prior assumption on the dimension of their of their geometry (curve, surface or volume), and enables mechanical coupling between organs. To obtain an interactive frame rate, we develop a parallel version suitable for to GPU computation. Finally we demonstrate the proper convergence of our finite element scheme

Non-rigid registration of 2-D/3-D dynamic data with feature alignment

In this work, we are computing the matching between 2D manifolds and 3D manifolds with temporal constraints, that is we are computing the matching among a time sequence of 2D/3D manifolds. It is solved by mapping all the manifolds to a common domain, then build their matching by composing the forward mapping and the inverse mapping. At first, we solve the matching problem between 2D manifolds with temporal constraints by using mesh-based registration method. We propose a surface parameterization method to compute the mapping between the 2D manifold and the common 2D planar domain. We can compute the matching among the time sequence of deforming geometry data through this common domain. Compared with previous work, our method is independent of the quality of mesh elements and more efficient for the time sequence data. Then we develop a global intensity-based registration method to solve the matching problem between 3D manifolds with temporal constraints. Our method is based on a 4D(3D+T) free-from B-spline deformation model which has both spatial and temporal smoothness. Compared with previous 4D image registration techniques, our method avoids some local minimum. Thus it can be solved faster and achieve better accuracy of landmark point predication. We demonstrate the efficiency of these works on the real applications. The first one is applied to the dynamic face registering and texture mapping. The second one is applied to lung tumor motion tracking in the medical image analysis. In our future work, we are developing more efficient mesh-based 4D registration method. It can be applied to tumor motion estimation and tracking, which can be used to calculate the read dose delivered to the lung and surrounding tissues. Thus this can support the online treatment of lung cancer radiotherapy

A parallel interaction potential approach coupled with the immersed boundary method for fully resolved simulations of deformable interfaces and membranes

In this paper we show and discuss the use of a versatile interaction potential approach coupled with an immersed boundary method to simulate a variety of flows involving deformable bodies. In particular, we focus on two kinds of problems, namely (i) deformation of liquid-liquid interfaces and (ii) flow in the left ventricle of the heart with either a mechanical or a natural valve. Both examples have in common the two-way interaction of the flow with a deformable interface or a membrane. The interaction potential approach (de Tullio & Pascazio, Jou. Comp. Phys., 2016; Tanaka, Wada and Nakamura, Computational Biomechanics, 2016) with minor modifications can be used to capture the deformation dynamics in both classes of problems. We show that the approach can be used to replicate the deformation dynamics of liquid-liquid interfaces through the use of ad-hoc elastic constants. The results from our simulations agree very well with previous studies on the deformation of drops in standard flow configurations such as deforming drop in a shear flow or a cross flow. We show that the same potential approach can also be used to study the flow in the left ventricle of the heart. The flow imposed into the ventricle interacts dynamically with the mitral valve (mechanical or natural) and the ventricle which are simulated using the same model. Results from these simulations are compared with ad- hoc in-house experimental measurements. Finally, a parallelisation scheme is presented, as parallelisation is unavoidable when studying large scale problems involving several thousands of simultaneously deforming bodies on hundreds of distributed memory computing processors

Collaborative soft object manipulation for game engine-based virtual reality surgery simulators

In this paper we analyse and evaluate the capabilities of popular game engines to simulate and interact with soft objects. We discuss how these engines can be used for simulated surgical training applications, determine their shortcomings and make suggestions how game engines can be extended to make them more suitable for such applications