Real-time simulation of surgery by Proper Generalized Decomposition techniques

La simulación quirúrgica por ordenador en tiempo real se ha convertido en una alternativa muy atractiva a los simuladores quirúrgicos tradicionales. Entre otras ventajas, los simuladores por ordenador consiguen ahorros importantes de tiempo y de costes de mantenimiento, y permiten que los estudiantes practiquen sus habilidades quirúrgicas en un entorno seguro tantas veces como sea necesario. Sin embargo, a pesar de las capacidades de los ordenadores actuales, la cirugía computacional sigue siendo un campo de investigación exigente. Uno de sus mayores retos es la alta velocidad a la que se tienen que resolver complejos problemas de mecánica de medios continuos para que los interfaces hápticos puedan proporcionar un sentido del tacto realista (en general, se necesitan velocidades de respuesta de 500-1000 Hz).Esta tesis presenta algunos métodos numéricos novedosos para la simulación interactiva de dos procedimientos quirúrgicos habituales: el corte y el rasgado (o desgarro) de tejidos blandos. El marco común de los métodos presentados es el uso de la Descomposición Propia Generalizada (PGD en inglés) para la generación de vademécums computacionales, esto es, metasoluciones generales de problemas paramétricos de altas dimensiones que se pueden evaluar a velocidades de respuesta compatibles con entornos hápticos.En el caso del corte, los vademécums computacionales se utilizan de forma conjunta con técnicas basadas en XFEM, mientras que la carga de cálculo se distribuye entre una etapa off-line (previa a la ejecución interactiva) y otra on-line (en tiempo de ejecución). Durante la fase off-line, para el órgano en cuestión se precalculan tanto un vademécum computacional para cualquier posición de una carga, como los desplazamientos producidos por un conjunto de cortes. Así, durante la etapa on-line, los resultados precalculados se combinan de la forma más adecuada para obtener en tiempo real la respuesta a las acciones dirigidas por el usuario. En cuanto al rasgado, a partir de una ecuación paramétrica basada en mecánica del daño continuo, se obtiene un vademécum computacional. La complejidad del modelo se reduce mediante técnicas de Descomposición Ortogonal Propia (POD en inglés), y el vademécum se incorpora a una formulación incremental explícita que se puede interpretar como una especie de integrador temporal.A modo de ejemplo, el método para el corte se aplica a la simulación de un procedimiento quirúrgico refractivo de la córnea conocido como queratotomía radial, mientras que el método para el rasgado se centra en la simulación de la colecistectomía laparoscópica (la extirpación de la vesícula biliar mediante laparoscopia). En ambos casos, los métodos implementados ofrecen excelentes resultados en términos de velocidades de respuesta y producen simulaciones muy realistas desde los puntos de vista visual y háptico.The real-time computer-based simulation of surgery has proven to be an appealing alternative to traditional surgical simulators. Amongst other advantages, computer-based simulators provide considerable savings on time and maintenance costs, and allow trainees to practice their surgical skills in a safe environment as often as necessary. However, in spite of the current computer capabilities, computational surgery continues to be a challenging field of research. One of its major issues is the high speed at which complex problems in continuum mechanics have to be solved so that haptic interfaces can render a realistic sense of touch (generally, feedback rates of 500–1 000 Hz are required). This thesis introduces some novel numerical methods for the interactive simulation of two usual surgical procedures: cutting and tearing of soft tissues. The common framework of the presented methods is the use of the Proper Generalised Decomposition (PGD) for the generation of computational vademecums, i. e. general meta-solutions of parametric high-dimensional problems that can be evaluated at feedback rates compatible with haptic environments. In the case of cutting, computational vademecums are used jointly with XFEM-based techniques, and the computing workload is distributed into an off-line and an on-line stage. During the off-line stage, both a computational vademecum for any position of a load and the displacements produced by a set of cuts are pre-computed for the organ under consideration. Thus, during the on-line stage, the pre-computed results are properly combined together to obtain in real-time the response to the actions driven by the user. Concerning tearing, a computational vademecum is obtained from a parametric equation based on continuum damage mechanics. The complexity of the model is reduced by Proper Orthogonal Decomposition (POD) techniques, and the vademecum is incorporated into an explicit incremental formulation that can be viewed as a sort of time integrator. By way of example, the cutting method is applied to the simulation of a corneal refractive surgical procedure known as radial keratotomy, whereas the tearing method focuses on the simulation of laparoscopic cholecystectomy (i. e. the removal of the gallbladder). In both cases, the implemented methods offer excellent performances in terms of feedback rates, and produce.<br /

Serial FEM/XFEM-Based Update of Preoperative Brain Images Using Intraoperative MRI

Current neuronavigation systems cannot adapt to changing intraoperative conditions over time. To overcome this limitation, we present an experimental end-to-end system capable of updating 3D preoperative images in the presence of brain shift and successive resections. The heart of our system is a nonrigid registration technique using a biomechanical model, driven by the deformations of key surfaces tracked in successive intraoperative images. The biomechanical model is deformed using FEM or XFEM, depending on the type of deformation under consideration, namely, brain shift or resection. We describe the operation of our system on two patient cases, each comprising five intraoperative MR images, and we demonstrate that our approach significantly improves the alignment of nonrigidly registered images

Volumetric modeling and interactive cutting of deformable bodies

Special Issue on Biomechanical Modelling of Soft Tissue MotionInternational audienceA new approach for the interactive simulation of viscoelastic object cutting is presented. Two synchronized geo- metrical models at different resolutions are used, both derived from medical images. In contrast with most previous approaches, the blade deforms the object, and cutting occurs once a contact pressure threshold is exceeded. Moreover, we achieve interactive simulation rates by embedding a high resolution geometry within a regular grid with arbitrary resolution. This allows to trade off accuracy for speed in the computation of deformations. The input data is a high- resolution volumetric model of the objects. The surface model of the object, used for rendering as well as collision detection and response, is a polygonal level set of the volumetric data. It is embedded in the volume model using barycentric coordinates. Cutting is performed by removing voxels at the fine level, and updating the surface and volume models accordingly. We introduce a new data structure, which we call a Dynamic Branched Grid, in order to preserve the fine level topology at the coarse level. When an element of the coarse volumetric model is cut, it is replaced by a number of superimposed elements with the same size and at the same rest position as the original one. Each new element is assigned a part of material contained in the original one, and the mass and stiffness are recomputed accordingly. The well-known problem of creating small, ill-shaped finite elements while remeshing is thus completely avoided

An interactive meshless cutting model for nonlinear viscoelastic soft tissue in surgical simulators

In this paper, we present a novel interactive cutting simulation model for soft tissue based on the meshless framework. Unlike most existing methods that consider the cutting process of soft tissue in an over simplified manner, the presented model is able to simulate the complete cutting process that includes three stages: deformation before cutting open (DBCO), cutting open (CO), and deformation after cutting open (DACO). To characterize the complicated physical and mechanical properties of soft tissue, both nonlinearity and viscoelasticity were incorporated into the equations governing the motion of soft tissue. A line contact model was used for simulating the cutting process after analyzing the two major types of surgical instruments, i.e., scalpel and electrostick. The cutting speed and angle were taken into account in order to improve haptic rendering. Biomechanical tests and simulation experiments verified the validity of the introduced model. Specifically, the displacement vs. cutting force curves can be divided into three segments corresponding to the three stages of the cutting process. The results were also applied in a liver cutting simulating system and satisfactory visual effect and haptic feedback were achieved

Accurate Real-Time Framework for Complex Pre-defined Cuts in Finite Element Modeling

PhD ThesisAchieving detailed pre-defined cuts on deformable materials is vitally pivotal for many commercial applications, such as cutting scenes in games and vandalism effects in virtual movies. In these types of applications, the majority of resources are allocated to achieve high-fidelity representations of materials and the virtual environments. In the case of limited computing resources, it is challenging to achieve a convincing cutting effect. On the premise of sacrificing realism effects or computational cost, a considerable amount of research work has been carried out, but the best solution that can be compatible with both cases has not yet been identified. This doctoral dissertation is dedicated to developing a unique framework for representing pre-defined cuts of deformable surface models, which can achieve real-time, detailed cutting while maintaining the realistic physical behaviours. In order to achieve this goal, we have made in-depth explorations from geometric and numerical perspectives. From a geometric perspective, we propose a robust subdivision mechanism that allows users to make arbitrary predetermined cuts on elastic surface models based on the finite element method (FEM). Specifically, after the user separates the elements in an arbitrary manner (i.e., linear or non-linear) on the topological mesh, we then optimise the resulting mesh by regenerating the triangulation within the element based on the Delaunay triangulation principle. The optimisation of regenerated triangles, as a process of refining the ill-shaped elements that have small Aspect Ratio, greatly improves the realism of physical behaviours and guarantees that the refinement process is balanced with real-time requirements. The above subdivision mechanism can improve the visual effect of cutting, but it neglects the fact that elements cannot be perfectly cut through any pre-defined trajectories. The number of ill-shaped elements generated yield a significant impact on the optimisation time: a large number of ill-shaped elements will render the cutting slow or even collapse, and vice versa. Our idea is based on the core observation that the producing of ill-shaped elements is largely attributed to the condition number of the global stiffness matrix. Practically, for a stiffness matrix, a large condition number means that it is almost singular, and the calculation of its inverse or the solution of a system of linear equations are prone to large numerical errors and time-consuming. It motivates us to alleviate the impact of condition number of the global stiffness matrix from the numerical aspects. Specifically, we address this issue in a novel manner by converting the global stiffness matrix into the form of a covariance matrix, in which the number of conditions of the matrix can be reduced by exploiting appropriate matrix normalisation to the eigenvalues. Furthermore, we investigated the efficiency of two different scenarios: an exact square-root normalisation and its approximation based on the Newton-Schulz iteration. Experimental tests of the proposed framework demonstrate that it can successfully reproduce competitive visuals of detailed pre-defined cuts compared with the state-of-the-art method (Manteaux et al. 2015) while obtaining a significant improvement on the FPS, increasing up to 46.49 FPS and 21.93 FPS during and after the cuts, respectively. Also, the new refinement method can stably maintain the average Aspect Ratio of the model mesh after the cuts at less than 3 and the average Area Ratio around 3%. Besides, the proposed two matrix normalisation strategies, including ES-CGM and AS-CGM, have shown the superiority of time efficiency compared with the baseline method (Xin et al. 2018). Specifically, the ES-CGM and AS-CGM methods obtained 5 FPS and 10 FPS higher than the baseline method, respectively. These experimental results strongly support our conclusion which is that this new framework would provide significant benefits when implemented for achieving detailed pre-defined cuts at a real-time rate

Cutting of cortical bone tissue: analysis of deformation and fracture process

Cortical bone tissue – one of the most intriguing materials found in nature – demonstrate some fascinating behaviours that have attracted great attention of many researchers from all over the world. In contrast to engineering materials, bone has its unique characters: it is a material that has both sufficient stiffness and toughness to provide physical support and protection to internal organs and yet adaptively balanced for its weight and functional requirements. Its structure and mechanical properties are of great importance to the physiological functioning of the body. Still, our understanding on the mechanical deformation processes of cortical bone tissue is rather limited. Penetration into a bone tissue is an intrinsic part of many clinical procedures, such as orthopaedic surgery, bone implant and repair operations. The success of bone-cutting surgery depends largely on precision of the operation and the extent of damage it causes to the surrounding tissues. The anisotropic behaviour of cortical bone acts as a distinctive protective mechanism and increases the difficulty during cutting process. A comprehensive understanding of deformation and damage mechanisms during the cutting process is necessary for improving the operational accuracy and postoperative recovery of patients. However, the current literature on experimental results provides limited information about processes in the vicinity of the cutting tool-bone interaction zone; while; numerical models cannot fully describe the material anisotropy and the effect of damage mechanisms of cortical bone tissue. In addition, a conventional finite-element scheme faces numerical challenges due to large deformation and highly localised distortion in the process zone. This PhD project is aimed at bridging the gap in current lack of understanding on cutting-induced deformation and fracture processes in the cortical bone tissue through experimental and numerical approaches. A number of experimental studies were accomplished to characterise the mechanical behaviour of bovine cortical bone tissue and to analyse deformation and damage mechanisms associated with the cutting process II along different bone axes in four anatomic cortices, namely, anterior, posterior, medial and lateral. These experiments included: (1) a Vickers hardness test to provide initial assessments on deformation and damage processes in the cortical bone tissue under a concentrated compressive load; (2) uniaxial tension and compression tests, performed to understand the effect of orientation and local variability of microstructure constituents on the macroscopic material properties of cortical bone; (3) fracture toughness tests, aimed at elucidating the anisotropic character of fracture toughness of cortical bone and its various fracture toughness mechanisms in relation to different orientations; (4) penetration tests, conducted to evaluate and validate mechanisms involved in bone cutting as well as orientation associated anisotropic deformation and damage processes at various different cortex positions. Information obtained in these experimental studies was used to assist the development of advanced finite-element models: (1) the effective homogenised XFEM models developed in conjunction with three-point bending test to represent a macroscopically, anisotropic elasticplastic fracture behaviour of cortical bone tissue; (2) three microstructured XFEM models to further investigate the effect of the randomly distributed microstructural constituents on the local fracture process and the variability of fracture toughness of cortical bone; (3) a novel finite-element modelling approach encompassing both conventional and SPH elements, incorporating anisotropic elastic-plastic material properties and progressive damage criteria to simulate large deformation and damage processes of cortical bone under penetration. The established models can adequately and accurately reflect large deformations and damage processes during the penetration in bone cutting. The results of this study made valuable contributions to our existing understanding of the mechanics of cortical bone tissue and most importantly to the understanding of its mechanical behaviours during the cutting process

Découpage virtuel interactif de corps élastiques pour simulation chirurgicale

''RÉSUMÉ : La simulation chirurgicale dans un environnement de réalité virtuelle fournit un moyen de pratiquer certaines opérations sans les risques associés à une intervention sur un patient ou le coût d’un mannequin. Afin de générer un sentiment de présence, on cherche à produire un environnement le plus complet possible, incluant une vision 3D, un retour haptique, des interactions crédibles et un comportement physique réaliste des objets présents. La recherche présentée ici porte sur la simulation physique du comportement d’organes mous, comme le foie ou le cerveau, ainsi que sur le découpage de ces organes à l’aide d’un scalpel, une interaction particulièrement difficile à reproduire virtuellement de façon réaliste. L’objectif principal est de développer une méthode de déformation à la fois réaliste et efficace, et de permettre à un utilisateur de découper interactivement un objet simulé par cette méthode, à l’aide d’un outil tranchant virtuel. De plus, nous voulons que la déformation et les interactions soient décrites avec une grande précision, tout en permettant d’effectuer les calculs très rapidement, pour une interaction fluide qui maintient le sentiment de présence.''----------''ABSTRACT : Surgery simulation in a virtual reality environment provides a way to practice certain operations without the risks associated with performing surgery on a patient or the cost of using arealistic dummy. To facilitate immersion, we seek to produce an environment as complete as possible, including 3D vision, haptic feedback, credible interactions and a realistic physical behavior of simulated objects. The research presented in this document focuses on the physical behavior of soft organs, like the brain or liver, and on cutting these organs using a scalpel. It is especially difficult to reproduce virtually that interaction in a realistic way. The main objective is to develop a deformation method that is both realistic and efficient, and to allow a user to interactively cut an object simulated through this method, using a virtual sharp tool. Furthermore, we want the deformation and interactions to be described with high precision while allowing for fast computations, for a smooth interaction that maintains immersion.'

Machine learning and interactive real-time simulation for training on relevant total hip replacement skills.

Virtual Reality simulators have proven to be an excellent tool in the medical sector to help trainees mastering surgical abilities by providing them with unlimited training opportunities. Total Hip Replacement (THR) is a procedure that can benefit significantly from VR/AR training, given its non-reversible nature. From all the different steps required while performing a THR, doctors agree that a correct fitting of the acetabular component of the implant has the highest relevance to ensure successful outcomes. Acetabular reaming is the step during which the acetabulum is resurfaced and prepared to receive the acetabular implant. The success of this step is directly related to the success of fitting the acetabular component. Therefore, this thesis will focus on developing digital tools that can be used to assist the training of acetabular reaming. Devices such as navigation systems and robotic arms have proven to improve the final accuracy of the procedure. However, surgeons must learn to adapt their instrument movements to be recognised by infrared cameras. When surgeons are initially introduced to these systems, surgical times can be extended up to 20 minutes, maximising surgical risks. Training opportunities are sparse, given the high investment required to purchase these devices. As a cheaper alternative, we developed an Augmented Reality (AR) alternative for training on the calibration of imageless navigation systems (INS). At the time, there were no alternative simulators that using head-mounted displays to train users into the steps to calibrate such systems. Our simulator replicates the presence of an infrared camera and its interaction with the reflecting markers located on the surgical tools. A group of 6 hip surgeons were invited to test the simulator. All of them expressed their satisfaction with the ease of use and attractiveness of the simulator as well as the similarity of interaction with the real procedure. The study confirmed that our simulator represents a cheaper and faster option to train multiple surgeons simultaneously in the use of Imageless Navigation Systems (INS) than learning exclusively on the surgical theatre. Current reviews on simulators for orthopaedic surgical procedures lack objective metrics of assessment given a standard set of design requirements. Instead, most of them rely exclusively on the level of interaction and functionality provided. We propose a comparative assessment rubric based on three different evaluation criteria. Namely immersion, interaction fidelity, and applied learning theories. After our assessment, we found that none of the simulators available for THR provides an accurate interactive representation of resurfacing procedures such as acetabular reaming based on force inputs exerted by the user. This feature is indispensable for an orthopaedics simulator, given that hand-eye coordination skills are essential skills to be trained before performing non-reversible bone removal on real patients. Based on the findings of our comparative assessment, we decided to develop a model to simulate the physically-based deformation expected during traditional acetabular reaming, given the user’s interaction with a volumetric mesh. Current interactive deformation methods on high-resolution meshes are based on geometrical collision detection and do not consider the contribution of the materials’ physical properties. By ignoring the effect of the material mechanics and the force exerted by the user, they become inadequate for training on hand- eye coordination skills transferable to the surgical theatre. Volumetric meshes are preferred in surgical simulation to geometric ones, given that they are able to represent the internal evolution of deformable solids resulting from cutting and shearing operations. Existing numerical methods for representing linear and corotational FEM cuts can only maintain interactive framerates at a low resolution of the mesh. Therefore, we decided to train a machine-learning model to learn the continuum mechanic laws relevant to acetabular reaming and predict deformations at interactive framerates. To the best of our knowledge, no research has been done previously on training a machine learning model on non-elastic FEM data to achieve results at interactive framerates. As training data, we used the results from XFEM simulations precomputed over 5000 frames for plastic deformations on tetrahedral meshes with 20406 elements each. We selected XFEM simulation as the physically-based deformation ground-truth given its accuracy and fast convergence to represent cuts, discontinuities and large strain rates. Our machine learning-based interactive model was trained following the Graph Neural Networks (GNN) blocks. GNNs were selected to learn on tetrahedral meshes as other supervised-learning architectures like the Multilayer perceptron (MLP), and Convolutional neural networks (CNN) are unable to learn the relationships between entities with an arbitrary number of neighbours. The learned simulator identifies the elements to be removed on each frame and describes the accumulated stress evolution in the whole machined piece. Using data generated from the results of XFEM allowed us to embed the effects of non-linearities in our interactive simulations without extra processing time. The trained model executed the prediction task using our tetrahedral mesh and unseen reamer orientations faster per frame than the time required to generate the training FEM dataset. Given an unseen orientation of the reamer, the trained GN model updates the value of accumulated stress on each of the 20406 tetrahedral elements that constitute our mesh during the prediction task. Once this value is updated, the tetrahedrons to be removed from the mesh are identified using a threshold condition. After using each single-frame output as input for the following prediction repeatedly for up to 60 iterations, our model can maintain an accuracy of up to 90.8% in identifying the status of each element given their value of accumulated stress. Finally, we demonstrate how the developed estimator can be easily connected to any game engine and included in developing a fully functional hip arthroplasty simulator

Preserving Topology and Elasticity for Embedded Deformable Models

International audienceIn this paper we introduce a new approach for the embedding of linear elastic deformable models. Our technique results in significant improvements in the efficient physically based simulation of highly detailed objects. First, our embedding takes into account topological details, that is, disconnected parts that fall into the same coarse element are simulated independently. Second, we account for the varying material properties by computing stiffness and interpolation functions for coarse elements which accurately approximate the behaviour of the embedded material. Finally, we also take into account empty space in the coarse embeddings, which provides a better simulation of the boundary. The result is a straightforward approach to simulating complex deformable models with the ease and speed associated with a coarse regular embedding, and with a quality of detail that would only be possible at much finer resolution