Real-time simulation of surgery by Proper Generalized Decomposition techniques

La simulación quirúrgica por ordenador en tiempo real se ha convertido en una alternativa muy atractiva a los simuladores quirúrgicos tradicionales. Entre otras ventajas, los simuladores por ordenador consiguen ahorros importantes de tiempo y de costes de mantenimiento, y permiten que los estudiantes practiquen sus habilidades quirúrgicas en un entorno seguro tantas veces como sea necesario. Sin embargo, a pesar de las capacidades de los ordenadores actuales, la cirugía computacional sigue siendo un campo de investigación exigente. Uno de sus mayores retos es la alta velocidad a la que se tienen que resolver complejos problemas de mecánica de medios continuos para que los interfaces hápticos puedan proporcionar un sentido del tacto realista (en general, se necesitan velocidades de respuesta de 500-1000 Hz).Esta tesis presenta algunos métodos numéricos novedosos para la simulación interactiva de dos procedimientos quirúrgicos habituales: el corte y el rasgado (o desgarro) de tejidos blandos. El marco común de los métodos presentados es el uso de la Descomposición Propia Generalizada (PGD en inglés) para la generación de vademécums computacionales, esto es, metasoluciones generales de problemas paramétricos de altas dimensiones que se pueden evaluar a velocidades de respuesta compatibles con entornos hápticos.En el caso del corte, los vademécums computacionales se utilizan de forma conjunta con técnicas basadas en XFEM, mientras que la carga de cálculo se distribuye entre una etapa off-line (previa a la ejecución interactiva) y otra on-line (en tiempo de ejecución). Durante la fase off-line, para el órgano en cuestión se precalculan tanto un vademécum computacional para cualquier posición de una carga, como los desplazamientos producidos por un conjunto de cortes. Así, durante la etapa on-line, los resultados precalculados se combinan de la forma más adecuada para obtener en tiempo real la respuesta a las acciones dirigidas por el usuario. En cuanto al rasgado, a partir de una ecuación paramétrica basada en mecánica del daño continuo, se obtiene un vademécum computacional. La complejidad del modelo se reduce mediante técnicas de Descomposición Ortogonal Propia (POD en inglés), y el vademécum se incorpora a una formulación incremental explícita que se puede interpretar como una especie de integrador temporal.A modo de ejemplo, el método para el corte se aplica a la simulación de un procedimiento quirúrgico refractivo de la córnea conocido como queratotomía radial, mientras que el método para el rasgado se centra en la simulación de la colecistectomía laparoscópica (la extirpación de la vesícula biliar mediante laparoscopia). En ambos casos, los métodos implementados ofrecen excelentes resultados en términos de velocidades de respuesta y producen simulaciones muy realistas desde los puntos de vista visual y háptico.The real-time computer-based simulation of surgery has proven to be an appealing alternative to traditional surgical simulators. Amongst other advantages, computer-based simulators provide considerable savings on time and maintenance costs, and allow trainees to practice their surgical skills in a safe environment as often as necessary. However, in spite of the current computer capabilities, computational surgery continues to be a challenging field of research. One of its major issues is the high speed at which complex problems in continuum mechanics have to be solved so that haptic interfaces can render a realistic sense of touch (generally, feedback rates of 500–1 000 Hz are required). This thesis introduces some novel numerical methods for the interactive simulation of two usual surgical procedures: cutting and tearing of soft tissues. The common framework of the presented methods is the use of the Proper Generalised Decomposition (PGD) for the generation of computational vademecums, i. e. general meta-solutions of parametric high-dimensional problems that can be evaluated at feedback rates compatible with haptic environments. In the case of cutting, computational vademecums are used jointly with XFEM-based techniques, and the computing workload is distributed into an off-line and an on-line stage. During the off-line stage, both a computational vademecum for any position of a load and the displacements produced by a set of cuts are pre-computed for the organ under consideration. Thus, during the on-line stage, the pre-computed results are properly combined together to obtain in real-time the response to the actions driven by the user. Concerning tearing, a computational vademecum is obtained from a parametric equation based on continuum damage mechanics. The complexity of the model is reduced by Proper Orthogonal Decomposition (POD) techniques, and the vademecum is incorporated into an explicit incremental formulation that can be viewed as a sort of time integrator. By way of example, the cutting method is applied to the simulation of a corneal refractive surgical procedure known as radial keratotomy, whereas the tearing method focuses on the simulation of laparoscopic cholecystectomy (i. e. the removal of the gallbladder). In both cases, the implemented methods offer excellent performances in terms of feedback rates, and produce.<br /

GPU Implementation of extended total Lagrangian explicit (gpuXTLED) for Surgical Incision Application

An extended total Lagrangian explicit dynamic (XTLED) is presented as a potential numerical method for simulating interactive or physics-based surgical incisions of soft tissues. The simulation of surgical incision is vital to the integrity of virtual reality simulators that are used for immersive surgical training. However, most existing numerical methods either compromise on computational speed for accuracy or vice versa. This is due to the challenge of modelling nonlinear behaviour of soft tissues, incorporating incision and subsequently updating topology to account for the incision. To tackle these challenges, XTLED method which combines the extended finite element method (XFEM) using total Lagrangian formulation with explicit time integration method was developed. The algorithm was developed and deformations of 3D geometries under tension, were simulated. An attempt was made to validate the XTLED method using silicon samples with different incision configuration and a comparison was made between XTLED and FEM. Results show that XTLED could potentially be used to simulate interactive soft tissue incision. However, further quantitative verification and validation are required. In addition, numerical analyses conducted show that solutions may not be obtainable due to simulation errors. However, it is unclear whether these errors are inherent in the XTLED method or the algorithm created for the XTLED method in this thesis

Accurate Real-Time Framework for Complex Pre-defined Cuts in Finite Element Modeling

PhD ThesisAchieving detailed pre-defined cuts on deformable materials is vitally pivotal for many commercial applications, such as cutting scenes in games and vandalism effects in virtual movies. In these types of applications, the majority of resources are allocated to achieve high-fidelity representations of materials and the virtual environments. In the case of limited computing resources, it is challenging to achieve a convincing cutting effect. On the premise of sacrificing realism effects or computational cost, a considerable amount of research work has been carried out, but the best solution that can be compatible with both cases has not yet been identified. This doctoral dissertation is dedicated to developing a unique framework for representing pre-defined cuts of deformable surface models, which can achieve real-time, detailed cutting while maintaining the realistic physical behaviours. In order to achieve this goal, we have made in-depth explorations from geometric and numerical perspectives. From a geometric perspective, we propose a robust subdivision mechanism that allows users to make arbitrary predetermined cuts on elastic surface models based on the finite element method (FEM). Specifically, after the user separates the elements in an arbitrary manner (i.e., linear or non-linear) on the topological mesh, we then optimise the resulting mesh by regenerating the triangulation within the element based on the Delaunay triangulation principle. The optimisation of regenerated triangles, as a process of refining the ill-shaped elements that have small Aspect Ratio, greatly improves the realism of physical behaviours and guarantees that the refinement process is balanced with real-time requirements. The above subdivision mechanism can improve the visual effect of cutting, but it neglects the fact that elements cannot be perfectly cut through any pre-defined trajectories. The number of ill-shaped elements generated yield a significant impact on the optimisation time: a large number of ill-shaped elements will render the cutting slow or even collapse, and vice versa. Our idea is based on the core observation that the producing of ill-shaped elements is largely attributed to the condition number of the global stiffness matrix. Practically, for a stiffness matrix, a large condition number means that it is almost singular, and the calculation of its inverse or the solution of a system of linear equations are prone to large numerical errors and time-consuming. It motivates us to alleviate the impact of condition number of the global stiffness matrix from the numerical aspects. Specifically, we address this issue in a novel manner by converting the global stiffness matrix into the form of a covariance matrix, in which the number of conditions of the matrix can be reduced by exploiting appropriate matrix normalisation to the eigenvalues. Furthermore, we investigated the efficiency of two different scenarios: an exact square-root normalisation and its approximation based on the Newton-Schulz iteration. Experimental tests of the proposed framework demonstrate that it can successfully reproduce competitive visuals of detailed pre-defined cuts compared with the state-of-the-art method (Manteaux et al. 2015) while obtaining a significant improvement on the FPS, increasing up to 46.49 FPS and 21.93 FPS during and after the cuts, respectively. Also, the new refinement method can stably maintain the average Aspect Ratio of the model mesh after the cuts at less than 3 and the average Area Ratio around 3%. Besides, the proposed two matrix normalisation strategies, including ES-CGM and AS-CGM, have shown the superiority of time efficiency compared with the baseline method (Xin et al. 2018). Specifically, the ES-CGM and AS-CGM methods obtained 5 FPS and 10 FPS higher than the baseline method, respectively. These experimental results strongly support our conclusion which is that this new framework would provide significant benefits when implemented for achieving detailed pre-defined cuts at a real-time rate

Découpage virtuel interactif de corps élastiques pour simulation chirurgicale

''RÉSUMÉ : La simulation chirurgicale dans un environnement de réalité virtuelle fournit un moyen de pratiquer certaines opérations sans les risques associés à une intervention sur un patient ou le coût d’un mannequin. Afin de générer un sentiment de présence, on cherche à produire un environnement le plus complet possible, incluant une vision 3D, un retour haptique, des interactions crédibles et un comportement physique réaliste des objets présents. La recherche présentée ici porte sur la simulation physique du comportement d’organes mous, comme le foie ou le cerveau, ainsi que sur le découpage de ces organes à l’aide d’un scalpel, une interaction particulièrement difficile à reproduire virtuellement de façon réaliste. L’objectif principal est de développer une méthode de déformation à la fois réaliste et efficace, et de permettre à un utilisateur de découper interactivement un objet simulé par cette méthode, à l’aide d’un outil tranchant virtuel. De plus, nous voulons que la déformation et les interactions soient décrites avec une grande précision, tout en permettant d’effectuer les calculs très rapidement, pour une interaction fluide qui maintient le sentiment de présence.''----------''ABSTRACT : Surgery simulation in a virtual reality environment provides a way to practice certain operations without the risks associated with performing surgery on a patient or the cost of using arealistic dummy. To facilitate immersion, we seek to produce an environment as complete as possible, including 3D vision, haptic feedback, credible interactions and a realistic physical behavior of simulated objects. The research presented in this document focuses on the physical behavior of soft organs, like the brain or liver, and on cutting these organs using a scalpel. It is especially difficult to reproduce virtually that interaction in a realistic way. The main objective is to develop a deformation method that is both realistic and efficient, and to allow a user to interactively cut an object simulated through this method, using a virtual sharp tool. Furthermore, we want the deformation and interactions to be described with high precision while allowing for fast computations, for a smooth interaction that maintains immersion.'

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Real-time Biomechanical Modeling for Intraoperative Soft Tissue Registration

Computer assisted surgery systems intraoperatively support the surgeon by providing information on the location of hidden risk and target structures during surgery. However, soft tissue deformations make intraoperative registration (and thus intraoperative navigation) difficult. In this work, a novel, biomechanics based approach for real-time soft tissue registration from sparse intraoperative sensor data such as stereo endoscopic images is presented to overcome this problem

Physically-based 6-DoF Nodes Deformable Models: Application to Connective Tissues Simulation and Soft-Robots Control

The medical simulation is an increasingly active research field. Yet, despite the promising advance observed over the past years, the complete virtual patient’s model is yet to come. There are still many avenues for improvements, especially concerning the mechanical modeling of boundary conditions on anatomical structures.So far, most of the work has been dedicated to organs simulation, which are generally simulated alone. This raises a real problem as the role of the surrounding organs in the boundary conditions is neglected. However, these interactions can be complex, involving contacts but also mechanical links provided by layers of soft tissues. The latter are known as connective tissues or fasciae. As a consequence, the mutual influences between the anatomical structures are generally simplified, weakening the realism of the simulations.This thesis aims at studying the importance of the connective tissues, and especially of a proper modeling of the boundary conditions. To this end, the role of the ligaments during laparoscopic liver surgery has been investigated. In order to enhance the simulations’ realism, a mechanical model dedicated to the connective tissues has been worked out. This has led to the development of a physically-based method relying on material points that can, not only translate, but also rotate themselves. The goal of this model is to enable the simulation of multiple organs linked by complex interactions.In addition, the work on the connective tissues model has been derived to be used in soft robotics. Indeed, the principle of relying on orientable material points has been used to developed a reduced model that can reproduce the behavior of more complex structures. The objective of this work is to provide the means to control – in real-time – a soft robot made of a deformable arm.La simulation médicale est un domaine de recherche de plus en plus actif. Cependant, malgré les avancées prometteuses observées ces dernières années, le modèle complet du patient virtuel reste un objectif ambitieux. Il existe encore de nombreuses opportunités de recherche, notamment concernant la modélisation mécanique des conditions aux limites des organes.Jusqu'à présent, la majorité des travaux était consacrée à la simulation d'organes, ces derniers étant généralement simulés seuls. Cette situation pose un réel problème car l'influence qu'ont les organes environnants sur les conditions aux limites est négligée. Ces interactions peuvent être complexes, impliquant des contacts mais aussi des liaisons mécaniques dues à des couches de tissus connus sous le nom de tissus conjonctifs ou fasciae. Pour cette raison, les influences mutuelles entre les structures anatomiques sont généralement simplifiées, diminuant le réalisme des simulations.Cette thèse visé à étudier l'importance des tissus conjonctifs, et plus particulièrement d'une bonne modélisation des conditions aux limites. Dans ce but, le rôle des ligaments lors d'une intervention chirurgicale sur la foie par laparoscopie a été étudié. Afin d'améliorer le réalisme des simulations, un modèle mécanique dédié aux tissus conjonctifs a été mis au point. Ainsi, une méthode basée sur la mécanique des milieux continus et un ensemble de nœuds à 6 degrés de liberté a été développée. L'objectif de ce modèle étant de permettre la simulation simultanée de plusieurs organes liés par des interaction complexes.En outre, les travaux sur les tissus conjonctifs ont donné lieu à la mise au point d'une méthode de modélisation utilisée dans le cadre des robots déformables. Cette méthode permet un contrôle précis, et temps-réel, d'un bras robotisé déformable. En effet, l'utilisation de nœuds orientables a permis de développer un modèle a nombre de degrés de liberté réduit, qui permet de reproduire le comportement de structures plus complexes