Research on real-time physics-based deformation for haptic-enabled medical simulation

This study developed a multiple effective visuo-haptic surgical engine to handle a variety of surgical manipulations in real-time. Soft tissue models are based on biomechanical experiment and continuum mechanics for greater accuracy. Such models will increase the realism of future training systems and the VR/AR/MR implementations for the operating room

Physically Based Forehead Modelling and Animation including Wrinkles

There has been a vast amount of research on the production of realistic facial models and animations, which is one of the most challenging areas of computer graphics. Recently, there has been an increased interest in the use of physically based approaches for facial animation, whereby the effects of muscle contractions are propagated through facial soft-tissue models to automatically deform them in a more realistic and anatomically accurate manner. Presented in this thesis is a fully physically based approach for efficiently producing realistic-looking animations of facial movement, including animation of expressive wrinkles, focussing on the forehead. This is done by modelling more physics-based behaviour than current computer graphics approaches. The presented research has two major components. The first is a novel model creation process to automatically create animatable non-conforming hexahedral finite element (FE) simulation models of facial soft tissue from any surface mesh that contains hole-free volumes. The generated multi-layered voxel-based models are immediately ready for simulation, with skin layers and element material properties, muscle properties, and boundary conditions being automatically computed. The second major component is an advanced optimised GPU-based process to simulate and visualise these models over time using the total Lagrangian explicit dynamic (TLED) formulation of the FE method. An anatomical muscle contraction model computes active and transversely isotropic passive muscle stresses, while advanced boundary conditions enable the sliding effect between the superficial and deep soft-tissue layers to be simulated. Soft-tissue models and animations with varying complexity are presented, from a simple soft-tissue-block model with uniform layers of skin and muscle, to a complex forehead model. These demonstrate the flexibility of the animation approach to produce detailed animations of realistic gross- and fine-scale soft-tissue movement, including wrinkles, with different muscle structures and material parameters, for example, to animate different-aged skin. Owing to the detail and accuracy of the models and simulations, the animation approach could also be used for applications outside of computer graphics, such as surgical applications. Furthermore, the animation approach can be used to animate any multi-layered soft body (not just soft tissue)

Book of Abstracts 15th International Symposium on Computer Methods in Biomechanics and Biomedical Engineering and 3rd Conference on Imaging and Visualization

In this edition, the two events will run together as a single conference, highlighting the strong connection with the Taylor & Francis journals: Computer Methods in Biomechanics and Biomedical Engineering (John Middleton and Christopher Jacobs, Eds.) and Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization (JoãoManuel R.S. Tavares, Ed.). The conference has become a major international meeting on computational biomechanics, imaging andvisualization. In this edition, the main program includes 212 presentations. In addition, sixteen renowned researchers will give plenary keynotes, addressing current challenges in computational biomechanics and biomedical imaging. In Lisbon, for the first time, a session dedicated to award the winner of the Best Paper in CMBBE Journal will take place. We believe that CMBBE2018 will have a strong impact on the development of computational biomechanics and biomedical imaging and visualization, identifying emerging areas of research and promoting the collaboration and networking between participants. This impact is evidenced through the well-known research groups, commercial companies and scientific organizations, who continue to support and sponsor the CMBBE meeting series. In fact, the conference is enriched with five workshops on specific scientific topics and commercial software.info:eu-repo/semantics/draf

NON-RIGID BODY MECHANICAL PROPERTY RECOVERY FROM IMAGES AND VIDEOS

Material property has great importance in surgical simulation and virtual reality. The mechanical properties of the human soft tissue are critical to characterize the tissue deformation of each patient. Studies have shown that the tissue stiffness described by the tissue properties may indicate abnormal pathological process. The (recovered) elasticity parameters can assist surgeons to perform better pre-op surgical planning and enable medical robots to carry out personalized surgical procedures. Traditional elasticity parameters estimation methods rely largely on known external forces measured by special devices and strain field estimated by landmarks on the deformable bodies. Or they are limited to mechanical property estimation for quasi-static deformation. For virtual reality applications such as virtual try-on, garment material capturing is of equal significance as the geometry reconstruction. In this thesis, I present novel approaches for automatically estimating the material properties of soft bodies from images or from a video capturing the motion of the deformable body. I use a coupled simulation-optimization-identification framework to deform one soft body at its original, non-deformed state to match the deformed geometry of the same object in its deformed state. The optimal set of material parameters is thereby determined by minimizing the error metric function. This method can simultaneously recover the elasticity parameters of multiple regions of soft bodies using Finite Element Method-based simulation (of either linear or nonlinear materials undergoing large deformation) and particle-swarm optimization methods. I demonstrate the effectiveness of this approach on real-time interaction with virtual organs in patient-specific surgical simulation, using parameters acquired from low-resolution medical images. With the recovered elasticity parameters and the age of the prostate cancer patients as features, I build a cancer grading and staging classifier. The classifier achieves up to 91% for predicting cancer T-Stage and 88% for predicting Gleason score. To recover the mechanical properties of soft bodies from a video, I propose a method which couples statistical graphical model with FEM simulation. Using this method, I can recover the material properties of a soft ball from a high-speed camera video that captures the motion of the ball. Furthermore, I extend the material recovery framework to fabric material identification. I propose a novel method for garment material extraction from a single-view image and a learning based cloth material recovery method from a video recording the motion of the cloth. Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, I propose a method that can compute a 3D model of a human body and its outfit from a single photograph with little human interaction. My proposed learning-based cloth material type recovery method exploits simulated data-set and deep neural network. I demonstrate the effectiveness of my algorithms by re-purposing the reconstructed garments for virtual try-on, garment transfer, and cloth animation on digital characters. With the recovered mechanical properties, one can construct a virtual world with soft objects exhibiting real-world behaviors.Doctor of Philosoph

Physically-based 6-DoF Nodes Deformable Models: Application to Connective Tissues Simulation and Soft-Robots Control

The medical simulation is an increasingly active research field. Yet, despite the promising advance observed over the past years, the complete virtual patient’s model is yet to come. There are still many avenues for improvements, especially concerning the mechanical modeling of boundary conditions on anatomical structures.So far, most of the work has been dedicated to organs simulation, which are generally simulated alone. This raises a real problem as the role of the surrounding organs in the boundary conditions is neglected. However, these interactions can be complex, involving contacts but also mechanical links provided by layers of soft tissues. The latter are known as connective tissues or fasciae. As a consequence, the mutual influences between the anatomical structures are generally simplified, weakening the realism of the simulations.This thesis aims at studying the importance of the connective tissues, and especially of a proper modeling of the boundary conditions. To this end, the role of the ligaments during laparoscopic liver surgery has been investigated. In order to enhance the simulations’ realism, a mechanical model dedicated to the connective tissues has been worked out. This has led to the development of a physically-based method relying on material points that can, not only translate, but also rotate themselves. The goal of this model is to enable the simulation of multiple organs linked by complex interactions.In addition, the work on the connective tissues model has been derived to be used in soft robotics. Indeed, the principle of relying on orientable material points has been used to developed a reduced model that can reproduce the behavior of more complex structures. The objective of this work is to provide the means to control – in real-time – a soft robot made of a deformable arm.La simulation médicale est un domaine de recherche de plus en plus actif. Cependant, malgré les avancées prometteuses observées ces dernières années, le modèle complet du patient virtuel reste un objectif ambitieux. Il existe encore de nombreuses opportunités de recherche, notamment concernant la modélisation mécanique des conditions aux limites des organes.Jusqu'à présent, la majorité des travaux était consacrée à la simulation d'organes, ces derniers étant généralement simulés seuls. Cette situation pose un réel problème car l'influence qu'ont les organes environnants sur les conditions aux limites est négligée. Ces interactions peuvent être complexes, impliquant des contacts mais aussi des liaisons mécaniques dues à des couches de tissus connus sous le nom de tissus conjonctifs ou fasciae. Pour cette raison, les influences mutuelles entre les structures anatomiques sont généralement simplifiées, diminuant le réalisme des simulations.Cette thèse visé à étudier l'importance des tissus conjonctifs, et plus particulièrement d'une bonne modélisation des conditions aux limites. Dans ce but, le rôle des ligaments lors d'une intervention chirurgicale sur la foie par laparoscopie a été étudié. Afin d'améliorer le réalisme des simulations, un modèle mécanique dédié aux tissus conjonctifs a été mis au point. Ainsi, une méthode basée sur la mécanique des milieux continus et un ensemble de nœuds à 6 degrés de liberté a été développée. L'objectif de ce modèle étant de permettre la simulation simultanée de plusieurs organes liés par des interaction complexes.En outre, les travaux sur les tissus conjonctifs ont donné lieu à la mise au point d'une méthode de modélisation utilisée dans le cadre des robots déformables. Cette méthode permet un contrôle précis, et temps-réel, d'un bras robotisé déformable. En effet, l'utilisation de nœuds orientables a permis de développer un modèle a nombre de degrés de liberté réduit, qui permet de reproduire le comportement de structures plus complexes