Towards retrieving force feedback in robotic-assisted surgery: a supervised neuro-recurrent-vision approach

Robotic-assisted minimally invasive surgeries have gained a lot of popularity over conventional procedures as they offer many benefits to both surgeons and patients. Nonetheless, they still suffer from some limitations that affect their outcome. One of them is the lack of force feedback which restricts the surgeon's sense of touch and might reduce precision during a procedure. To overcome this limitation, we propose a novel force estimation approach that combines a vision based solution with supervised learning to estimate the applied force and provide the surgeon with a suitable representation of it. The proposed solution starts with extracting the geometry of motion of the heart's surface by minimizing an energy functional to recover its 3D deformable structure. A deep network, based on a LSTM-RNN architecture, is then used to learn the relationship between the extracted visual-geometric information and the applied force, and to find accurate mapping between the two. Our proposed force estimation solution avoids the drawbacks usually associated with force sensing devices, such as biocompatibility and integration issues. We evaluate our approach on phantom and realistic tissues in which we report an average root-mean square error of 0.02 N.Peer ReviewedPostprint (author's final draft

GPU-based Real-Time Soft Tissue Deformation with Cutting and Haptic Feedback

Special Issue on Biomechanical Modelling of Soft Tissue MotionInternational audienceThis article describes a series of contributions in the field of real-time simulation of soft tissue biomechanics. These contributions address various requirements for interactive simulation of complex surgical procedures. In particular, this article presents results in the areas of soft tissue deformation, contact modelling, simulation of cutting, and haptic rendering, which are all relevant to a variety of medical interventions. The contributions described in this article share a common underlying model of deformation and rely on GPU implementations to significantly improve computation times. This consistency in the modelling technique and computational approach ensures coherent results as well as efficient, robust and flexible solutions

A Heterogeneous and Multi-Range Soft-Tissue Deformation Model for Applications in Adaptive Radiotherapy

During fractionated radiotherapy, anatomical changes result in uncertainties in the applied dose distribution. With increasing steepness of applied dose gradients, the relevance of patient deformations increases. Especially in proton therapy, small anatomical changes in the order of millimeters can result in large range uncertainties and therefore in substantial deviations from the planned dose. To quantify the anatomical changes, deformation models are required. With upcoming MR-guidance, the soft-tissue deformations gain visibility, but so far only few soft-tissue models meeting the requirements of high-precision radiotherapy exist. Most state-of-the-art models either lack anatomical detail or exhibit long computation times. In this work, a fast soft-tissue deformation model is developed which is capable of considering tissue properties of heterogeneous tissue. The model is based on the chainmail (CM)-concept, which is improved by three basic features. For the first time, rotational degrees of freedom are introduced into the CM-concept to improve the characteristic deformation behavior. A novel concept for handling multiple deformation initiators is developed to cope with global deformation input. And finally, a concept for handling various shapes of deformation input is proposed to provide a high flexibility concerning the design of deformation input. To demonstrate the model flexibility, it was coupled to a kinematic skeleton model for the head and neck region, which provides anatomically correct deformation input for the bones. For exemplary patient CTs, the combined model was shown to be capable of generating artificially deformed CT images with realistic appearance. This was achieved for small-range deformations in the order of interfractional deformations, as well as for large-range deformations like an arms-up to arms-down deformation, as can occur between images of different modalities. The deformation results showed a strong improvement in biofidelity, compared to the original chainmail-concept, as well as compared to clinically used image-based deformation methods. The computation times for the model are in the order of 30 min for single-threaded calculations, by simple code parallelization times in the order of 1 min can be achieved. Applications that require realistic forward deformations of CT images will benefit from the improved biofidelity of the developed model. Envisioned applications are the generation of plan libraries and virtual phantoms, as well as data augmentation for deep learning approaches. Due to the low computation times, the model is also well suited for image registration applications. In this context, it will contribute to an improved calculation of accumulated dose, as is required in high-precision adaptive radiotherapy

Biomechanics

Biomechanics is a vast discipline within the field of Biomedical Engineering. It explores the underlying mechanics of how biological and physiological systems move. It encompasses important clinical applications to address questions related to medicine using engineering mechanics principles. Biomechanics includes interdisciplinary concepts from engineers, physicians, therapists, biologists, physicists, and mathematicians. Through their collaborative efforts, biomechanics research is ever changing and expanding, explaining new mechanisms and principles for dynamic human systems. Biomechanics is used to describe how the human body moves, walks, and breathes, in addition to how it responds to injury and rehabilitation. Advanced biomechanical modeling methods, such as inverse dynamics, finite element analysis, and musculoskeletal modeling are used to simulate and investigate human situations in regard to movement and injury. Biomechanical technologies are progressing to answer contemporary medical questions. The future of biomechanics is dependent on interdisciplinary research efforts and the education of tomorrow’s scientists

SOFA: A Multi-Model Framework for Interactive Physical Simulation

International audienceSOFA (Simulation Open Framework Architecture) is an open-source C++ library primarily targeted at interactive computational medical simulation. SOFA facilitates collaborations between specialists from various domains, by decomposing complex simulators into components designed independently and organized in a scenegraph data structure. Each component encapsulates one of the aspects of a simulation, such as the degrees of freedom, the forces and constraints, the differential equations, the main loop algorithms, the linear solvers, the collision detection algorithms or the interaction devices. The simulated objects can be represented using several models, each of them optimized for a different task such as the computation of internal forces, collision detection, haptics or visual display. These models are synchronized during the simulation using a mapping mechanism. CPU and GPU implementations can be transparently combined to exploit the computational power of modern hardware architectures. Thanks to this flexible yet efficient architecture, \sofa{} can be used as a test-bed to compare models and algorithms, or as a basis for the development of complex, high-performance simulators

The Development of a Viscoelastic Ellipsoidal Model for use in Measuring Plantar Tissue Material Properties during Walking

Introduction: The mechanical characteristics of the plantar tissues during walking is not well understood as most of the current research focuses on testing specific plantar regions in cadavers or while the feet of the participants are raised. In this work, it is hypothesized that a viscoelastic geometric ellipsoid model used to assess multiple structures of the foot would be accurate and robust. This model would be participant-specific and applicable to the entire stance phase of gait. Methods: The proposed viscoelastic ellipsoid model would represent several key anatomical areas: Heel, Posterior Midfoot, Anterior Midfoot, Metatarsals 1-2, Metatarsals 3-5, Toe 1, Toe 2, and Toes 3-5. The ellipsoid model required measurement of force and contact area simultaneously. This was done using pressure-measuring insoles (Medilogic ®, Schönefeld, Germany), worn by multiple, college-aged participants. The insole force and area data were used to optimize the model for each participant as the material properties and geometry of each participant’s foot will differ. Results: The results of the model application was able to show that the ellipsoid model was fairly successful in producing the ground reaction force during walking. Further, the ellipsoid model was able to characterize stiffness and damping results, that were different for all the plantar regions. These results were also different from previous research that used data from mechanical tests and experiments where the participant’s foot was static. Conclusion: The viscoelastic ellipsoidal model was able to reproduce ground reaction force and determine the unique mechanical characteristics for each plantar region. Future uses of the model will be with clinical data collected from persons with plantar diseases, which could lead to predictions and preventions of plantar disease

A fibre-reinforced poroviscoelastic finite element model for the Achilles tendon

Tendons connect muscles to bone and consist of a complex structure of almost parallel collagen fibres embedded in a hydrated matrix. The mechanical behaviour of tendons is viscoelastic and highly non-linear. The Achilles tendon is the largest tendon in the body and it is essential for walking and jumping. It is the most frequently ruptured tendon in humans and today there is no consensus on what is the best treatment. To be able to improve and develop suitable treatments for tendon injuries more knowledge in tendon mechanics is necessary. Computational models can increase the understanding of tendon mechanics and also be used to predict tendon rupture or the mechanical response in unknown loading situations. There are different models used for tendons today but most of them provide a mathematical description of the mechanical behaviour without any direct coupling to the physiological structure of tendons. In this study, a structural fibre-reinforced poroviscoelastic finite element model is developed for the Achilles tendon, based on a model for articular cartilage. It is assumed that the collagen fibres, the non-fibrillar matrix and the fluid flow contribute to the total stress in the tendon and specific constitutive models are used for the different components. The fibres are modelled as non-linear viscoelastic one dimensional units, the surrounding matrix is modelled as a neo-Hookean material and the permeability is assumed to be strain-dependent. The model is curve fitted to experimental test data from rat Achilles tendons subjected to cyclic loading by optimizing nine material parameters. In total, the model is curve fitted to 21 specimens from two different data sets and two average tendons representing each set. It was found that an exponential stress-strain relationship for the collagen fibres was necessary to capture the non-linear behaviour of the Achilles tendon. With this exponential formulation, very good curve fits were obtained for all specimens from both groups and for the average tendons. Statistical analyses show that the optimized material parameters from the two data sets are not significantly different. The proposed model can accurately simulate the Achilles tendons under cyclic loading and it identifies the collagen fibres to be the most important load bearing structural parts in tensile loading. This model also looks promising for predicting the general mechanical behaviour of the Achilles tendon under other similar loading conditions

Design of a pneumatic soft robotic actuator using model-based optimization

In this thesis, the design and optimization process of a novel soft intelligent modular pad (IntelliPad) for the purpose of pressure injury prevention is presented. The structure of the IntelliPad consists of multiple individual multi-chamber soft pneumatic-driven actuators that use pressurized air and vacuum. Each actuator is able to provide both vertical and horizontal motions that can be controlled independently. An analytical modeling approach using multiple cantilever beams and virtual springs connected in a closed formed structure was developed to analyze the mechanical performance of the actuator. The analytical approach was validated by a finite element analysis. For optimizing the actuator\u27s mechanical performance, firefly algorithm and deep reinforcement learning-based design optimization frameworks were developed with the purpose of maximizing the horizontal motion of the top surface of the actuators, while minimizing its corresponding effect on the vertical motion. Four optimized designs were fabricated. The actuators were tested and validated experimentally to demonstrate their required mechanical performance in order to regulate normal and shear stresses at the skin-pad interface for pressure injury prevention applications

A physically based trunk soft tissue modeling for scoliosis surgery planning systems

One of the major concerns of scoliotic patients undergoing spinal correction surgery is the trunk's external appearance after the surgery. This paper presents a novel incremental approach for simulating postoperative trunk shape in scoliosis surgery. Preoperative and postoperative trunk shapes data were obtained using three-dimensional medical imaging techniques for seven patients with adolescent idiopathic scoliosis. Results of qualitative and quantitative evaluations, based on the comparison of the simulated and actual postoperative trunk surfaces, showed an adequate accuracy of the method. Our approach provides a candidate simulation tool to be used in a clinical environment for the surgery planning process.IRSC / CIH

A virtual environment for the modelling, simulation and manufacturing of orthopaedic devices

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The objective of this work is to investigate whether the game physics based modelling is accurate enough to be used in modelling the motion of the human body, in particular musculoskeletal motion. Hitherto, the implementation of game physics in the medical field focused only on anatomical representation for education and training purposes. Introducing gaming platforms and physics engines into orthopaedics applications will help to overcome several difficulties encountered in the modelling of articular joints. Implementing a physics engine (PhysX), which is mainly designed for video games, handles intensive computations in optimized ways at an interactive speed. In this study, the capabilities of the physics engine (PhysX) and gaming platform for modelling and simulating articular joints are evaluated. First, a preliminary validation is carried out for mechanical systems with analytical solutions, before constructing the musculoskeletal model to evaluate the consistency of gaming platforms. The developed musculoskeletal model deals with the human joint as an unconstrained system with 6 DOF which is not available with other joint modeller. The model articulation is driven by contact surfaces and the stiffness of surrounding tissues. A number of contributions, such as contact modelling and muscle wrapping, have been made in this research to overcome some existing challenges in joint modelling. Using muscle segmentation, the proposed technique effectively handles the problem of muscle wrapping, a major concern for many; thus the shortest path and line of action are no longer problematic. Collision behaviour has also shown a stable response for colliding as well as resting objects, provided that it is based on the principles of surface properties and the conservation of linear and angular momentums. The precision of collision detection and response are within an acceptable tolerance controllable by varying the mesh density. An image based analysis system is developed in this thesis, mainly in order to validate the proposed physics based modelling solution. This minimally invasive method is based on the analysis of marker positions located at bony positions with minimal skin movement. The image based system overcomes several challenges associated with the currently existing methods, such as inaccuracy, complication, impracticability and cost. The analysis part of this research has considered the elbow joint as a case study to investigate and validate the proposed physics based model. Beside the interactive 3D simulation, the obtained results are validated by comparing them with the image based system developed within the current research to investigate joint kinematics and laxity and also with published material, MJM and results from experiments performed at the Brunel Orthopaedic Research and Learning Centre. The proposed modelling shows the advantageous speed, reliability and flexibility of the proposed model. It is shown that the gaming platform and physics engine provide a viable solution to human musculoskeletal modelling. Finally, this thesis considers an extended implementation of the proposed platform for testing and assessing the design of custom-made implants, to enhance joint performance. The developed simulation software is expected to give indicative results as well as testing different types of prosthetic implant. Design parameterization and sensitivity analysis for geometrical features are discussed. Thus, an integrated environment is proposed to link the real-time simulation software with a manufacturing environment so as to assist the production of patient specific implants by rapid manufacturing