Development of a Reality-Based, Haptics-Enabled Simulator for Tool-Tissue Interactions

The advent of complex surgical procedures has driven the need for finite element based surgical training simulators which provide realistic visual and haptic feedback throughout the surgical task. The foundation of a simulator stems from the use of accurate, reality-based models for the global tissue response as well as the tool-tissue interactions. To that end, ex vivo and in vivo tests were conducted for soft-tissue probing and in vivo tests were conducted for soft-tissue cutting for the purpose of model development. In formulating a surgical training system, there is a desire to replicate the surgical task as accurately as possible for haptic and visual realism. However, for many biological tissues, there is a discrepancy between the mechanical characteristics of ex vivo and in vivo tissue. The efficacy of utilizing an ex vivo model for simulation of in vivo probing tasks on porcine liver was evaluated by comparing the simulated probing task to an identical in vivo probing experiment. The models were then further improved upon to better replicate the in vivo response. During the study of cutting modeling, in vivo cutting experiments were performed on porcine liver to derive the force-displacement response of the tissue to a scalpel blade. Using this information, a fracture mechanics based approach was applied to develop a fully defined cohesive zone model governing the separation properties of the liver directly in front of the scalpel blade. Further, a method of scaling the cohesive zone parameters was presented to minimize the computational expense in an effort to apply the cohesive based cutting approach to real-time simulators. The development of the models for the global tissue response and local tool-tissue interactions for probing and cutting of soft-tissue provided the framework for real-time simulation of basic surgical skills training. Initially, a pre-processing approach was used for the development of reality-based, haptics enabled simulators for probing and cutting of soft tissue. Then a real-time finite element based simulator was developed to simulate the probing task without the need to know the tool path prior to simulation

Real-time modelling and visualization of soft tissue thermomechanical behaviour for radiofrequency thermal ablation

A review of current literature indicates a limited knowledge and documentation of thermomechanical response of soft tissue during Minimally Invasive Surgical (MIS) hyperthermia procedures such as Radiofrequency Thermal Ablation (RFA). Furthermore, current models and simulations have not accounted for the temperature-dependence of the stress-strain behaviour of soft tissue. The only quantified data for temperature-dependent stress-strain relationships in literature is yielded from Xu and Lu (2009) and Xu, Seffen and Lu (2008a). As well as this, hardware-accelerated (by use of Graphics Processing Units (GPUs)) heat transfer simulations of RFA had not been documented prior to commencement of this project, and the first conference paper announcing this achievement was published in April of 2014 following research and implementation by NE Scientific LLC. A computational three-dimensional (3D) simulated virtual model of liver tissue is developed to establish temperature distributions resulting from single point temperature sources in emulation of the RFA heat treatment procedure. The temperature distribution in the virtual tissue domain is produced by Finite Element (FEM) spatial discretization and Finite Difference (FDM) temporal discretization of the Pennes bioheat transfer equation. The modelled temperature distribution is utilized to determine the degree of transient thermal damage to the virtual tissue based on the Arrhenius Burn Integration. Furthermore, the temperature distribution is used in conjunction with linear thermal expansion to model thermal strains and thermal stresses within the virtual tissue, resulting from the heat source. Novel work is undertaken in producing a thermal stress profile of virtual liver tissue under operational temperatures based on temperature-dependent material stress-strain. The simulation of tissue bioheat transfer and thermal damage is developed in C++ and the High-Level Shader Language (HLSL) with Microsoft’s Direct3D11 Application Programming Interface (API), where the numerical solution process is parallelized and accelerated in performance upon an NVIDIA GTX 770M GPU far beyond its performance upon a single thread/core of an Intel® Core™ i7-4700MQ Central Processing Unit (CPU). A maximum mesh resolution is determined for producing visual real-time post-processing output data based on the GPU accelerated simulation performance. A commercial FEM software package (LISA) is used for determining thermal strain and thermal stress distributions from the temperature distribution data. Examination of simulation results when comparing tissue thermomechanical response for temperature-independent and temperature-dependent stress-strain relationships indicates a dramatic difference in magnitude and distribution of the thermally-induced stresses within the soft tissue. The implications are that RFA simulations must account for this stress-strain temperature-dependence in order to produce remotely accurate stress and strain distributions (both thermomechanical and mechanical) due to the behavioural response of the collagenous biological soft tissue. Furthermore, GPU acceleration is highly recommended for RFA simulation, as the visual real-time maximum mesh resolution far surpasses that of real-time performed on a single modern CPU core

Virtual clinical trials in medical imaging: a review

The accelerating complexity and variety of medical imaging devices and methods have outpaced the ability to evaluate and optimize their design and clinical use. This is a significant and increasing challenge for both scientific investigations and clinical applications. Evaluations would ideally be done using clinical imaging trials. These experiments, however, are often not practical due to ethical limitations, expense, time requirements, or lack of ground truth. Virtual clinical trials (VCTs) (also known as in silico imaging trials or virtual imaging trials) offer an alternative means to efficiently evaluate medical imaging technologies virtually. They do so by simulating the patients, imaging systems, and interpreters. The field of VCTs has been constantly advanced over the past decades in multiple areas. We summarize the major developments and current status of the field of VCTs in medical imaging. We review the core components of a VCT: computational phantoms, simulators of different imaging modalities, and interpretation models. We also highlight some of the applications of VCTs across various imaging modalities

Brain-Inspired Computing

This open access book constitutes revised selected papers from the 4th International Workshop on Brain-Inspired Computing, BrainComp 2019, held in Cetraro, Italy, in July 2019. The 11 papers presented in this volume were carefully reviewed and selected for inclusion in this book. They deal with research on brain atlasing, multi-scale models and simulation, HPC and data infra-structures for neuroscience as well as artificial and natural neural architectures

A virtual environment for the modelling, simulation and manufacturing of orthopaedic devices

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The objective of this work is to investigate whether the game physics based modelling is accurate enough to be used in modelling the motion of the human body, in particular musculoskeletal motion. Hitherto, the implementation of game physics in the medical field focused only on anatomical representation for education and training purposes. Introducing gaming platforms and physics engines into orthopaedics applications will help to overcome several difficulties encountered in the modelling of articular joints. Implementing a physics engine (PhysX), which is mainly designed for video games, handles intensive computations in optimized ways at an interactive speed. In this study, the capabilities of the physics engine (PhysX) and gaming platform for modelling and simulating articular joints are evaluated. First, a preliminary validation is carried out for mechanical systems with analytical solutions, before constructing the musculoskeletal model to evaluate the consistency of gaming platforms. The developed musculoskeletal model deals with the human joint as an unconstrained system with 6 DOF which is not available with other joint modeller. The model articulation is driven by contact surfaces and the stiffness of surrounding tissues. A number of contributions, such as contact modelling and muscle wrapping, have been made in this research to overcome some existing challenges in joint modelling. Using muscle segmentation, the proposed technique effectively handles the problem of muscle wrapping, a major concern for many; thus the shortest path and line of action are no longer problematic. Collision behaviour has also shown a stable response for colliding as well as resting objects, provided that it is based on the principles of surface properties and the conservation of linear and angular momentums. The precision of collision detection and response are within an acceptable tolerance controllable by varying the mesh density. An image based analysis system is developed in this thesis, mainly in order to validate the proposed physics based modelling solution. This minimally invasive method is based on the analysis of marker positions located at bony positions with minimal skin movement. The image based system overcomes several challenges associated with the currently existing methods, such as inaccuracy, complication, impracticability and cost. The analysis part of this research has considered the elbow joint as a case study to investigate and validate the proposed physics based model. Beside the interactive 3D simulation, the obtained results are validated by comparing them with the image based system developed within the current research to investigate joint kinematics and laxity and also with published material, MJM and results from experiments performed at the Brunel Orthopaedic Research and Learning Centre. The proposed modelling shows the advantageous speed, reliability and flexibility of the proposed model. It is shown that the gaming platform and physics engine provide a viable solution to human musculoskeletal modelling. Finally, this thesis considers an extended implementation of the proposed platform for testing and assessing the design of custom-made implants, to enhance joint performance. The developed simulation software is expected to give indicative results as well as testing different types of prosthetic implant. Design parameterization and sensitivity analysis for geometrical features are discussed. Thus, an integrated environment is proposed to link the real-time simulation software with a manufacturing environment so as to assist the production of patient specific implants by rapid manufacturing

Appearance Modelling and Reconstruction for Navigation in Minimally Invasive Surgery

Minimally invasive surgery is playing an increasingly important role for patient care. Whilst its direct patient benefit in terms of reduced trauma, improved recovery and shortened hospitalisation has been well established, there is a sustained need for improved training of the existing procedures and the development of new smart instruments to tackle the issue of visualisation, ergonomic control, haptic and tactile feedback. For endoscopic intervention, the small field of view in the presence of a complex anatomy can easily introduce disorientation to the operator as the tortuous access pathway is not always easy to predict and control with standard endoscopes. Effective training through simulation devices, based on either virtual reality or mixed-reality simulators, can help to improve the spatial awareness, consistency and safety of these procedures. This thesis examines the use of endoscopic videos for both simulation and navigation purposes. More specifically, it addresses the challenging problem of how to build high-fidelity subject-specific simulation environments for improved training and skills assessment. Issues related to mesh parameterisation and texture blending are investigated. With the maturity of computer vision in terms of both 3D shape reconstruction and localisation and mapping, vision-based techniques have enjoyed significant interest in recent years for surgical navigation. The thesis also tackles the problem of how to use vision-based techniques for providing a detailed 3D map and dynamically expanded field of view to improve spatial awareness and avoid operator disorientation. The key advantage of this approach is that it does not require additional hardware, and thus introduces minimal interference to the existing surgical workflow. The derived 3D map can be effectively integrated with pre-operative data, allowing both global and local 3D navigation by taking into account tissue structural and appearance changes. Both simulation and laboratory-based experiments are conducted throughout this research to assess the practical value of the method proposed

High-performance geometric vascular modelling

Image-based high-performance geometric vascular modelling and reconstruction is an essential component of computer-assisted surgery on the diagnosis, analysis and treatment of cardiovascular diseases. However, it is an extremely challenging task to efficiently reconstruct the accurate geometric structures of blood vessels out of medical images. For one thing, the shape of an individual section of a blood vessel is highly irregular because of the squeeze of other tissues and the deformation caused by vascular diseases. For another, a vascular system is a very complicated network of blood vessels with different types of branching structures. Although some existing vascular modelling techniques can reconstruct the geometric structure of a vascular system, they are either time-consuming or lacking sufficient accuracy. What is more, these techniques rarely consider the interior tissue of the vascular wall, which consists of complicated layered structures. As a result, it is necessary to develop a better vascular geometric modelling technique, which is not only of high performance and high accuracy in the reconstruction of vascular surfaces, but can also be used to model the interior tissue structures of the vascular walls.This research aims to develop a state-of-the-art patient-specific medical image-based geometric vascular modelling technique to solve the above problems. The main contributions of this research are:- Developed and proposed the Skeleton Marching technique to reconstruct the geometric structures of blood vessels with high performance and high accuracy. With the proposed technique, the highly complicated vascular reconstruction task is reduced to a set of simple localised geometric reconstruction tasks, which can be carried out in a parallel manner. These locally reconstructed vascular geometric segments are then combined together using shape-preserving blending operations to faithfully represent the geometric shape of the whole vascular system.- Developed and proposed the Thin Implicit Patch method to realistically model the interior geometric structures of the vascular tissues. This method allows the multi-layer interior tissue structures to be embedded inside the vascular wall to illustrate the geometric details of the blood vessel in real world

Experimental validation of a spectroscopic Monte Carlo light transport simulation technique and Raman scattering depth sensing analysis in biological tissue

ABSTRACT: Significance: Raman spectroscopy (RS) applied to surgical guidance is attracting attention among scientists in biomedical optics. Offering a computational platform for studying depthresolved RS and probing molecular specificity of different tissue layers is of crucial importance to increase the precision of these techniques and facilitate their clinical adoption. Aim: The aim of this work was to present a rigorous analysis of inelastic scattering depth sampling and elucidate the relationship between sensing depth of the Raman effect and optical properties of the tissue under interrogation. Approach: A new Monte Carlo (MC) package was developed to simulate absorption, fluorescence, elastic, and inelastic scattering of light in tissue. The validity of the MC algorithm was demonstrated by comparison with experimental Raman spectra in phantoms of known optical properties using nylon and polydimethylsiloxane as Raman-active compounds. A series of MC simulations were performed to study the effects of optical properties on Raman sensing depth for an imaging geometry consistent with single-point detection using a handheld fiber optics probe system. Results: The MC code was used to estimate the Raman sensing depth of a handheld fiber optics system. For absorption and reduced scattering coefficients of 0.001 and 1 mm−1, the sensing depth varied from 105 to 225 μm for a range of Raman probabilities from 10−6 to 10−3. Further, for a realistic Raman probability of 10−6, the sensing depth ranged between 10 and 600 μm for the range of absorption coefficients 0.001 to 1.4 mm−1 and reduced scattering coefficients of 0.5 to 30 mm−1. Conclusions: A spectroscopic MC light transport simulation platform was developed and validated against experimental measurements in tissue phantoms and used to predict depth sensing in tissue. It is hoped that the current package and reported results provide the research community with an effective simulating tool to improve the development of clinical applications of RS

Translating computational modelling tools for clinical practice in congenital heart disease

Increasingly large numbers of medical centres worldwide are equipped with the means to acquire 3D images of patients by utilising magnetic resonance (MR) or computed tomography (CT) scanners. The interpretation of patient 3D image data has significant implications on clinical decision-making and treatment planning. In their raw form, MR and CT images have become critical in routine practice. However, in congenital heart disease (CHD), lesions are often anatomically and physiologically complex. In many cases, 3D imaging alone can fail to provide conclusive information for the clinical team. In the past 20-30 years, several image-derived modelling applications have shown major advancements. Tools such as computational fluid dynamics (CFD) and virtual reality (VR) have successfully demonstrated valuable uses in the management of CHD. However, due to current software limitations, these applications have remained largely isolated to research settings, and have yet to become part of clinical practice. The overall aim of this project was to explore new routes for making conventional computational modelling software more accessible for CHD clinics. The first objective was to create an automatic and fast pipeline for performing vascular CFD simulations. By leveraging machine learning, a solution was built using synthetically generated aortic anatomies, and was seen to be able to predict 3D aortic pressure and velocity flow fields with comparable accuracy to conventional CFD. The second objective was to design a virtual reality (VR) application tailored for supporting the surgical planning and teaching of CHD. The solution was a Unity-based application which included numerous specialised tools, such as mesh-editing features and online networking for group learning. Overall, the outcomes of this ongoing project showed strong indications that the integration of VR and CFD into clinical settings is possible, and has potential for extending 3D imaging and supporting the diagnosis, management and teaching of CHD