VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR

Vision and haptic are two most important modalities in a medical simulation. While visual cues assist one to see his actions when performing a medical procedure, haptic cues enable feeling the object being manipulated during the interaction. Despite their importance in a computer simulation, the combination of both modalities has not been adequately assessed, especially that in a haptic dominant environment. Thus, resulting in poor emphasis in resource allocation management in terms of effort spent in rendering the two modalities for simulators with realistic real-time interactions. Addressing this problem requires an investigation on whether a single modality (haptic) or a combination of both visual and haptic could be better for learning skills in a haptic dominant environment such as in a palpation simulator. However, before such an investigation could take place one main technical implementation issue in visio-haptic rendering needs to be addresse

VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR

Vision and haptic are two most important modalities in a medical simulation. While visual cues assist one to see his actions when performing a medical procedure, haptic cues enable feeling the object being manipulated during the interaction. Despite their importance in a computer simulation, the combination of both modalities has not been adequately assessed, especially that in a haptic dominant environment. Thus, resulting in poor emphasis in resource allocation management in terms of effort spent in rendering the two modalities for simulators with realistic real-time interactions. Addressing this problem requires an investigation on whether a single modality (haptic) or a combination of both visual and haptic could be better for learning skills in a haptic dominant environment such as in a palpation simulator. However, before such an investigation could take place one main technical implementation issue in visio-haptic rendering needs to be addresse

Real-time measurement corrected prediction of soft tissue response for medical simulations

Medical simulators, such as in palpation and disease diagnosis, require an efficient model of the biological soft tissue deformation. Hence, a computationally fast and accurate algorithm is required to support and enhance user interactions in near real-time simulations. The visual accuracy of such simulators is dependent on the user¿s reaction time. Static visual images that update at a rate of 25 Hz are perceived as real-time moving images. Hence, visualizing software requires fast algorithms to compute the deformation of soft tissue to facilitate a meaningful simulation. Furthermore, soft tissue behaviour should be modelled accurately while compatible with real-time computation. This work proposes a fast solver for the linearized finite element method (FEM) and validates the proposed algorithm with experimental results. The novelty of the method lies in the utilization of real-time force/displacement measurements that are embedded in the solution via the Kalman filter. A novel computational algorithm that utilizes the strength of the FEM in terms of accuracy and employs direct measurements from the manipulated tissue to overcome the slow computational process of the FEM is proposed in the first part of the thesis. As the behaviour of the mechanically loaded tissue can be regarded as linearly responding at each time step, a constant acceleration temporal discretization method, i.e., the Newmark-ß is employed. In real-time applications, the accuracy of the target variable highly depends on the accuracy of the inputs while differentiating noise from the signal is hardly ever possible. To address this problem, a Kalman filter-based method is developed. The proposed algorithm not only filters the noise from the measurements but also adapts the filter gain to the estimates of the target variable, i.e., the resulting tissue deformation. For a simulated tension test of a cubic model, the proposed algorithm achieves the update frequency of 63.3 Hz. This rate is a significant improvement in computational speed compared to the 5.8 Hz update rate by the classic FEM. Besides, this novel combination of the KF and the FEM makes it possible to expand the displacement estimates in the spatial domain when the measurements are only partially available at certain points. The performance of the above method is validated experimentally through a comparison with indentation tests on artificial human tissue-like material and with the FEM result under identical simulation conditions. The test is repeated on several samples, and the displacement variation from the FEM outcome is considered as the model error. Simulation results show that the proposed method achieves the deformation update frequency of 145.7 Hz compared to the 2.7 Hz from the reference FEM. The proposed method shows the same predictive ability, only 0.47% difference from FEM on average. Experimental validation of the proposed KF-FEM confirms that by consideration of both the measurement noise and the model error, the proposed method is capable of achieving high-frequency response without sacrificing the accuracy. Further to this, the experiments confirmed the linearized model response is reliable within the applied displacement range and therefore proving that KF can be employed. The developed KF-FEM was modified in the next study to address the problem resulting from inaccurate external loads measurements by the force sensors. In the modified version, both the external force, i.e., driving variable, and the displacement, i.e., driven variable, are taken as system states. It is considered that the uncertainty of the model input influences the accuracy of the system estimates. The modified model is calibrated to differentiate the system noise from the input noise. Numerical simulations were conducted on a liver shape geometrical model, and the simulation results demonstrate that more than 90% of the measurement noise is removed. The computational speed is also increased, delivering up to 89 Hz update rate. While the uncertainty of the external load is replicated in the displacements in an FEM solution, the developed algorithm can differentiate the measurement noise, including the displacement and external forces, from the system error, i.e., the FE model error. In the last study, the proposed model was developed to reflect the nonlinear behaviour of the manipulated tissue. The Central Difference time discretization method was used to model large deformations. A novel feature is that the Equation of motion is formulated within the element level rather than in the global spatial domain. This approach helped to improve the computational speed. Indentation with strains of slightly over 10% was simulated to assess the performance of the proposed model. The developed algorithm achieved the 33.85 Hz update frequency on a standard-issue PC and confirmed its suitability for real-time applications. Also, the proposed model achieved estimates with a maximum 5.75% mean absolute error (MAE) concerning the measurements while the classic FEM showed 6.20% MAE under identical simulation condition. Results confirm that deformation estimates for noisy boundary loads of the FEM can be improved with the help of direct measurements and yet be realistic in terms of real-time visual update. This study proposed a novel computational algorithm that achieved update frequencies of higher than 25 Hz to be perceived as real-time in human eyes. The developed KF-FEM model has also shown the potential of improving the FEM accuracy with the help of direct measurements. The proposed algorithm used partially available measurements and expanded its estimates in the spatial domain. The method was experimentally validated, and the model input uncertainty, as well as the nonlinear behaviour of the soft tissue, were assessed and verified

An Overview of Self-Adaptive Technologies Within Virtual Reality Training

This overview presents the current state-of-the-art of self-adaptive technologies within virtual reality (VR) training. Virtual reality training and assessment is increasingly used for five key areas: medical, industrial & commercial training, serious games, rehabilitation and remote training such as Massive Open Online Courses (MOOCs). Adaptation can be applied to five core technologies of VR including haptic devices, stereo graphics, adaptive content, assessment and autonomous agents. Automation of VR training can contribute to automation of actual procedures including remote and robotic assisted surgery which reduces injury and improves accuracy of the procedure. Automated haptic interaction can enable tele-presence and virtual artefact tactile interaction from either remote or simulated environments. Automation, machine learning and data driven features play an important role in providing trainee-specific individual adaptive training content. Data from trainee assessment can form an input to autonomous systems for customised training and automated difficulty levels to match individual requirements. Self-adaptive technology has been developed previously within individual technologies of VR training. One of the conclusions of this research is that while it does not exist, an enhanced portable framework is needed and it would be beneficial to combine automation of core technologies, producing a reusable automation framework for VR training

Development of a Reality-Based, Haptics-Enabled Simulator for Tool-Tissue Interactions

The advent of complex surgical procedures has driven the need for finite element based surgical training simulators which provide realistic visual and haptic feedback throughout the surgical task. The foundation of a simulator stems from the use of accurate, reality-based models for the global tissue response as well as the tool-tissue interactions. To that end, ex vivo and in vivo tests were conducted for soft-tissue probing and in vivo tests were conducted for soft-tissue cutting for the purpose of model development. In formulating a surgical training system, there is a desire to replicate the surgical task as accurately as possible for haptic and visual realism. However, for many biological tissues, there is a discrepancy between the mechanical characteristics of ex vivo and in vivo tissue. The efficacy of utilizing an ex vivo model for simulation of in vivo probing tasks on porcine liver was evaluated by comparing the simulated probing task to an identical in vivo probing experiment. The models were then further improved upon to better replicate the in vivo response. During the study of cutting modeling, in vivo cutting experiments were performed on porcine liver to derive the force-displacement response of the tissue to a scalpel blade. Using this information, a fracture mechanics based approach was applied to develop a fully defined cohesive zone model governing the separation properties of the liver directly in front of the scalpel blade. Further, a method of scaling the cohesive zone parameters was presented to minimize the computational expense in an effort to apply the cohesive based cutting approach to real-time simulators. The development of the models for the global tissue response and local tool-tissue interactions for probing and cutting of soft-tissue provided the framework for real-time simulation of basic surgical skills training. Initially, a pre-processing approach was used for the development of reality-based, haptics enabled simulators for probing and cutting of soft tissue. Then a real-time finite element based simulator was developed to simulate the probing task without the need to know the tool path prior to simulation

FAST PHYSICS-BASED SIMULATION OF VASCULAR SURGERY

Ph.DDOCTOR OF PHILOSOPH

A Surface Mass-Spring Model with New Flexion Springs and Collision Detection Algorithms Based on Volume Structure for Real-time Soft-tissue Deformation Interaction

A critical problem associated with surgical simulation is balancing deformation accuracy with real-time performance. Although the canonical surface mass-spring model (MSM) can provide an excellent real-time performance, it fails to provide effective shape restoration behavior when generating large deformations. This significantly influences its deformation accuracy. To address this problem, this paper proposes a modified surface MSM. In the proposed MSM, a new flexion spring is first developed to oppose bending based on the included angle between the initial position vector and the deformational position vector, improving the shape restoration performance and enhance the deformational accuracy of MSM; then, a new type of surface triangular topological unit is developed for enhancing the computational efficiency and better adapting to the different topological soft tissue deformational models. In addition, to further improve the accuracy of deformational interactions between the soft tissue and surgical instruments, we also propose two new collision detection algorithms. One is the discrete collision detection with the volumetric structure (DCDVS), applying a volumetric structure to extend the effective range of collision detection; the other is the hybrid collision detection with the volumetric structure (HCDVS), introducing the interpolation techniques of the continuous collision detection to DCDVS. Experimental results show that the proposed MSM with DCDVS or HCDVS can achieve accurate and stable shape restoration and show the real-time interactive capability in the virtual artery vessel and heart compared with the canonical surface MSM and new volume MSM

Real-time simulation of soft tissue deformation for surgical simulation

Surgical simulation plays an important role in the training, planning and evaluation of many surgical procedures. It requires realistic and real-time simulation of soft tissue deformation under interaction with surgical tools. However, it is challenging to satisfy both of these conflicting requirements. On one hand, biological soft tissues are complex in terms of material compositions, structural formations, and mechanical behaviours, resulting in nonlinear deformation characteristics under an external load. Due to the involvement of both material and geometric nonlinearities, the use of nonlinear elasticity causes a highly expensive computational load, leading to the difficulty to achieve the real-time computational performance required by surgical simulation. On the other hand, in order to satisfy the real-time computational requirement, most of the existing methods are mainly based on linear elasticity under the assumptions of small deformation and homogeneity to describe deformation of soft tissues. Such simplifications allow reduced runtime computation; however, they are inadequate for modelling nonlinear material properties such as anisotropy, heterogeneity and large deformation of soft tissues. In general, the two conflicting requirements of surgical simulation raise immense complexity in modelling of soft tissue deformation. This thesis focuses on establishment of new methodologies for modelling of soft tissue deformation for surgical simulation. Due to geometric and material nonlinearities in soft tissue deformation, the existing methods have only limited capabilities in achieving nonlinear soft tissue deformation in real-time. In this thesis, the main focus is devoted to the real-time and realistic modelling of nonlinear soft tissue deformation for surgical simulation. New methodologies, namely new ChainMail algorithms, energy propagation method, and energy balance method, are proposed to address soft tissue deformation. Results demonstrate that the proposed methods can simulate the typical soft tissue mechanical properties, accommodate isotropic and homogeneous, anisotropic and heterogeneous materials, handle incompressibility and viscoelastic behaviours, conserve system energy, and achieve realistic, real-time and stable deformation. In the future, it is projected to extend the proposed methodologies to handle surgical operations, such as cutting, joining and suturing, for topology changes occurred in surgical simulation