Reaction force and surface deformation estimation based on heuristic tissue models

In many of the advanced surgical robotic systems, soft tissue mechanics and tool–tissue interaction modeling play an important role in achieving optimal control, relying on model-based control methods. This approach allows one to address crucial issues during teleoperation, such as time-delay, stability or state observation. This paper presents a novel approach for modeling the behavior of soft tissue during surgical interventions, employing the widely-used concept of rheological models. The nonlinear Wiecher model is used for reaction force estimation during tissue indentation, tested on beef liver samples for acquiring mechanical parameters from experimental data. Curve fitting methods have been used in both stress relaxation and constant indentation speed compression phases. Reaction forces are estimated using the proposed model, followed by verification tests on ex-vivo beef liver samples. The results of this research showed that the proposed novel rheological soft tissue model is capable of estimating the reaction forces acting on the tool, if the shape of the deformed tissue is known in time

Nonlinear Elastic Material Property Estimation of Lower Extremity Residual Limb Tissues

The interface stresses between the residual limb and prosthetic socket have been studied to investigate prosthetic fit. Finite-element models of the residual limb-prosthetic socket interface facilitate investigation of the mechanical interface and may serve as a potential tool for future prosthetic socket design. However, the success of such residual limb models to date has been limited, in large part due to inadequate material formulations used to approximate the mechanical behavior of residual limb soft tissues. Nonlinear finite-element analysis was used to simulate force-displacement data obtained during in vivo rate-controlled (1, 5, and 10 mm/s) cyclic indentation of the residual limb soft tissues of seven individuals with transtibial amputation. The finite-element models facilitated determination of an appropriate set of nonlinear elastic material coefficients for bulk soft tissue at discrete clinically relevant test locations. Axisymmetric finite-element models of the residual limb bulk soft tissue in the vicinity of the test location, the socket wall and the indentor tip were developed incorporating contact analysis, large displacement, and large strain, and the James-Green-Simpson nonlinear elastic material formulation. Model dimensions were based on medical imaging studies of the residual limbs. The material coefficients were selected such that the normalized sum of square error (NSSE) between the experimental and finite-element model indentor tip reaction force was minimized. A total of 95% of the experimental data were simulated using the James-Green-Simpson material formulation with an NSSE less than 5%. The respective James-Green-Simpson material coefficients varied with subject, test location, and indentation rate. Therefore, these coefficients cannot be readily extrapolated to other sites or individuals, or to the same site and individual some time after testing

Real-time measurement corrected prediction of soft tissue response for medical simulations

Medical simulators, such as in palpation and disease diagnosis, require an efficient model of the biological soft tissue deformation. Hence, a computationally fast and accurate algorithm is required to support and enhance user interactions in near real-time simulations. The visual accuracy of such simulators is dependent on the user¿s reaction time. Static visual images that update at a rate of 25 Hz are perceived as real-time moving images. Hence, visualizing software requires fast algorithms to compute the deformation of soft tissue to facilitate a meaningful simulation. Furthermore, soft tissue behaviour should be modelled accurately while compatible with real-time computation. This work proposes a fast solver for the linearized finite element method (FEM) and validates the proposed algorithm with experimental results. The novelty of the method lies in the utilization of real-time force/displacement measurements that are embedded in the solution via the Kalman filter. A novel computational algorithm that utilizes the strength of the FEM in terms of accuracy and employs direct measurements from the manipulated tissue to overcome the slow computational process of the FEM is proposed in the first part of the thesis. As the behaviour of the mechanically loaded tissue can be regarded as linearly responding at each time step, a constant acceleration temporal discretization method, i.e., the Newmark-ß is employed. In real-time applications, the accuracy of the target variable highly depends on the accuracy of the inputs while differentiating noise from the signal is hardly ever possible. To address this problem, a Kalman filter-based method is developed. The proposed algorithm not only filters the noise from the measurements but also adapts the filter gain to the estimates of the target variable, i.e., the resulting tissue deformation. For a simulated tension test of a cubic model, the proposed algorithm achieves the update frequency of 63.3 Hz. This rate is a significant improvement in computational speed compared to the 5.8 Hz update rate by the classic FEM. Besides, this novel combination of the KF and the FEM makes it possible to expand the displacement estimates in the spatial domain when the measurements are only partially available at certain points. The performance of the above method is validated experimentally through a comparison with indentation tests on artificial human tissue-like material and with the FEM result under identical simulation conditions. The test is repeated on several samples, and the displacement variation from the FEM outcome is considered as the model error. Simulation results show that the proposed method achieves the deformation update frequency of 145.7 Hz compared to the 2.7 Hz from the reference FEM. The proposed method shows the same predictive ability, only 0.47% difference from FEM on average. Experimental validation of the proposed KF-FEM confirms that by consideration of both the measurement noise and the model error, the proposed method is capable of achieving high-frequency response without sacrificing the accuracy. Further to this, the experiments confirmed the linearized model response is reliable within the applied displacement range and therefore proving that KF can be employed. The developed KF-FEM was modified in the next study to address the problem resulting from inaccurate external loads measurements by the force sensors. In the modified version, both the external force, i.e., driving variable, and the displacement, i.e., driven variable, are taken as system states. It is considered that the uncertainty of the model input influences the accuracy of the system estimates. The modified model is calibrated to differentiate the system noise from the input noise. Numerical simulations were conducted on a liver shape geometrical model, and the simulation results demonstrate that more than 90% of the measurement noise is removed. The computational speed is also increased, delivering up to 89 Hz update rate. While the uncertainty of the external load is replicated in the displacements in an FEM solution, the developed algorithm can differentiate the measurement noise, including the displacement and external forces, from the system error, i.e., the FE model error. In the last study, the proposed model was developed to reflect the nonlinear behaviour of the manipulated tissue. The Central Difference time discretization method was used to model large deformations. A novel feature is that the Equation of motion is formulated within the element level rather than in the global spatial domain. This approach helped to improve the computational speed. Indentation with strains of slightly over 10% was simulated to assess the performance of the proposed model. The developed algorithm achieved the 33.85 Hz update frequency on a standard-issue PC and confirmed its suitability for real-time applications. Also, the proposed model achieved estimates with a maximum 5.75% mean absolute error (MAE) concerning the measurements while the classic FEM showed 6.20% MAE under identical simulation condition. Results confirm that deformation estimates for noisy boundary loads of the FEM can be improved with the help of direct measurements and yet be realistic in terms of real-time visual update. This study proposed a novel computational algorithm that achieved update frequencies of higher than 25 Hz to be perceived as real-time in human eyes. The developed KF-FEM model has also shown the potential of improving the FEM accuracy with the help of direct measurements. The proposed algorithm used partially available measurements and expanded its estimates in the spatial domain. The method was experimentally validated, and the model input uncertainty, as well as the nonlinear behaviour of the soft tissue, were assessed and verified

Patient-specific simulation for autonomous surgery

An Autonomous Robotic Surgical System (ARSS) has to interact with the complex anatomical environment, which is deforming and whose properties are often uncertain. Within this context, an ARSS can benefit from the availability of patient-specific simulation of the anatomy. For example, simulation can provide a safe and controlled environment for the design, test and validation of the autonomous capabilities. Moreover, it can be used to generate large amounts of patient-specific data that can be exploited to learn models and/or tasks. The aim of this Thesis is to investigate the different ways in which simulation can support an ARSS and to propose solutions to favor its employability in robotic surgery. We first address all the phases needed to create such a simulation, from model choice in the pre-operative phase based on the available knowledge to its intra-operative update to compensate for inaccurate parametrization. We propose to rely on deep neural networks trained with synthetic data both to generate a patient-specific model and to design a strategy to update model parametrization starting directly from intra-operative sensor data. Afterwards, we test how simulation can assist the ARSS, both for task learning and during task execution. We show that simulation can be used to efficiently train approaches that require multiple interactions with the environment, compensating for the riskiness to acquire data from real surgical robotic systems. Finally, we propose a modular framework for autonomous surgery that includes deliberative functions to handle real anatomical environments with uncertain parameters. The integration of a personalized simulation proves fundamental both for optimal task planning and to enhance and monitor real execution. The contributions presented in this Thesis have the potential to introduce significant step changes in the development and actual performance of autonomous robotic surgical systems, making them closer to applicability to real clinical conditions

A Heuristic Image Search Algorithm for Active Shape Model Segmentation of the Caudate Nucleus and Hippocampus in Brain MR Images of Children with FASD

Magnetic Resonance Imaging provides a non-invasive means to study the neural correlates of Fetal Alcohol Spectrum Disorder (FASD) - the most common form of preventable mental retardation worldwide. One approach aims to detect brain abnormalities through an assessment of volume and shape of two sub-cortical structures, the caudate nucleus and hippocampus. We present a method for automatically segmenting these structures from high-resolution MR images captured as part of an ongoing study into the neural correlates of FASD. Our method incorporates an Active Shape Model, which is used to learn shape variation from manually segmented training data. A modified discrete Geometrically Deformable Model is used to generate point correspondence between training models. An ASM is then created from the landmark points. Experiments were conducted on the image search phase of ASM segmentation, in order to find the technique best suited to segmentation of the hippocampus and caudate nucleus. Various popular image search techniques were tested, including an edge detection method and a method based on grey profile Mahalanobis distance measurement. A novel heuristic image search method was also developed and tested. This heuristic method improves image segmentation by taking advantage of characteristics specific to the target data, such as a relatively homogeneous tissue colour in target structures. Results show that ASMs that use the heuristic image search technique produce the most accurate segmentations. An ASM constructed using this technique will enable researchers to quickly, reliably, and automatically segment test data for use in the FASD study

Soft volume simulation using a deformable surface model

The aim of the research is to contribute to the modelling of deformable objects, such as soft tissues in medical simulation. Interactive simulation for medical training is a concept undergoing rapid growth as the underlying technologies support the increasingly more realstic and functional training environments. The prominent issues in the deployment of such environments centre on a fine balance between the accuracy of the deformable model and real-time interactivity. Acknowledging the importance of interacting with non-rigid materials such as the palpation of a breast for breast assessment, this thesis has explored the physics-based modelling techniques for both volume and surface approach. This thesis identified that the surface approach based on the mass spring system (MSS) has the benefits of rapid prototyping, reduced mesh complexity, computational efficiency and the support for large material deformation compared to the continuum approach. However, accuracy relative to real material properties is often over looked in the configuration of the resulting model. This thesis has investigated the potential and the feasibility of surface modelling for simulating soft objects regardless of the design of the mesh topology and the non-existence of internal volume discretisation. The assumptions of the material parameters such as elasticity, homogeneity and incompressibility allow a reduced set of material values to be implemented in order to establish the association with the surface configuration. A framework for a deformable surface model was generated in accordance with the issues of the estimation of properties and volume behaviour corresponding to the material parameters. The novel extension to the surface MSS enables the tensile properties of the material to be integrated into an enhanced configuration despite its lack of volume information. The benefits of the reduced complexity of a surface model are now correlated with the improved accuracy in the estimation of properties and volume behaviour. Despite the irregularity of the underlying mesh topology and the absence of volume, the model reflected the original material values and preserved volume with minimal deviations. Global deformation effect which is essential to emulate the run time behaviour of a real soft material upon interaction, such as the palpation of a generic breast, was also demonstrated, thus indicating the potential of this novel technique in the application of soft tissue simulation