Towards retrieving force feedback in robotic-assisted surgery: a supervised neuro-recurrent-vision approach

Robotic-assisted minimally invasive surgeries have gained a lot of popularity over conventional procedures as they offer many benefits to both surgeons and patients. Nonetheless, they still suffer from some limitations that affect their outcome. One of them is the lack of force feedback which restricts the surgeon's sense of touch and might reduce precision during a procedure. To overcome this limitation, we propose a novel force estimation approach that combines a vision based solution with supervised learning to estimate the applied force and provide the surgeon with a suitable representation of it. The proposed solution starts with extracting the geometry of motion of the heart's surface by minimizing an energy functional to recover its 3D deformable structure. A deep network, based on a LSTM-RNN architecture, is then used to learn the relationship between the extracted visual-geometric information and the applied force, and to find accurate mapping between the two. Our proposed force estimation solution avoids the drawbacks usually associated with force sensing devices, such as biocompatibility and integration issues. We evaluate our approach on phantom and realistic tissues in which we report an average root-mean square error of 0.02 N.Peer ReviewedPostprint (author's final draft

Soft volume simulation using a deformable surface model

The aim of the research is to contribute to the modelling of deformable objects, such as soft tissues in medical simulation. Interactive simulation for medical training is a concept undergoing rapid growth as the underlying technologies support the increasingly more realstic and functional training environments. The prominent issues in the deployment of such environments centre on a fine balance between the accuracy of the deformable model and real-time interactivity. Acknowledging the importance of interacting with non-rigid materials such as the palpation of a breast for breast assessment, this thesis has explored the physics-based modelling techniques for both volume and surface approach. This thesis identified that the surface approach based on the mass spring system (MSS) has the benefits of rapid prototyping, reduced mesh complexity, computational efficiency and the support for large material deformation compared to the continuum approach. However, accuracy relative to real material properties is often over looked in the configuration of the resulting model. This thesis has investigated the potential and the feasibility of surface modelling for simulating soft objects regardless of the design of the mesh topology and the non-existence of internal volume discretisation. The assumptions of the material parameters such as elasticity, homogeneity and incompressibility allow a reduced set of material values to be implemented in order to establish the association with the surface configuration. A framework for a deformable surface model was generated in accordance with the issues of the estimation of properties and volume behaviour corresponding to the material parameters. The novel extension to the surface MSS enables the tensile properties of the material to be integrated into an enhanced configuration despite its lack of volume information. The benefits of the reduced complexity of a surface model are now correlated with the improved accuracy in the estimation of properties and volume behaviour. Despite the irregularity of the underlying mesh topology and the absence of volume, the model reflected the original material values and preserved volume with minimal deviations. Global deformation effect which is essential to emulate the run time behaviour of a real soft material upon interaction, such as the palpation of a generic breast, was also demonstrated, thus indicating the potential of this novel technique in the application of soft tissue simulation

Chapter 4: simulating a deformable object using a surface mass spring system

This paper introduces volume springs that provide the volume effect to a surface model when it is deformed The estimation of the properties of the model takes the real material properties into consideration, where each spring stiffness is derived based on the elasticity, rigidity and compressibility modulus. The proposed model can be adopted to simulate soft objects such as a deformable human breast, and it can be further extended to address other material properties

VISIO-HAPTIC DEFORMABLE MODEL FOR HAPTIC DOMINANT PALPATION SIMULATOR

Vision and haptic are two most important modalities in a medical simulation. While visual cues assist one to see his actions when performing a medical procedure, haptic cues enable feeling the object being manipulated during the interaction. Despite their importance in a computer simulation, the combination of both modalities has not been adequately assessed, especially that in a haptic dominant environment. Thus, resulting in poor emphasis in resource allocation management in terms of effort spent in rendering the two modalities for simulators with realistic real-time interactions. Addressing this problem requires an investigation on whether a single modality (haptic) or a combination of both visual and haptic could be better for learning skills in a haptic dominant environment such as in a palpation simulator. However, before such an investigation could take place one main technical implementation issue in visio-haptic rendering needs to be addresse

Reconfiguration of a smart surface using heteroclinic connections

A reconfigurable smart surface with multiple equilibria is presented, modelled using discrete point masses and linear springs with geometric nonlinearity. An energy-efficient reconfiguration scheme is then investigated to connect equal-energy unstable (but actively controlled) equilibria. In principle zero net energy input is required to transition the surface between these unstable states, compared to transitions between stable equilibria across a potential barrier. These transitions between equal-energy unstable states therefore form heteroclinic connections in the phase space of the problem. Moreover, the smart surface model developed can be considered as a unit module for a range of applications, including modules which can aggregate together to form larger distributed smart surface systems