Visual kinematic force estimation in robot-assisted surgery – application to knot tying

Robot-assisted surgery has potential advantages but lacks force feedback, which can lead to errors such as broken stitches or tissue damage. More experienced surgeons can judge the tool-tissue forces visually and an automated way of capturing this skill is desirable. Methods to measure force tend to involve complex measurement devices or visual tracking of tissue deformation. We investigate whether surgical forces can be estimated simply from the discrepancy between kinematic and visual measurement of the tool position. We show that combined visual and kinematic force estimation can be achieved without external measurements or modelling of tissue deformation. After initial alignment when no force is applied to the tool, visual and kinematic estimates of tool position diverge under force. We plot visual/kinematic displacement with force using vision and marker-based tracking. We demonstrate the ability to discern the forces involved in knot tying and visualize the displacement force using the publicly available JIGSAWS dataset as well as clinical examples of knot tying with the da Vinci surgical system. The ability to visualize or feel forces using this method may offer an advantage to those learning robotic surgery as well as adding to the information available to more experienced surgeons

The benefits of haptic feedback in robot assisted surgery and their moderators: a metaanalysis

Robot assisted surgery (RAS) provides medical practitioners with valuable tools, decreasing strain during surgery and leading to better patient outcomes. While the loss of haptic sensation is a commonly cited disadvantage of RAS, new systems aim to address this problem by providing artificial haptic feedback. N = 56 papers that compared robotic surgery systems with and without haptic feedback were analyzed to quantify the performance benefits of restoring the haptic modality. Additionally, this study identifies factors moderating the effect of restoring haptic sensation. Overall results showed haptic feedback was effective in reducing average forces (Hedges' g = 0.83) and peak forces (Hedges' g = 0.69) applied during surgery, as well as reducing the completion time (Hedges' g = 0.83). Haptic feedback has also been found to lead to higher accuracy (Hedges' g = 1.50) and success rates (Hedges' g = 0.80) during surgical tasks. Effect sizes on several measures varied between tasks, the type of provided feedback, and the subjects' levels of surgical expertise, with higher levels of expertise generally associated with smaller effect sizes. No significant differences were found between virtual fixtures and rendering contact forces. Implications for future research are discussed

Suture Breakage Warning System for Robotic Surgery

As robotic surgery has increased in popularity, the lack of haptic feedback has become a growing issue due to the application of excessive forces that may lead to clinical problems such as intraoperative and postoperative suture breakage. Previous suture breakage warning systems have largely depended on visual and/or auditory feedback modalities, which have been shown to increase cognitive load and reduce operator performance. This work catalogues a new sensing technology and haptic feedback system (HFS) that can reduce instances of suture failure without negatively impacting performance outcomes including knot quality. Suture breakage is common in knot-tying as the pulling motion introduces prominent shear forces. A shear sensor mountable on the da Vinci robotic surgical system's Cadiere grasper detects forces that correlate to the suture's internal tension. HFS then provides vibration feedback to the operator as forces near a particular material's failure load. To validate the system, subjects tightened a total of four knots, two with the Haptic Feedback System (HFS) and two without feedback. The number of suture breakages were recorded and knot fidelity was evaluated by measuring knot slippage. Results showed that instances of suture failure were significantly reduced when HFS was enabled (p = 0.0078). Notably, knots tied with HFS also showed improved quality compared to those tied without feedback (p = 0.010). The results highlight the value of HFS in improving robotic procedure outcomes by reducing instances of suture failures, producing better knots, and reducing the need for corrective measures

Biaxial sensing suture breakage warning system for robotic surgery

The number of procedures performed with robotic surgery may exceed one million globally in 2018. The continual lack of haptic feedback, however, forces surgeons to rely on visual cues in order to avoid breaking sutures due to excessive applied force. To mitigate this problem, the authors developed and validated a novel grasper-integrated system with biaxial shear sensing and haptic feedback to warn the operator prior to anticipated suture breakage. Furthermore, the design enables facile suture manipulation without a degradation in efficacy, as determined via measured tightness of resulting suture knots. Biaxial shear sensors were integrated with a da Vinci robotic surgical system. Novice subjects (n = 17) were instructed to tighten 10 knots, five times with the Haptic Feedback System (HFS) enabled, five times with the system disabled. Seven suture failures occurred in trials with HFS enabled while seventeen occurred in trials without feedback. The biaxial shear sensing system reduced the incidence of suture failure by 59% (p = 0.0371). It also resulted in 25% lower average applied force in comparison to trials without feedback (p = 0.00034), which is relevant because average force was observed to play a role in suture breakage (p = 0.03925). An observed 55% decrease in standard deviation of knot quality when using the HFS also indicates an improvement in consistency when using the feedback system. These results suggest this system may improve outcomes related to knot tying tasks in robotic surgery and reduce instances of suture failure while not degrading the quality of knots produced