4 research outputs found
Gesture Recognition in Robotic Surgery: a Review
OBJECTIVE: Surgical activity recognition is a fundamental step in computer-assisted interventions. This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery focusing on recent data-driven approaches and outlines the open questions and future research directions. METHODS: An article search was performed on 5 bibliographic databases with combinations of the following search terms: robotic, robot-assisted, JIGSAWS, surgery, surgical, gesture, fine-grained, surgeme, action, trajectory, segmentation, recognition, parsing. Selected articles were classified based on the level of supervision required for training and divided into different groups representing major frameworks for time series analysis and data modelling. RESULTS: A total of 52 articles were reviewed. The research field is showing rapid expansion, with the majority of articles published in the last 4 years. Deep-learning-based temporal models with discriminative feature extraction and multi-modal data integration have demonstrated promising results on small surgical datasets. Currently, unsupervised methods perform significantly less well than the supervised approaches. CONCLUSION: The development of large and diverse open-source datasets of annotated demonstrations is essential for development and validation of robust solutions for surgical gesture recognition. While new strategies for discriminative feature extraction and knowledge transfer, or unsupervised and semi-supervised approaches, can mitigate the need for data and labels, they have not yet been demonstrated to achieve comparable performance. Important future research directions include detection and forecast of gesture-specific errors and anomalies. SIGNIFICANCE: This paper is a comprehensive and structured analysis of surgical gesture recognition methods aiming to summarize the status of this rapidly evolving field
Robot Assisted Object Manipulation for Minimally Invasive Surgery
Robotic systems have an increasingly important role in facilitating minimally invasive surgical treatments. In robot-assisted minimally invasive surgery, surgeons remotely control instruments from a console to perform operations inside the patient. However, despite the advanced technological status of surgical robots, fully autonomous systems, with decision-making capabilities, are not yet available.
In 2017, a structure to classify the research efforts toward autonomy achievable with surgical robots was proposed by Yang et al. Six different levels were identified: no autonomy, robot assistance, task autonomy,
conditional autonomy, high autonomy, and full autonomy. All the commercially available platforms in robot-assisted
surgery is still in level 0 (no autonomy). Despite increasing the level of autonomy remains an open challenge, its adoption could potentially introduce multiple benefits, such as decreasing surgeons’ workload and fatigue and pursuing a consistent
quality of procedures. Ultimately, allowing the surgeons to interpret the ample
and intelligent information from the system will enhance the surgical outcome and
positively reflect both on patients and society. Three main aspects are required to
introduce automation into surgery: the surgical robot must move with high precision,
have motion planning capabilities and understand the surgical scene. Besides
these main factors, depending on the type of surgery, there could be other aspects
that might play a fundamental role, to name some compliance, stiffness, etc. This
thesis addresses three technological challenges encountered when trying to achieve
the aforementioned goals, in the specific case of robot-object interaction. First,
how to overcome the inaccuracy of cable-driven systems when executing fine and
precise movements. Second, planning different tasks in dynamically changing environments.
Lastly, how the understanding of a surgical scene can be used to solve
more than one manipulation task.
To address the first challenge, a control scheme relying on accurate calibration is
implemented to execute the pick-up of a surgical needle. Regarding the planning of
surgical tasks, two approaches are explored: one is learning from demonstration to
pick and place a surgical object, and the second is using a gradient-based approach
to trigger a smoother object repositioning phase during intraoperative procedures.
Finally, to improve scene understanding, this thesis focuses on developing a simulation
environment where multiple tasks can be learned based on the surgical scene
and then transferred to the real robot. Experiments proved that automation of the pick and place task of different surgical objects is possible. The robot was successfully
able to autonomously pick up a suturing needle, position a surgical device for
intraoperative ultrasound scanning and manipulate soft tissue for intraoperative organ
retraction. Despite automation of surgical subtasks has been demonstrated in
this work, several challenges remain open, such as the capabilities of the generated
algorithm to generalise over different environment conditions and different patients
Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit
Robotic-assisted surgery is now well-established in clinical practice and has
become the gold standard clinical treatment option for several clinical
indications. The field of robotic-assisted surgery is expected to grow
substantially in the next decade with a range of new robotic devices emerging
to address unmet clinical needs across different specialities. A vibrant
surgical robotics research community is pivotal for conceptualizing such new
systems as well as for developing and training the engineers and scientists to
translate them into practice. The da Vinci Research Kit (dVRK), an academic and
industry collaborative effort to re-purpose decommissioned da Vinci surgical
systems (Intuitive Surgical Inc, CA, USA) as a research platform for surgical
robotics research, has been a key initiative for addressing a barrier to entry
for new research groups in surgical robotics. In this paper, we present an
extensive review of the publications that have been facilitated by the dVRK
over the past decade. We classify research efforts into different categories
and outline some of the major challenges and needs for the robotics community
to maintain this initiative and build upon it