756 research outputs found
Concurrent Segmentation and Localization for Tracking of Surgical Instruments
Real-time instrument tracking is a crucial requirement for various
computer-assisted interventions. In order to overcome problems such as specular
reflections and motion blur, we propose a novel method that takes advantage of
the interdependency between localization and segmentation of the surgical tool.
In particular, we reformulate the 2D instrument pose estimation as heatmap
regression and thereby enable a concurrent, robust and near real-time
regression of both tasks via deep learning. As demonstrated by our experimental
results, this modeling leads to a significantly improved performance than
directly regressing the tool position and allows our method to outperform the
state of the art on a Retinal Microsurgery benchmark and the MICCAI EndoVis
Challenge 2015.Comment: I. Laina and N. Rieke contributed equally to this work. Accepted to
MICCAI 201
Computer- and robot-assisted Medical Intervention
Medical robotics includes assistive devices used by the physician in order to
make his/her diagnostic or therapeutic practices easier and more efficient.
This chapter focuses on such systems. It introduces the general field of
Computer-Assisted Medical Interventions, its aims, its different components and
describes the place of robots in that context. The evolutions in terms of
general design and control paradigms in the development of medical robots are
presented and issues specific to that application domain are discussed. A view
of existing systems, on-going developments and future trends is given. A
case-study is detailed. Other types of robotic help in the medical environment
(such as for assisting a handicapped person, for rehabilitation of a patient or
for replacement of some damaged/suppressed limbs or organs) are out of the
scope of this chapter.Comment: Handbook of Automation, Shimon Nof (Ed.) (2009) 000-00
Articulated Multi-Instrument 2D Pose Estimation Using Fully Convolutional Networks
Instrument detection, pose estimation and tracking in surgical videos is an important vision component for computer assisted interventions. While significant advances have been made in recent years, articulation detection is still a major challenge. In this paper, we propose a deep neural network for articulated multi-instrument 2D pose estimation, which is trained on a detailed annotations of endoscopic and microscopic datasets. Our model is formed by a fully convolutional detection-regression network. Joints and associations between joint pairs in our instrument model are located by the detection subnetwork and are subsequently refined through a regression subnetwork. Based on the output from the model, the poses of the instruments are inferred using maximum bipartite graph matching. Our estimation framework is powered by deep learning techniques without any direct kinematic information from a robot. Our framework is tested on single-instrument RMIT data, and also on multi-instrument EndoVis and in vivo data with promising results. In addition, the dataset annotations are publicly released along with our code and model
Simultaneous recognition and pose estimation of instruments in minimally invasive surgery
Detection of surgical instruments plays a key role in ensuring patient safety in minimally invasive surgery. In this paper, we present a novel method for 2D vision-based recognition and pose estimation of surgical instruments that generalizes to different surgical applications. At its core, we propose a novel scene model in order to simultaneously recognize multiple instruments as well as their parts. We use a Convolutional Neural Network architecture to embody our model and show that the cross-entropy loss is well suited to optimize its parameters which can be trained in an end-to-end fashion. An additional advantage of our approach is that instrument detection at test time is achieved while avoiding the need for scale-dependent sliding window evaluation. This allows our approach to be relatively parameter free at test time and shows good performance for both instrument detection and tracking. We show that our approach surpasses state-of-the-art results on in-vivo retinal microsurgery image data, as well as ex-vivo laparoscopic sequences
Unsupervised out-of-distribution detection for safer robotically-guided retinal microsurgery
Purpose: A fundamental problem in designing safe machine learning systems is
identifying when samples presented to a deployed model differ from those
observed at training time. Detecting so-called out-of-distribution (OoD)
samples is crucial in safety-critical applications such as robotically-guided
retinal microsurgery, where distances between the instrument and the retina are
derived from sequences of 1D images that are acquired by an
instrument-integrated optical coherence tomography (iiOCT) probe.
Methods: This work investigates the feasibility of using an OoD detector to
identify when images from the iiOCT probe are inappropriate for subsequent
machine learning-based distance estimation. We show how a simple OoD detector
based on the Mahalanobis distance can successfully reject corrupted samples
coming from real-world ex-vivo porcine eyes.
Results: Our results demonstrate that the proposed approach can successfully
detect OoD samples and help maintain the performance of the downstream task
within reasonable levels. MahaAD outperformed a supervised approach trained on
the same kind of corruptions and achieved the best performance in detecting OoD
cases from a collection of iiOCT samples with real-world corruptions.
Conclusion: The results indicate that detecting corrupted iiOCT data through
OoD detection is feasible and does not need prior knowledge of possible
corruptions. Consequently, MahaAD could aid in ensuring patient safety during
robotically-guided microsurgery by preventing deployed prediction models from
estimating distances that put the patient at risk.Comment: Accepted at IPCAI 202
Computerized Evaluatution of Microsurgery Skills Training
The style of imparting medical training has evolved, over the years. The traditional methods of teaching and practicing basic surgical skills under apprenticeship model, no longer occupy the first place in modern technically demanding advanced surgical disciplines like neurosurgery. Furthermore, the legal and ethical concerns for patient safety as well as cost-effectiveness have forced neurosurgeons to master the necessary microsurgical techniques to accomplish desired results. This has lead to increased emphasis on assessment of clinical and surgical techniques of the neurosurgeons. However, the subjective assessment of microsurgical techniques like micro-suturing under the apprenticeship model cannot be completely unbiased. A few initiatives using computer-based techniques, have been made to introduce objective evaluation of surgical skills. This thesis presents a novel approach involving computerized evaluation of different components of micro-suturing techniques, to eliminate the bias of subjective assessment. The work involved acquisition of cine clips of micro-suturing activity on synthetic material. Image processing and computer vision based techniques were then applied to these videos to assess different characteristics of micro-suturing viz. speed, dexterity and effectualness. In parallel subjective grading on these was done by a senior neurosurgeon. Further correlation and comparative study of both the assessments was done to analyze the efficacy of objective and subjective evaluation
Vision-based and marker-less surgical tool detection and tracking: a review of the literature
In recent years, tremendous progress has been made in surgical practice for example with Minimally Invasive Surgery (MIS). To overcome challenges coming from deported eye-to-hand manipulation, robotic and computer-assisted systems have been developed. Having real-time knowledge of the pose of surgical tools with respect to the surgical camera and underlying anatomy is a key ingredient for such systems. In this paper, we present a review of the literature dealing with vision-based and marker-less surgical tool detection. This paper includes three primary contributions: (1) identification and analysis of data-sets used for developing and testing detection algorithms, (2) in-depth comparison of surgical tool detection methods from the feature extraction process to the model learning strategy and highlight existing shortcomings, and (3) analysis of validation techniques employed to obtain detection performance results and establish comparison between surgical tool detectors. The papers included in the review were selected through PubMed and Google Scholar searches using the keywords: “surgical tool detection”, “surgical tool tracking”, “surgical instrument detection” and “surgical instrument tracking” limiting results to the year range 2000 2015. Our study shows that despite significant progress over the years, the lack of established surgical tool data-sets, and reference format for performance assessment and method ranking is preventing faster improvement
- …