1,794 research outputs found
Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery
Intraoperative segmentation and tracking of minimally invasive instruments is
a prerequisite for computer- and robotic-assisted surgery. Since additional
hardware like tracking systems or the robot encoders are cumbersome and lack
accuracy, surgical vision is evolving as promising techniques to segment and
track the instruments using only the endoscopic images. However, what is
missing so far are common image data sets for consistent evaluation and
benchmarking of algorithms against each other. The paper presents a comparative
validation study of different vision-based methods for instrument segmentation
and tracking in the context of robotic as well as conventional laparoscopic
surgery. The contribution of the paper is twofold: we introduce a comprehensive
validation data set that was provided to the study participants and present the
results of the comparative validation study. Based on the results of the
validation study, we arrive at the conclusion that modern deep learning
approaches outperform other methods in instrument segmentation tasks, but the
results are still not perfect. Furthermore, we show that merging results from
different methods actually significantly increases accuracy in comparison to
the best stand-alone method. On the other hand, the results of the instrument
tracking task show that this is still an open challenge, especially during
challenging scenarios in conventional laparoscopic surgery
Artificial intelligence and automation in endoscopy and surgery
Modern endoscopy relies on digital technology, from high-resolution imaging sensors and displays to electronics connecting configurable illumination and actuation systems for robotic articulation. In addition to enabling more effective diagnostic and therapeutic interventions, the digitization of the procedural toolset enables video data capture of the internal human anatomy at unprecedented levels. Interventional video data encapsulate functional and structural information about a patient’s anatomy as well as events, activity and action logs about the surgical process. This detailed but difficult-to-interpret record from endoscopic procedures can be linked to preoperative and postoperative records or patient imaging information. Rapid advances in artificial intelligence, especially in supervised deep learning, can utilize data from endoscopic procedures to develop systems for assisting procedures leading to computer-assisted interventions that can enable better navigation during procedures, automation of image interpretation and robotically assisted tool manipulation. In this Perspective, we summarize state-of-the-art artificial intelligence for computer-assisted interventions in gastroenterology and surgery
A comprehensive survey on recent deep learning-based methods applied to surgical data
Minimally invasive surgery is highly operator dependant with a lengthy
procedural time causing fatigue to surgeon and risks to patients such as injury
to organs, infection, bleeding, and complications of anesthesia. To mitigate
such risks, real-time systems are desired to be developed that can provide
intra-operative guidance to surgeons. For example, an automated system for tool
localization, tool (or tissue) tracking, and depth estimation can enable a
clear understanding of surgical scenes preventing miscalculations during
surgical procedures. In this work, we present a systematic review of recent
machine learning-based approaches including surgical tool localization,
segmentation, tracking, and 3D scene perception. Furthermore, we provide a
detailed overview of publicly available benchmark datasets widely used for
surgical navigation tasks. While recent deep learning architectures have shown
promising results, there are still several open research problems such as a
lack of annotated datasets, the presence of artifacts in surgical scenes, and
non-textured surfaces that hinder 3D reconstruction of the anatomical
structures. Based on our comprehensive review, we present a discussion on
current gaps and needed steps to improve the adaptation of technology in
surgery.Comment: This paper is to be submitted to International journal of computer
visio
Plug-in for visualizing 3D tool tracking from videos of Minimally Invasive Surgeries
This paper tackles instrument tracking and 3D visualization challenges in
minimally invasive surgery (MIS), crucial for computer-assisted interventions.
Conventional and robot-assisted MIS encounter issues with limited 2D camera
projections and minimal hardware integration. The objective is to track and
visualize the entire surgical instrument, including shaft and metallic clasper,
enabling safe navigation within the surgical environment. The proposed method
involves 2D tracking based on segmentation maps, facilitating creation of
labeled dataset without extensive ground-truth knowledge. Geometric changes in
2D intervals express motion, and kinematics based algorithms process results
into 3D tracking information. Synthesized and experimental results in 2D and 3D
motion estimates demonstrate negligible errors, validating the method for
labeling and motion tracking of instruments in MIS videos. The conclusion
underscores the proposed 2D segmentation technique's simplicity and
computational efficiency, emphasizing its potential as direct plug-in for 3D
visualization in instrument tracking and MIS practices
Recent trends, technical concepts and components of computer-assisted orthopedic surgery systems: A comprehensive review
Computer-assisted orthopedic surgery (CAOS) systems have become one of the most important and challenging types of system in clinical orthopedics, as they enable precise treatment of musculoskeletal diseases, employing modern clinical navigation systems and surgical tools. This paper brings a comprehensive review of recent trends and possibilities of CAOS systems. There are three types of the surgical planning systems, including: systems based on the volumetric images (computer tomography (CT), magnetic resonance imaging (MRI) or ultrasound images), further systems utilize either 2D or 3D fluoroscopic images, and the last one utilizes the kinetic information about the joints and morphological information about the target bones. This complex review is focused on three fundamental aspects of CAOS systems: their essential components, types of CAOS systems, and mechanical tools used in CAOS systems. In this review, we also outline the possibilities for using ultrasound computer-assisted orthopedic surgery (UCAOS) systems as an alternative to conventionally used CAOS systems.Web of Science1923art. no. 519
MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation
We propose Masked-Attention Transformers for Surgical Instrument Segmentation
(MATIS), a two-stage, fully transformer-based method that leverages modern
pixel-wise attention mechanisms for instrument segmentation. MATIS exploits the
instance-level nature of the task by employing a masked attention module that
generates and classifies a set of fine instrument region proposals. Our method
incorporates long-term video-level information through video transformers to
improve temporal consistency and enhance mask classification. We validate our
approach in the two standard public benchmarks, Endovis 2017 and Endovis 2018.
Our experiments demonstrate that MATIS' per-frame baseline outperforms previous
state-of-the-art methods and that including our temporal consistency module
boosts our model's performance further
- …