41 research outputs found
A comprehensive survey on recent deep learning-based methods applied to surgical data
Minimally invasive surgery is highly operator dependant with a lengthy
procedural time causing fatigue to surgeon and risks to patients such as injury
to organs, infection, bleeding, and complications of anesthesia. To mitigate
such risks, real-time systems are desired to be developed that can provide
intra-operative guidance to surgeons. For example, an automated system for tool
localization, tool (or tissue) tracking, and depth estimation can enable a
clear understanding of surgical scenes preventing miscalculations during
surgical procedures. In this work, we present a systematic review of recent
machine learning-based approaches including surgical tool localization,
segmentation, tracking, and 3D scene perception. Furthermore, we provide a
detailed overview of publicly available benchmark datasets widely used for
surgical navigation tasks. While recent deep learning architectures have shown
promising results, there are still several open research problems such as a
lack of annotated datasets, the presence of artifacts in surgical scenes, and
non-textured surfaces that hinder 3D reconstruction of the anatomical
structures. Based on our comprehensive review, we present a discussion on
current gaps and needed steps to improve the adaptation of technology in
surgery.Comment: This paper is to be submitted to International journal of computer
visio
LightDepth: Single-View Depth Self-Supervision from Illumination Decline
Single-view depth estimation can be remarkably effective if there is enough
ground-truth depth data for supervised training. However, there are scenarios,
especially in medicine in the case of endoscopies, where such data cannot be
obtained. In such cases, multi-view self-supervision and synthetic-to-real
transfer serve as alternative approaches, however, with a considerable
performance reduction in comparison to supervised case. Instead, we propose a
single-view self-supervised method that achieves a performance similar to the
supervised case. In some medical devices, such as endoscopes, the camera and
light sources are co-located at a small distance from the target surfaces.
Thus, we can exploit that, for any given albedo and surface orientation, pixel
brightness is inversely proportional to the square of the distance to the
surface, providing a strong single-view self-supervisory signal. In our
experiments, our self-supervised models deliver accuracies comparable to those
of fully supervised ones, while being applicable without depth ground-truth
data
Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging
Endoscopy is the most widely used imaging technique for the diagnosis of
cancerous lesions in hollow organs. However, endoscopic images are often
affected by illumination artefacts: image parts may be over- or underexposed
according to the light source pose and the tissue orientation. These artifacts
have a strong negative impact on the performance of computer vision or AI-based
diagnosis tools. Although endoscopic image enhancement methods are greatly
required, little effort has been devoted to over- and under-exposition
enhancement in real-time. This contribution presents an extension to the
objective function of LMSPEC, a method originally introduced to enhance images
from natural scenes. It is used here for the exposure correction in endoscopic
imaging and the preservation of structural information. To the best of our
knowledge, this contribution is the first one that addresses the enhancement of
endoscopic images using deep learning (DL) methods. Tested on the Endo4IE
dataset, the proposed implementation has yielded a significant improvement over
LMSPEC reaching a SSIM increase of 4.40% and 4.21% for over- and underexposed
images, respectively.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessibl
3 Dimensional Dense Reconstruction: A Review of Algorithms and Dataset
3D dense reconstruction refers to the process of obtaining the complete shape
and texture features of 3D objects from 2D planar images. 3D reconstruction is
an important and extensively studied problem, but it is far from being solved.
This work systematically introduces classical methods of 3D dense
reconstruction based on geometric and optical models, as well as methods based
on deep learning. It also introduces datasets for deep learning and the
performance and advantages and disadvantages demonstrated by deep learning
methods on these datasets.Comment: 16 page
Learning-based depth and pose prediction for 3D scene reconstruction in endoscopy
Colorectal cancer is the third most common cancer worldwide. Early detection and treatment of pre-cancerous tissue during colonoscopy is critical to improving prognosis. However, navigating within the colon and inspecting the endoluminal tissue comprehensively are challenging, and success in both varies based on the endoscopist's skill and experience. Computer-assisted interventions in colonoscopy show much promise in improving navigation and inspection. For instance, 3D reconstruction of the colon during colonoscopy could promote more thorough examinations and increase adenoma detection rates which are associated with improved survival rates. Given the stakes, this thesis seeks to advance the state of research from feature-based traditional methods closer to a data-driven 3D reconstruction pipeline for colonoscopy.
More specifically, this thesis explores different methods that improve subtasks of learning-based 3D reconstruction. The main tasks are depth prediction and camera pose estimation. As training data is unavailable, the author, together with her co-authors, proposes and publishes several synthetic datasets and promotes domain adaptation models to improve applicability to real data. We show, through extensive experiments, that our depth prediction methods produce more robust results than previous work. Our pose estimation network trained on our new synthetic data outperforms self-supervised methods on real sequences. Our box embeddings allow us to interpret the geometric relationship and scale difference between two images of the same surface without the need for feature matches that are often unobtainable in surgical scenes. Together, the methods introduced in this thesis help work towards a complete, data-driven 3D reconstruction pipeline for endoscopy
Deep Causal Learning for Robotic Intelligence
This invited review discusses causal learning in the context of robotic
intelligence. The paper introduced the psychological findings on causal
learning in human cognition, then it introduced the traditional statistical
solutions on causal discovery and causal inference. The paper reviewed recent
deep causal learning algorithms with a focus on their architectures and the
benefits of using deep nets and discussed the gap between deep causal learning
and the needs of robotic intelligence
Artificial intelligence and automation in endoscopy and surgery
Modern endoscopy relies on digital technology, from high-resolution imaging sensors and displays to electronics connecting configurable illumination and actuation systems for robotic articulation. In addition to enabling more effective diagnostic and therapeutic interventions, the digitization of the procedural toolset enables video data capture of the internal human anatomy at unprecedented levels. Interventional video data encapsulate functional and structural information about a patient’s anatomy as well as events, activity and action logs about the surgical process. This detailed but difficult-to-interpret record from endoscopic procedures can be linked to preoperative and postoperative records or patient imaging information. Rapid advances in artificial intelligence, especially in supervised deep learning, can utilize data from endoscopic procedures to develop systems for assisting procedures leading to computer-assisted interventions that can enable better navigation during procedures, automation of image interpretation and robotically assisted tool manipulation. In this Perspective, we summarize state-of-the-art artificial intelligence for computer-assisted interventions in gastroenterology and surgery