609 research outputs found
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery
One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions
GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks
In the last decade, supervised deep learning approaches have been extensively
employed in visual odometry (VO) applications, which is not feasible in
environments where labelled data is not abundant. On the other hand,
unsupervised deep learning approaches for localization and mapping in unknown
environments from unlabelled data have received comparatively less attention in
VO research. In this study, we propose a generative unsupervised learning
framework that predicts 6-DoF pose camera motion and monocular depth map of the
scene from unlabelled RGB image sequences, using deep convolutional Generative
Adversarial Networks (GANs). We create a supervisory signal by warping view
sequences and assigning the re-projection minimization to the objective loss
function that is adopted in multi-view pose estimation and single-view depth
generation network. Detailed quantitative and qualitative evaluations of the
proposed framework on the KITTI and Cityscapes datasets show that the proposed
method outperforms both existing traditional and unsupervised deep VO methods
providing better results for both pose estimation and depth recovery.Comment: ICRA 2019 - accepte
Tracking and Mapping in Medical Computer Vision: A Review
As computer vision algorithms are becoming more capable, their applications
in clinical systems will become more pervasive. These applications include
diagnostics such as colonoscopy and bronchoscopy, guiding biopsies and
minimally invasive interventions and surgery, automating instrument motion and
providing image guidance using pre-operative scans. Many of these applications
depend on the specific visual nature of medical scenes and require designing
and applying algorithms to perform in this environment.
In this review, we provide an update to the field of camera-based tracking
and scene mapping in surgery and diagnostics in medical computer vision. We
begin with describing our review process, which results in a final list of 515
papers that we cover. We then give a high-level summary of the state of the art
and provide relevant background for those who need tracking and mapping for
their clinical applications. We then review datasets provided in the field and
the clinical needs therein. Then, we delve in depth into the algorithmic side,
and summarize recent developments, which should be especially useful for
algorithm designers and to those looking to understand the capability of
off-the-shelf methods. We focus on algorithms for deformable environments while
also reviewing the essential building blocks in rigid tracking and mapping
since there is a large amount of crossover in methods. Finally, we discuss the
current state of the tracking and mapping methods along with needs for future
algorithms, needs for quantification, and the viability of clinical
applications in the field. We conclude that new methods need to be designed or
combined to support clinical applications in deformable environments, and more
focus needs to be put into collecting datasets for training and evaluation.Comment: 31 pages, 17 figure
EndoSLAM Dataset and An Unsupervised Monocular Visual Odometry and Depth Estimation Approach for Endoscopic Videos: Endo-SfMLearner
Deep learning techniques hold promise to develop dense topography
reconstruction and pose estimation methods for endoscopic videos. However,
currently available datasets do not support effective quantitative
benchmarking. In this paper, we introduce a comprehensive endoscopic SLAM
dataset consisting of 3D point cloud data for six porcine organs, capsule and
standard endoscopy recordings as well as synthetically generated data. A Panda
robotic arm, two commercially available capsule endoscopes, two conventional
endoscopes with different camera properties, and two high precision 3D scanners
were employed to collect data from 8 ex-vivo porcine gastrointestinal
(GI)-tract organs. In total, 35 sub-datasets are provided with 6D pose ground
truth for the ex-vivo part: 18 sub-dataset for colon, 12 sub-datasets for
stomach and 5 sub-datasets for small intestine, while four of these contain
polyp-mimicking elevations carried out by an expert gastroenterologist.
Synthetic capsule endoscopy frames from GI-tract with both depth and pose
annotations are included to facilitate the study of simulation-to-real transfer
learning algorithms. Additionally, we propound Endo-SfMLearner, an unsupervised
monocular depth and pose estimation method that combines residual networks with
spatial attention module in order to dictate the network to focus on
distinguishable and highly textured tissue regions. The proposed approach makes
use of a brightness-aware photometric loss to improve the robustness under fast
frame-to-frame illumination changes. To exemplify the use-case of the EndoSLAM
dataset, the performance of Endo-SfMLearner is extensively compared with the
state-of-the-art. The codes and the link for the dataset are publicly available
at https://github.com/CapsuleEndoscope/EndoSLAM. A video demonstrating the
experimental setup and procedure is accessible through
https://www.youtube.com/watch?v=G_LCe0aWWdQ.Comment: 27 pages, 16 figure
- …