257 research outputs found
2D Reconstruction of Small Intestine's Interior Wall
Examining and interpreting of a large number of wireless endoscopic images
from the gastrointestinal tract is a tiresome task for physicians. A practical
solution is to automatically construct a two dimensional representation of the
gastrointestinal tract for easy inspection. However, little has been done on
wireless endoscopic image stitching, let alone systematic investigation. The
proposed new wireless endoscopic image stitching method consists of two main
steps to improve the accuracy and efficiency of image registration. First, the
keypoints are extracted by Principle Component Analysis and Scale Invariant
Feature Transform (PCA-SIFT) algorithm and refined with Maximum Likelihood
Estimation SAmple Consensus (MLESAC) outlier removal to find the most reliable
keypoints. Second, the optimal transformation parameters obtained from first
step are fed to the Normalised Mutual Information (NMI) algorithm as an initial
solution. With modified Marquardt-Levenberg search strategy in a multiscale
framework, the NMI can find the optimal transformation parameters in the
shortest time. The proposed methodology has been tested on two different
datasets - one with real wireless endoscopic images and another with images
obtained from Micro-Ball (a new wireless cubic endoscopy system with six image
sensors). The results have demonstrated the accuracy and robustness of the
proposed methodology both visually and quantitatively.Comment: Journal draf
Retrieval and Registration of Long-Range Overlapping Frames for Scalable Mosaicking of In Vivo Fetoscopy
Purpose: The standard clinical treatment of Twin-to-Twin Transfusion Syndrome
consists in the photo-coagulation of undesired anastomoses located on the
placenta which are responsible to a blood transfer between the two twins. While
being the standard of care procedure, fetoscopy suffers from a limited
field-of-view of the placenta resulting in missed anastomoses. To facilitate
the task of the clinician, building a global map of the placenta providing a
larger overview of the vascular network is highly desired. Methods: To overcome
the challenging visual conditions inherent to in vivo sequences (low contrast,
obstructions or presence of artifacts, among others), we propose the following
contributions: (i) robust pairwise registration is achieved by aligning the
orientation of the image gradients, and (ii) difficulties regarding long-range
consistency (e.g. due to the presence of outliers) is tackled via a bag-of-word
strategy, which identifies overlapping frames of the sequence to be registered
regardless of their respective location in time. Results: In addition to visual
difficulties, in vivo sequences are characterised by the intrinsic absence of
gold standard. We present mosaics motivating qualitatively our methodological
choices and demonstrating their promising aspect. We also demonstrate
semi-quantitatively, via visual inspection of registration results, the
efficacy of our registration approach in comparison to two standard baselines.
Conclusion: This paper proposes the first approach for the construction of
mosaics of placenta in in vivo fetoscopy sequences. Robustness to visual
challenges during registration and long-range temporal consistency are
proposed, offering first positive results on in vivo data for which standard
mosaicking techniques are not applicable.Comment: Accepted for publication in International Journal of Computer
Assisted Radiology and Surgery (IJCARS
Matching algorithm performance analysis for autocalibration method of stereo vision
Stereo vision is one of the interesting research topics in the computer vision field. Two cameras are used to generate a disparity map, resulting in the depth estimation. Camera calibration is the most important step in stereo vision. The calibration step is used to generate an intrinsic parameter of each camera to get a better disparity map. In general, the calibration process is done manually by using a chessboard pattern, but this process is an exhausting task. Self-calibration is an important ability required to overcome this problem. Self-calibration required a robust and good matching algorithm to find the key feature between images as reference. The purpose of this paper is to analyze the performance of three matching algorithms for the autocalibration process. The matching algorithms used in this research are SIFT, SURF, and ORB. The result shows that SIFT performs better than other methods
Tracking and Mapping in Medical Computer Vision: A Review
As computer vision algorithms are becoming more capable, their applications
in clinical systems will become more pervasive. These applications include
diagnostics such as colonoscopy and bronchoscopy, guiding biopsies and
minimally invasive interventions and surgery, automating instrument motion and
providing image guidance using pre-operative scans. Many of these applications
depend on the specific visual nature of medical scenes and require designing
and applying algorithms to perform in this environment.
In this review, we provide an update to the field of camera-based tracking
and scene mapping in surgery and diagnostics in medical computer vision. We
begin with describing our review process, which results in a final list of 515
papers that we cover. We then give a high-level summary of the state of the art
and provide relevant background for those who need tracking and mapping for
their clinical applications. We then review datasets provided in the field and
the clinical needs therein. Then, we delve in depth into the algorithmic side,
and summarize recent developments, which should be especially useful for
algorithm designers and to those looking to understand the capability of
off-the-shelf methods. We focus on algorithms for deformable environments while
also reviewing the essential building blocks in rigid tracking and mapping
since there is a large amount of crossover in methods. Finally, we discuss the
current state of the tracking and mapping methods along with needs for future
algorithms, needs for quantification, and the viability of clinical
applications in the field. We conclude that new methods need to be designed or
combined to support clinical applications in deformable environments, and more
focus needs to be put into collecting datasets for training and evaluation.Comment: 31 pages, 17 figure
Unleashing the Power of Depth and Pose Estimation Neural Networks by Designing Compatible Endoscopic Images
Deep learning models have witnessed depth and pose estimation framework on
unannotated datasets as a effective pathway to succeed in endoscopic
navigation. Most current techniques are dedicated to developing more advanced
neural networks to improve the accuracy. However, existing methods ignore the
special properties of endoscopic images, resulting in an inability to fully
unleash the power of neural networks. In this study, we conduct a detail
analysis of the properties of endoscopic images and improve the compatibility
of images and neural networks, to unleash the power of current neural networks.
First, we introcude the Mask Image Modelling (MIM) module, which inputs partial
image information instead of complete image information, allowing the network
to recover global information from partial pixel information. This enhances the
network' s ability to perceive global information and alleviates the phenomenon
of local overfitting in convolutional neural networks due to local artifacts.
Second, we propose a lightweight neural network to enhance the endoscopic
images, to explicitly improve the compatibility between images and neural
networks. Extensive experiments are conducted on the three public datasets and
one inhouse dataset, and the proposed modules improve baselines by a large
margin. Furthermore, the enhanced images we proposed, which have higher network
compatibility, can serve as an effective data augmentation method and they are
able to extract more stable feature points in traditional feature point
matching tasks and achieve outstanding performance
A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy from Monocular Endoscopic Video
Generating accurate 3D reconstructions from endoscopic video is a promising
avenue for longitudinal radiation-free analysis of sinus anatomy and surgical
outcomes. Several methods for monocular reconstruction have been proposed,
yielding visually pleasant 3D anatomical structures by retrieving relative
camera poses with structure-from-motion-type algorithms and fusion of monocular
depth estimates. However, due to the complex properties of the underlying
algorithms and endoscopic scenes, the reconstruction pipeline may perform
poorly or fail unexpectedly. Further, acquiring medical data conveys additional
challenges, presenting difficulties in quantitatively benchmarking these
models, understanding failure cases, and identifying critical components that
contribute to their precision. In this work, we perform a quantitative analysis
of a self-supervised approach for sinus reconstruction using endoscopic
sequences paired with optical tracking and high-resolution computed tomography
acquired from nine ex-vivo specimens. Our results show that the generated
reconstructions are in high agreement with the anatomy, yielding an average
point-to-mesh error of 0.91 mm between reconstructions and CT segmentations.
However, in a point-to-point matching scenario, relevant for endoscope tracking
and navigation, we found average target registration errors of 6.58 mm. We
identified that pose and depth estimation inaccuracies contribute equally to
this error and that locally consistent sequences with shorter trajectories
generate more accurate reconstructions. These results suggest that achieving
global consistency between relative camera poses and estimated depths with the
anatomy is essential. In doing so, we can ensure proper synergy between all
components of the pipeline for improved reconstructions that will facilitate
clinical application of this innovative technology
- …