21 research outputs found
Don't Forget The Past: Recurrent Depth Estimation from Monocular Video
Autonomous cars need continuously updated depth information. Thus far, depth
is mostly estimated independently for a single frame at a time, even if the
method starts from video input. Our method produces a time series of depth
maps, which makes it an ideal candidate for online learning approaches. In
particular, we put three different types of depth estimation (supervised depth
prediction, self-supervised depth prediction, and self-supervised depth
completion) into a common framework. We integrate the corresponding networks
with a ConvLSTM such that the spatiotemporal structures of depth across frames
can be exploited to yield a more accurate depth estimation. Our method is
flexible. It can be applied to monocular videos only or be combined with
different types of sparse depth patterns. We carefully study the architecture
of the recurrent network and its training strategy. We are first to
successfully exploit recurrent networks for real-time self-supervised monocular
depth estimation and completion. Extensive experiments show that our recurrent
method outperforms its image-based counterpart consistently and significantly
in both self-supervised scenarios. It also outperforms previous depth
estimation methods of the three popular groups. Please refer to
https://www.trace.ethz.ch/publications/2020/rec_depth_estimation/ for details.Comment: Please refer to our webpage for details
https://www.trace.ethz.ch/publications/2020/rec_depth_estimation
SimCol3D -- 3D Reconstruction during Colonoscopy Challenge
Colorectal cancer is one of the most common cancers in the world. While
colonoscopy is an effective screening technique, navigating an endoscope
through the colon to detect polyps is challenging. A 3D map of the observed
surfaces could enhance the identification of unscreened colon tissue and serve
as a training platform. However, reconstructing the colon from video footage
remains unsolved due to numerous factors such as self-occlusion, reflective
surfaces, lack of texture, and tissue deformation that limit feature-based
methods. Learning-based approaches hold promise as robust alternatives, but
necessitate extensive datasets. By establishing a benchmark, the 2022 EndoVis
sub-challenge SimCol3D aimed to facilitate data-driven depth and pose
prediction during colonoscopy. The challenge was hosted as part of MICCAI 2022
in Singapore. Six teams from around the world and representatives from academia
and industry participated in the three sub-challenges: synthetic depth
prediction, synthetic pose prediction, and real pose prediction. This paper
describes the challenge, the submitted methods, and their results. We show that
depth prediction in virtual colonoscopy is robustly solvable, while pose
estimation remains an open research question