6 research outputs found
UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models
In classical computer vision, rectification is an integral part of multi-view
depth estimation. It typically includes epipolar rectification and lens
distortion correction. This process simplifies the depth estimation
significantly, and thus it has been adopted in CNN approaches. However,
rectification has several side effects, including a reduced field of view
(FOV), resampling distortion, and sensitivity to calibration errors. The
effects are particularly pronounced in case of significant distortion (e.g.,
wide-angle fisheye cameras). In this paper, we propose a generic scale-aware
self-supervised pipeline for estimating depth, euclidean distance, and visual
odometry from unrectified monocular videos. We demonstrate a similar level of
precision on the unrectified KITTI dataset with barrel distortion comparable to
the rectified KITTI dataset. The intuition being that the rectification step
can be implicitly absorbed within the CNN model, which learns the distortion
model without increasing complexity. Our approach does not suffer from a
reduced field of view and avoids computational costs for rectification at
inference time. To further illustrate the general applicability of the proposed
framework, we apply it to wide-angle fisheye cameras with 190
horizontal field of view. The training framework UnRectDepthNet takes in the
camera distortion model as an argument and adapts projection and unprojection
functions accordingly. The proposed algorithm is evaluated further on the KITTI
rectified dataset, and we achieve state-of-the-art results that improve upon
our previous work FisheyeDistanceNet. Qualitative results on a distorted test
scene video sequence indicate excellent performance
https://youtu.be/K6pbx3bU4Ss.Comment: Minor fixes added after IROS 2020 Camera ready submission. IROS 2020
presentation video - https://www.youtube.com/watch?v=3Br2KSWZRr
On Martian Surface Exploration: Development of Automated 3D Reconstruction and Super-Resolution Restoration Techniques for Mars Orbital Images
Very high spatial resolution imaging and topographic (3D) data play an important role in modern Mars science research and engineering applications. This work describes a set of image processing and machine learning methods to produce the ābest possibleā high-resolution and high-quality 3D and imaging products from existing Mars orbital imaging datasets. The research work is described in nine chapters of which seven are based on separate published journal papers. These include a) a hybrid photogrammetric processing chain that combines the advantages of different stereo matching algorithms to compute stereo disparity with optimal completeness, fine-scale details, and minimised matching artefacts; b) image and 3D co-registration methods that correct a target image and/or 3D data to a reference image and/or 3D data to achieve robust cross-instrument multi-resolution 3D and image co-alignment; c) a deep learning network and processing chain to estimate pixel-scale surface topography from single-view imagery that outperforms traditional photogrammetric methods in terms of product quality and processing speed; d) a deep learning-based single-image super-resolution restoration (SRR) method to enhance the quality and effective resolution of Mars orbital imagery; e) a subpixel-scale 3D processing system using a combination of photogrammetric 3D reconstruction, SRR, and photoclinometric 3D refinement; and f) an optimised subpixel-scale 3D processing system using coupled deep learning based single-view SRR and deep learning based 3D estimation to derive the best possible (in terms of visual quality, effective resolution, and accuracy) 3D products out of present epoch Mars orbital images. The resultant 3D imaging products from the above listed new developments are qualitatively and quantitatively evaluated either in comparison with products from the official NASA planetary data system (PDS) and/or ESA planetary science archive (PSA) releases, and/or in comparison with products generated with different open-source systems. Examples of the scientific application of these novel 3D imaging products are discussed