3 research outputs found
A Deep Ordinal Distortion Estimation Approach for Distortion Rectification
Distortion is widely existed in the images captured by popular wide-angle
cameras and fisheye cameras. Despite the long history of distortion
rectification, accurately estimating the distortion parameters from a single
distorted image is still challenging. The main reason is these parameters are
implicit to image features, influencing the networks to fully learn the
distortion information. In this work, we propose a novel distortion
rectification approach that can obtain more accurate parameters with higher
efficiency. Our key insight is that distortion rectification can be cast as a
problem of learning an ordinal distortion from a single distorted image. To
solve this problem, we design a local-global associated estimation network that
learns the ordinal distortion to approximate the realistic distortion
distribution. In contrast to the implicit distortion parameters, the proposed
ordinal distortion have more explicit relationship with image features, and
thus significantly boosts the distortion perception of neural networks.
Considering the redundancy of distortion information, our approach only uses a
part of distorted image for the ordinal distortion estimation, showing
promising applications in the efficient distortion rectification. To our
knowledge, we first unify the heterogeneous distortion parameters into a
learning-friendly intermediate representation through ordinal distortion,
bridging the gap between image feature and distortion rectification. The
experimental results demonstrate that our approach outperforms the
state-of-the-art methods by a significant margin, with approximately 23%
improvement on the quantitative evaluation while displaying the best
performance on visual appearance
Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images
Traditional feature-based image stitching technologies rely heavily on
feature detection quality, often failing to stitch images with few features or
low resolution. The learning-based image stitching solutions are rarely studied
due to the lack of labeled data, making the supervised methods unreliable. To
address the above limitations, we propose an unsupervised deep image stitching
framework consisting of two stages: unsupervised coarse image alignment and
unsupervised image reconstruction. In the first stage, we design an
ablation-based loss to constrain an unsupervised homography network, which is
more suitable for large-baseline scenes. Moreover, a transformer layer is
introduced to warp the input images in the stitching-domain space. In the
second stage, motivated by the insight that the misalignments in pixel-level
can be eliminated to a certain extent in feature-level, we design an
unsupervised image reconstruction network to eliminate the artifacts from
features to pixels. Specifically, the reconstruction network can be implemented
by a low-resolution deformation branch and a high-resolution refined branch,
learning the deformation rules of image stitching and enhancing the resolution
simultaneously. To establish an evaluation benchmark and train the learning
framework, a comprehensive real-world image dataset for unsupervised deep image
stitching is presented and released. Extensive experiments well demonstrate the
superiority of our method over other state-of-the-art solutions. Even compared
with the supervised solutions, our image stitching quality is still preferred
by users.Comment: Accepted by IEEE Transactions on Image Processin
Wide-angle Image Rectification: A Survey
Wide field-of-view (FOV) cameras, which capture a larger scene area than
narrow FOV cameras, are used in many applications including 3D reconstruction,
autonomous driving, and video surveillance. However, wide-angle images contain
distortions that violate the assumptions underlying pinhole camera models,
resulting in object distortion, difficulties in estimating scene distance,
area, and direction, and preventing the use of off-the-shelf deep models
trained on undistorted images for downstream computer vision tasks. Image
rectification, which aims to correct these distortions, can solve these
problems. In this paper, we comprehensively survey progress in wide-angle image
rectification from transformation models to rectification methods.
Specifically, we first present a detailed description and discussion of the
camera models used in different approaches. Then, we summarize several
distortion models including radial distortion and projection distortion. Next,
we review both traditional geometry-based image rectification methods and deep
learning-based methods, where the former formulate distortion parameter
estimation as an optimization problem and the latter treat it as a regression
problem by leveraging the power of deep neural networks. We evaluate the
performance of state-of-the-art methods on public datasets and show that
although both kinds of methods can achieve good results, these methods only
work well for specific camera models and distortion types. We also provide a
strong baseline model and carry out an empirical study of different distortion
models on synthetic datasets and real-world wide-angle images. Finally, we
discuss several potential research directions that are expected to further
advance this area in the future.Comment: Accepted by the International Journal of Computer Vision (IJCV). Both
the datasets and source code are available at
https://github.com/loong8888/WAI