85 research outputs found
Image stitching with perspective-preserving warping
Image stitching algorithms often adopt the global transformation, such as
homography, and work well for planar scenes or parallax free camera motions.
However, these conditions are easily violated in practice. With casual camera
motions, variable taken views, large depth change, or complex structures, it is
a challenging task for stitching these images. The global transformation model
often provides dreadful stitching results, such as misalignments or projective
distortions, especially perspective distortion. To this end, we suggest a
perspective-preserving warping for image stitching, which spatially combines
local projective transformations and similarity transformation. By weighted
combination scheme, our approach gradually extrapolates the local projective
transformations of the overlapping regions into the non-overlapping regions,
and thus the final warping can smoothly change from projective to similarity.
The proposed method can provide satisfactory alignment accuracy as well as
reduce the projective distortions and maintain the multi-perspective view.
Experiments on a variety of challenging images confirm the efficiency of the
approach.Comment: ISPRS 2016 - XXIII ISPRS Congress: Prague, Czech Republic, 201
Learning Thin-Plate Spline Motion and Seamless Composition for Parallax-Tolerant Unsupervised Deep Image Stitching
Traditional image stitching approaches tend to leverage increasingly complex
geometric features (point, line, edge, etc.) for better performance. However,
these hand-crafted features are only suitable for specific natural scenes with
adequate geometric structures. In contrast, deep stitching schemes overcome the
adverse conditions by adaptively learning robust semantic features, but they
cannot handle large-parallax cases due to homography-based registration. To
solve these issues, we propose UDIS++, a parallax-tolerant unsupervised deep
image stitching technique. First, we propose a robust and flexible warp to
model the image registration from global homography to local thin-plate spline
motion. It provides accurate alignment for overlapping regions and shape
preservation for non-overlapping regions by joint optimization concerning
alignment and distortion. Subsequently, to improve the generalization
capability, we design a simple but effective iterative strategy to enhance the
warp adaption in cross-dataset and cross-resolution applications. Finally, to
further eliminate the parallax artifacts, we propose to composite the stitched
image seamlessly by unsupervised learning for seam-driven composition masks.
Compared with existing methods, our solution is parallax-tolerant and free from
laborious designs of complicated geometric features for specific scenes.
Extensive experiments show our superiority over the SoTA methods, both
quantitatively and qualitatively. The code will be available at
https://github.com/nie-lang/UDIS2
Stitching for multi-view videos with large parallax based on adaptive pixel warping
Conventional stitching techniques for images and videos are based on smooth warping models, and therefore, they often fail to work on multi-view images and videos with large parallax captured by cameras with wide baselines. In this paper, we propose a novel video stitching algorithm for such challenging multi-view videos. We estimate the parameters of ground plane homography, fundamental matrix, and vertical vanishing points reliably, using both of the appearance and activity based feature matches validated by geometric constraints. We alleviate the parallax artifacts in stitching by adaptively warping the off-plane pixels into geometrically accurate matching positions through their ground plane pixels based on the epipolar geometry. We also exploit the inter-view and inter-frame correspondence matching information together to estimate the ground plane pixels reliably, which are then refined by energy minimization. Experimental results show that the proposed algorithm provides geometrically accurate stitching results of multi-view videos with large parallax and outperforms the state-of-the-art stitching methods qualitatively and quantitatively
Deep Rectangling for Image Stitching: A Learning Baseline
Stitched images provide a wide field-of-view (FoV) but suffer from unpleasant
irregular boundaries. To deal with this problem, existing image rectangling
methods devote to searching an initial mesh and optimizing a target mesh to
form the mesh deformation in two stages. Then rectangular images can be
generated by warping stitched images. However, these solutions only work for
images with rich linear structures, leading to noticeable distortions for
portraits and landscapes with non-linear objects. In this paper, we address
these issues by proposing the first deep learning solution to image
rectangling. Concretely, we predefine a rigid target mesh and only estimate an
initial mesh to form the mesh deformation, contributing to a compact one-stage
solution. The initial mesh is predicted using a fully convolutional network
with a residual progressive regression strategy. To obtain results with high
content fidelity, a comprehensive objective function is proposed to
simultaneously encourage the boundary rectangular, mesh shape-preserving, and
content perceptually natural. Besides, we build the first image stitching
rectangling dataset with a large diversity in irregular boundaries and scenes.
Experiments demonstrate our superiority over traditional methods both
quantitatively and qualitatively.Comment: Accepted by CVPR2022 (oral); Codes and dataset:
https://github.com/nie-lang/DeepRectanglin
- …