5 research outputs found
Assembling convolution neural networks for automatic viewing transformation
Images taken under different camera poses are
rotated or distorted, which leads to poor perception experiences.
This paper proposes a new framework to automatically transform
the images to the conformable view setting by assembling
different convolution neural networks. Specifically, a referential
3D ground plane is firstly derived from the RGB image and
a novel projection mapping algorithm is developed to achieve
automatic viewing transformation. Extensive experimental results
demonstrate that the proposed method outperforms the state-ofthe-art vanishing points based methods by a large margin in
terms of accuracy and robustness
Video Upright Adjustment and Stabilization
Upright adjustment, Video stabilization, Camera pathWe propose a novel video upright adjustment method that can reliably correct slanted video contents that are often found in casual videos. Our approach combines deep learning and Bayesian inference to estimate accurate rotation angles from video frames. We train a convolutional neural network to obtain initial estimates of the rotation angles of input video frames. The initial estimates from the network are temporally inconsistent and inaccurate. To resolve this, we use Bayesian inference. We analyze estimation errors of the network, and derive an error model. We then use the error model to formulate video upright adjustment as a maximum a posteriori problem where we estimate consistent rotation angles from the initial estimates, while respecting relative rotations between consecutive frames. Finally, we propose a joint approach to video stabilization and upright adjustment, which minimizes information loss caused by separately handling stabilization and upright adjustment. Experimental results show that our video upright adjustment method can effectively correct slanted video contents, and its combination with video stabilization can achieve visually pleasing results from shaky and slanted videos.openI. INTRODUCTION
1.1. Related work
II. ROTATION ESTIMATION NETWORK
III. ERROR ANALYSIS
IV. VIDEO UPRIGHT ADJUSTMENT
4.1. Initial angle estimation
4.2. Robust angle estimation
4.3. Optimization
4.4. Warping
V. JOINT UPRIGHT ADJUSTMENT AND STABILIZATION
5.1. Bundled camera paths for video stabilization
5.2. Joint approach
VI. EXPERIMENTS
VII. CONCLUSION
ReferencesCNN)μ νλ ¨μν¨λ€. μ κ²½λ§μ μ΄κΈ° μΆμ μΉλ μμ ν μ ννμ§ μμΌλ©° μκ°μ μΌλ‘λ μΌκ΄λμ§ μλλ€. μ΄λ₯Ό ν΄κ²°νκΈ° μν΄ λ² μ΄μ§μ μΈνΌλ°μ€λ₯Ό μ¬μ©νλ€. λ³Έ λ
Όλ¬Έμ μ κ²½λ§μ μΆμ μ€λ₯λ₯Ό λΆμνκ³ μ€λ₯ λͺ¨λΈμ λμΆνλ€. κ·Έλ° λ€μ μ€λ₯ λͺ¨λΈμ μ¬μ©νμ¬ μ°μ νλ μ κ°μ μλ νμ κ°λ(Relative rotation angle)λ₯Ό λ°μνλ©΄μ μ΄κΈ° μΆμ μΉλ‘λΆν° μκ°μ μΌλ‘ μΌκ΄λ νμ κ°λλ₯Ό μΆμ νλ μ΅λ μ¬ν λ¬Έμ (Maximum a posteriori problem)λ‘ λμμ μν 보μ μ 곡μννλ€. λ§μ§λ§μΌλ‘, λμμ μν 보μ λ° λμμ μμ ν(Video stabilization)μ λν λμ μ κ·Ό λ°©λ²μ μ μνμ¬ μν 보μ κ³Ό μμ νλ₯Ό λ³λλ‘ μνν λ λ°μνλ κ³΅κ° μ 보 μμ€κ³Ό μ°μ°λμ μ΅μννλ©° μμ νμ μ±λ₯μ μ΅λννλ€. μ€ν κ²°κ³Όμ λ°λ₯΄λ©΄ λμμ μν 보μ μΌλ‘ κΈ°μΈμ΄μ§ λμμμ ν¨κ³Όμ μΌλ‘ 보μ ν μ μμΌλ©° λμμ μμ ν λ°©λ²κ³Ό κ²°ν©νμ¬ νλ€λ¦¬κ³ κΈ°μΈμ΄μ§ λμμμΌλ‘λΆν° μκ°μ μΌλ‘ λ§μ‘±μ€λ¬μ΄ μλ‘μ΄ λμμμ νλν μ μλ€.λ³Έ λ
Όλ¬Έμ μΌλ°μΈλ€μ΄ 촬μν λμμμμ νν λ°μνλ λ¬Έμ μΈ κΈ°μΈμ΄μ§μ μ κ±°νμ¬ μνμ΄ μ¬λ°λ₯Έ λμμμ νλν μ μκ² νλ λμμ μν 보μ (Video upright adjustment) λ°©λ²μ μ μνλ€. λ³Έ λ
Όλ¬Έμ μ κ·Ό λ°©μμ λ₯ λ¬λ(Deep learning)κ³Ό λ² μ΄μ§μ μΈνΌλ°μ€(Bayesian inference)λ₯Ό κ²°ν©νμ¬ λμμ νλ μ(Frame)μμ μ νν κ°λλ₯Ό μΆμ νλ€. λ¨Όμ μ
λ ₯ λμμ νλ μμ νμ κ°λμ μ΄κΈ° μΆμ μΉλ₯Ό μ»κΈ° μν΄ νμ μ κ²½λ§(Convolutional neural networkMasterdCollectio
Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery
Street-view imagery provides us with novel experiences to explore different
places remotely. Carefully calibrated street-view images (e.g. Google Street
View) can be used for different downstream tasks, e.g. navigation, map features
extraction. As personal high-quality cameras have become much more affordable
and portable, an enormous amount of crowdsourced street-view images are
uploaded to the internet, but commonly with missing or noisy sensor
information. To prepare this hidden treasure for "ready-to-use" status,
determining missing location information and camera orientation angles are two
equally important tasks. Recent methods have achieved high performance on
geo-localization of street-view images by cross-view matching with a pool of
geo-referenced satellite imagery. However, most of the existing works focus
more on geo-localization than estimating the image orientation. In this work,
we re-state the importance of finding fine-grained orientation for street-view
images, formally define the problem and provide a set of evaluation metrics to
assess the quality of the orientation estimation. We propose two methods to
improve the granularity of the orientation estimation, achieving 82.4% and
72.3% accuracy for images with estimated angle errors below 2 degrees for CVUSA
and CVACT datasets, corresponding to 34.9% and 28.2% absolute improvement
compared to previous works. Integrating fine-grained orientation estimation in
training also improves the performance on geo-localization, giving top 1 recall
95.5%/85.5% and 86.8%/80.4% for orientation known/unknown tests on the two
datasets.Comment: This paper has been accepted by ACM Multimedia 2022. The version
contains additional supplementary material
Robust upright adjustment of 360 spherical panoramas
With the recent advent of 360 cameras, spherical panorama images are becoming more popular and widely available. In a spherical panorama, alignment of the scene orientation to the image axes is important for providing comfortable and pleasant viewing experiences using VR headsets and traditional displays. This paper presents an automatic method for upright adjustment of 360 spherical panorama images without any prior information, such as depths and gyro sensor data. We take the Atlanta world assumption and use the horizontal and vertical lines in the scene to formulate a cost function for upright adjustment. In addition to fast optimization of the cost function, our method includes outlier handling to improve the robustness and accuracy of upright adjustment. Our method produces visually pleasing results for a variety of real-world spherical panoramas in less than a second, and the accuracy is verified using ground-truth data. ? 2017, Springer-Verlag Berlin Heidelberg.112sciescopu