92 research outputs found
GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning
Existing optical flow methods are erroneous in challenging scenes, such as
fog, rain, and night because the basic optical flow assumptions such as
brightness and gradient constancy are broken. To address this problem, we
present an unsupervised learning approach that fuses gyroscope into optical
flow learning. Specifically, we first convert gyroscope readings into motion
fields named gyro field. Then, we design a self-guided fusion module to fuse
the background motion extracted from the gyro field with the optical flow and
guide the network to focus on motion details. To the best of our knowledge,
this is the first deep learning-based framework that fuses gyroscope data and
image content for optical flow learning. To validate our method, we propose a
new dataset that covers regular and challenging scenes. Experiments show that
our method outperforms the state-of-art methods in both regular and challenging
scenes
Exposure Fusion for Hand-held Camera Inputs with Optical Flow and PatchMatch
This paper proposes a hybrid synthesis method for multi-exposure image fusion
taken by hand-held cameras. Motions either due to the shaky camera or caused by
dynamic scenes should be compensated before any content fusion. Any
misalignment can easily cause blurring/ghosting artifacts in the fused result.
Our hybrid method can deal with such motions and maintain the exposure
information of each input effectively. In particular, the proposed method first
applies optical flow for a coarse registration, which performs well with
complex non-rigid motion but produces deformations at regions with missing
correspondences. The absence of correspondences is due to the occlusions of
scene parallax or the moving contents. To correct such error registration, we
segment images into superpixels and identify problematic alignments based on
each superpixel, which is further aligned by PatchMatch. The method combines
the efficiency of optical flow and the accuracy of PatchMatch. After PatchMatch
correction, we obtain a fully aligned image stack that facilitates a
high-quality fusion that is free from blurring/ghosting artifacts. We compare
our method with existing fusion algorithms on various challenging examples,
including the static/dynamic, the indoor/outdoor and the daytime/nighttime
scenes. Experiment results demonstrate the effectiveness and robustness of our
method
GyroFlow+: Gyroscope-Guided Unsupervised Deep Homography and Optical Flow Learning
Existing homography and optical flow methods are erroneous in challenging
scenes, such as fog, rain, night, and snow because the basic assumptions such
as brightness and gradient constancy are broken. To address this issue, we
present an unsupervised learning approach that fuses gyroscope into homography
and optical flow learning. Specifically, we first convert gyroscope readings
into motion fields named gyro field. Second, we design a self-guided fusion
module (SGF) to fuse the background motion extracted from the gyro field with
the optical flow and guide the network to focus on motion details. Meanwhile,
we propose a homography decoder module (HD) to combine gyro field and
intermediate results of SGF to produce the homography. To the best of our
knowledge, this is the first deep learning framework that fuses gyroscope data
and image content for both deep homography and optical flow learning. To
validate our method, we propose a new dataset that covers regular and
challenging scenes. Experiments show that our method outperforms the
state-of-the-art methods in both regular and challenging scenes.Comment: 12 pages. arXiv admin note: substantial text overlap with
arXiv:2103.1372
Nonlinear fatigue damage of cracked cement paste after grouting enhancement
Grouting reinforcement is an important part of modern engineering and has grown in popularity due to the benefits of grouting enhancement on cyclic loading. Understanding the fatigue mechanism of grouting-enhanced structures is vital to the design and the long-term stability analysis of such structures. In this study, the fatigue mechanical properties of cracked cement paste after epoxy resin grouting enhancement under different cyclic conditions were investigated in the laboratory and an inverted S-shaped curve was proposed to describe the damage accumulation. The test results indicated that the fatigue axial deformation can be divided into three stages: the initial stage, constant velocity stage and accelerating stage. The irreversible deformation can be used to describe the damage accumulation. The fatigue process is significantly affected by the upper limit stress level and the stress amplitude. In addition, the exponential relationship between stress amplitude and fatigue life was obtained. The proposed S-shaped curve was validated by an experimental fatigue strain test. The tests result upon various load conditions and crack types represented a good agreement with the predicted data
Supervised Homography Learning with Realistic Dataset Generation
In this paper, we propose an iterative framework, which consists of two
phases: a generation phase and a training phase, to generate realistic training
data and yield a supervised homography network. In the generation phase, given
an unlabeled image pair, we utilize the pre-estimated dominant plane masks and
homography of the pair, along with another sampled homography that serves as
ground truth to generate a new labeled training pair with realistic motion. In
the training phase, the generated data is used to train the supervised
homography network, in which the training data is refined via a content
consistency module and a quality assessment module. Once an iteration is
finished, the trained network is used in the next data generation phase to
update the pre-estimated homography. Through such an iterative strategy, the
quality of the dataset and the performance of the network can be gradually and
simultaneously improved. Experimental results show that our method achieves
state-of-the-art performance and existing supervised methods can be also
improved based on the generated dataset. Code and dataset are available at
https://github.com/megvii-research/RealSH.Comment: Accepted by ICCV 202
Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction
Image super-resolution (SR) aims to learn a mapping from low-resolution (LR)
to high-resolution (HR) using paired HR-LR training images. Conventional SR
methods typically gather the paired training data by synthesizing LR images
from HR images using a predetermined degradation model, e.g., Bicubic
down-sampling. However, the realistic degradation type of test images may
mismatch with the training-time degradation type due to the dynamic changes of
the real-world scenarios, resulting in inferior-quality SR images. To address
this, existing methods attempt to estimate the degradation model and train an
image-specific model, which, however, is quite time-consuming and impracticable
to handle rapidly changing domain shifts. Moreover, these methods largely
concentrate on the estimation of one degradation type (e.g., blur degradation),
overlooking other degradation types like noise and JPEG in real-world test-time
scenarios, thus limiting their practicality. To tackle these problems, we
present an efficient test-time adaptation framework for SR, named SRTTA, which
is able to quickly adapt SR models to test domains with different/unknown
degradation types. Specifically, we design a second-order degradation scheme to
construct paired data based on the degradation type of the test image, which is
predicted by a pre-trained degradation classifier. Then, we adapt the SR model
by implementing feature-level reconstruction learning from the initial test
image to its second-order degraded counterparts, which helps the SR model
generate plausible HR images. Extensive experiments are conducted on newly
synthesized corrupted DIV2K datasets with 8 different degradations and several
real-world datasets, demonstrating that our SRTTA framework achieves an
impressive improvement over existing methods with satisfying speed. The source
code is available at https://github.com/DengZeshuai/SRTTA.Comment: Accepted by 37th Conference on Neural Information Processing Systems
(NeurIPS 2023
GAFlow: Incorporating Gaussian Attention into Optical Flow
Optical flow, or the estimation of motion fields from image sequences, is one
of the fundamental problems in computer vision. Unlike most pixel-wise tasks
that aim at achieving consistent representations of the same category, optical
flow raises extra demands for obtaining local discrimination and smoothness,
which yet is not fully explored by existing approaches. In this paper, we push
Gaussian Attention (GA) into the optical flow models to accentuate local
properties during representation learning and enforce the motion affinity
during matching. Specifically, we introduce a novel Gaussian-Constrained Layer
(GCL) which can be easily plugged into existing Transformer blocks to highlight
the local neighborhood that contains fine-grained structural information.
Moreover, for reliable motion analysis, we provide a new Gaussian-Guided
Attention Module (GGAM) which not only inherits properties from Gaussian
distribution to instinctively revolve around the neighbor fields of each point
but also is empowered to put the emphasis on contextually related regions
during matching. Our fully-equipped model, namely Gaussian Attention Flow
network (GAFlow), naturally incorporates a series of novel Gaussian-based
modules into the conventional optical flow framework for reliable motion
analysis. Extensive experiments on standard optical flow datasets consistently
demonstrate the exceptional performance of the proposed approach in terms of
both generalization ability evaluation and online benchmark testing. Code is
available at https://github.com/LA30/GAFlow.Comment: To appear in ICCV-202
- …