719 research outputs found
Modeling Camera Effects to Improve Visual Learning from Synthetic Data
Recent work has focused on generating synthetic imagery to increase the size
and variability of training data for learning visual tasks in urban scenes.
This includes increasing the occurrence of occlusions or varying environmental
and weather effects. However, few have addressed modeling variation in the
sensor domain. Sensor effects can degrade real images, limiting
generalizability of network performance on visual tasks trained on synthetic
data and tested in real environments. This paper proposes an efficient,
automatic, physically-based augmentation pipeline to vary sensor effects
--chromatic aberration, blur, exposure, noise, and color cast-- for synthetic
imagery. In particular, this paper illustrates that augmenting synthetic
training datasets with the proposed pipeline reduces the domain gap between
synthetic and real domains for the task of object detection in urban driving
scenes
Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding
This work addresses the problem of semantic scene understanding under dense
fog. Although considerable progress has been made in semantic scene
understanding, it is mainly related to clear-weather scenes. Extending
recognition methods to adverse weather conditions such as fog is crucial for
outdoor applications. In this paper, we propose a novel method, named
Curriculum Model Adaptation (CMAda), which gradually adapts a semantic
segmentation model from light synthetic fog to dense real fog in multiple
steps, using both synthetic and real foggy data. In addition, we present three
other main stand-alone contributions: 1) a novel method to add synthetic fog to
real, clear-weather scenes using semantic input; 2) a new fog density
estimator; 3) the Foggy Zurich dataset comprising real foggy images,
with pixel-level semantic annotations for images with dense fog. Our
experiments show that 1) our fog simulation slightly outperforms a
state-of-the-art competing simulation with respect to the task of semantic
foggy scene understanding (SFSU); 2) CMAda improves the performance of
state-of-the-art models for SFSU significantly by leveraging unlabeled real
foggy data. The datasets and code are publicly available.Comment: final version, ECCV 201
Realistic Noise Synthesis with Diffusion Models
Deep learning-based approaches have achieved remarkable performance in
single-image denoising. However, training denoising models typically requires a
large amount of data, which can be difficult to obtain in real-world scenarios.
Furthermore, synthetic noise used in the past has often produced significant
differences compared to real-world noise due to the complexity of the latter
and the poor modeling ability of noise distributions of Generative Adversarial
Network (GAN) models, resulting in residual noise and artifacts within
denoising models. To address these challenges, we propose a novel method for
synthesizing realistic noise using diffusion models. This approach enables us
to generate large amounts of high-quality data for training denoising models by
controlling camera settings to simulate different environmental conditions and
employing guided multi-scale content information to ensure that our method is
more capable of generating real noise with multi-frequency spatial
correlations. In particular, we design an inversion mechanism for the setting,
which extends our method to more public datasets without setting information.
Based on the noise dataset we synthesized, we have conducted sufficient
experiments on multiple benchmarks, and experimental results demonstrate that
our method outperforms state-of-the-art methods on multiple benchmarks and
metrics, demonstrating its effectiveness in synthesizing realistic noise for
training denoising models
- …