321 research outputs found
Model Adaptation with Synthetic and Real Data for Semantic Dense Foggy Scene Understanding
This work addresses the problem of semantic scene understanding under dense
fog. Although considerable progress has been made in semantic scene
understanding, it is mainly related to clear-weather scenes. Extending
recognition methods to adverse weather conditions such as fog is crucial for
outdoor applications. In this paper, we propose a novel method, named
Curriculum Model Adaptation (CMAda), which gradually adapts a semantic
segmentation model from light synthetic fog to dense real fog in multiple
steps, using both synthetic and real foggy data. In addition, we present three
other main stand-alone contributions: 1) a novel method to add synthetic fog to
real, clear-weather scenes using semantic input; 2) a new fog density
estimator; 3) the Foggy Zurich dataset comprising real foggy images,
with pixel-level semantic annotations for images with dense fog. Our
experiments show that 1) our fog simulation slightly outperforms a
state-of-the-art competing simulation with respect to the task of semantic
foggy scene understanding (SFSU); 2) CMAda improves the performance of
state-of-the-art models for SFSU significantly by leveraging unlabeled real
foggy data. The datasets and code are publicly available.Comment: final version, ECCV 201
Synthetic image generation and the use of virtual environments for image enhancement tasks
Deep learning networks are often difficult to train if there are insufficient image samples. Gathering real-world images tailored for a specific job takes a lot of work to perform. This dissertation explores techniques for synthetic image generation and virtual environments for various image enhancement/ correction/restoration tasks, specifically distortion correction, dehazing, shadow removal, and intrinsic image decomposition. First, given various image formation equations, such as those used in distortion correction and dehazing, synthetic image samples can be produced, provided that the equation is well-posed. Second, using virtual environments to train various image models is applicable for simulating real-world effects that are otherwise difficult to gather or replicate, such as dehazing and shadow removal. Given synthetic images, one cannot train a network directly on it as there is a possible gap between the synthetic and real domains. We have devised several techniques for generating synthetic images and formulated domain adaptation methods where our trained deep-learning networks perform competitively in distortion correction, dehazing, and shadow removal. Additional studies and directions are provided for the intrinsic image decomposition problem and the exploration of procedural content generation, where a virtual Philippine city was created as an initial prototype.
Keywords: image generation, image correction, image dehazing, shadow removal, intrinsic image decomposition, computer graphics, rendering, machine learning, neural networks, domain adaptation, procedural content generation
Source-Free Domain Adaptation for Real-world Image Dehazing
Deep learning-based source dehazing methods trained on synthetic datasets
have achieved remarkable performance but suffer from dramatic performance
degradation on real hazy images due to domain shift. Although certain Domain
Adaptation (DA) dehazing methods have been presented, they inevitably require
access to the source dataset to reduce the gap between the source synthetic and
target real domains. To address these issues, we present a novel Source-Free
Unsupervised Domain Adaptation (SFUDA) image dehazing paradigm, in which only a
well-trained source model and an unlabeled target real hazy dataset are
available. Specifically, we devise the Domain Representation Normalization
(DRN) module to make the representation of real hazy domain features match that
of the synthetic domain to bridge the gaps. With our plug-and-play DRN module,
unlabeled real hazy images can adapt existing well-trained source networks.
Besides, the unsupervised losses are applied to guide the learning of the DRN
module, which consists of frequency losses and physical prior losses. Frequency
losses provide structure and style constraints, while the prior loss explores
the inherent statistic property of haze-free images. Equipped with our DRN
module and unsupervised loss, existing source dehazing models are able to
dehaze unlabeled real hazy images. Extensive experiments on multiple baselines
demonstrate the validity and superiority of our method visually and
quantitatively.Comment: Accepted to ACM MM 202
- …