98 research outputs found
Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity
Most of the existing learning-based deraining methods are supervisedly
trained on synthetic rainy-clean pairs. The domain gap between the synthetic
and real rain makes them less generalized to complex real rainy scenes.
Moreover, the existing methods mainly utilize the property of the image or rain
layers independently, while few of them have considered their mutually
exclusive relationship. To solve above dilemma, we explore the intrinsic
intra-similarity within each layer and inter-exclusiveness between two layers
and propose an unsupervised non-local contrastive learning (NLCL) deraining
method. The non-local self-similarity image patches as the positives are
tightly pulled together, rain patches as the negatives are remarkably pushed
away, and vice versa. On one hand, the intrinsic self-similarity knowledge
within positive/negative samples of each layer benefits us to discover more
compact representation; on the other hand, the mutually exclusive property
between the two layers enriches the discriminative decomposition. Thus, the
internal self-similarity within each layer (similarity) and the external
exclusive relationship of the two layers (dissimilarity) serving as a generic
image prior jointly facilitate us to unsupervisedly differentiate the rain from
clean image. We further discover that the intrinsic dimension of the non-local
image patches is generally higher than that of the rain patches. This motivates
us to design an asymmetric contrastive loss to precisely model the compactness
discrepancy of the two layers for better discriminative decomposition. In
addition, considering that the existing real rain datasets are of low quality,
either small scale or downloaded from the internet, we collect a real
large-scale dataset under various rainy kinds of weather that contains
high-resolution rainy images.Comment: 16 pages, 15 figures. arXiv admin note: substantial text overlap with
arXiv:2203.1150
FD-GAN: Generative Adversarial Networks with Fusion-discriminator for Single Image Dehazing
Recently, convolutional neural networks (CNNs) have achieved great
improvements in single image dehazing and attained much attention in research.
Most existing learning-based dehazing methods are not fully end-to-end, which
still follow the traditional dehazing procedure: first estimate the medium
transmission and the atmospheric light, then recover the haze-free image based
on the atmospheric scattering model. However, in practice, due to lack of
priors and constraints, it is hard to precisely estimate these intermediate
parameters. Inaccurate estimation further degrades the performance of dehazing,
resulting in artifacts, color distortion and insufficient haze removal. To
address this, we propose a fully end-to-end Generative Adversarial Networks
with Fusion-discriminator (FD-GAN) for image dehazing. With the proposed
Fusion-discriminator which takes frequency information as additional priors,
our model can generator more natural and realistic dehazed images with less
color distortion and fewer artifacts. Moreover, we synthesize a large-scale
training dataset including various indoor and outdoor hazy images to boost the
performance and we reveal that for learning-based dehazing methods, the
performance is strictly influenced by the training data. Experiments have shown
that our method reaches state-of-the-art performance on both public synthetic
datasets and real-world images with more visually pleasing dehazed results.Comment: Accepted by AAAI2020 (with supplementary files
Mapping and Deep Analysis of Image Dehazing: Coherent Taxonomy, Datasets, Open Challenges, Motivations, and Recommendations
Our study aims to review and analyze the most relevant studies in the image dehazing field. Many aspects have been deemed necessary to provide a broad understanding of various studies that have been examined through surveying the existing literature. These aspects are as follows: datasets that have been used in the literature, challenges that other researchers have faced, motivations, and recommendations for diminishing the obstacles in the reported literature. A systematic protocol is employed to search all relevant articles on image dehazing, with variations in keywords, in addition to searching for evaluation and benchmark studies. The search process is established on three online databases, namely, IEEE Xplore, Web of Science (WOS), and ScienceDirect (SD), from 2008 to 2021. These indices are selected because they are sufficient in terms of coverage. Along with definition of the inclusion and exclusion criteria, we include 152 articles to the final set. A total of 55 out of 152 articles focused on various studies that conducted image dehazing, and 13 out 152 studies covered most of the review papers based on scenarios and general overviews. Finally, most of the included articles centered on the development of image dehazing algorithms based on real-time scenario (84/152) articles. Image dehazing removes unwanted visual effects and is often considered an image enhancement technique, which requires a fully automated algorithm to work under real-time outdoor applications, a reliable evaluation method, and datasets based on different weather conditions. Many relevant studies have been conducted to meet these critical requirements. We conducted objective image quality assessment experimental comparison of various image dehazing algorithms. In conclusions unlike other review papers, our study distinctly reflects different observations on image dehazing areas. We believe that the result of this study can serve as a useful guideline for practitioners who are looking for a comprehensive view on image dehazing
Joint Depth Estimation and Mixture of Rain Removal From a Single Image
Rainy weather significantly deteriorates the visibility of scene objects,
particularly when images are captured through outdoor camera lenses or
windshields. Through careful observation of numerous rainy photos, we have
found that the images are generally affected by various rainwater artifacts
such as raindrops, rain streaks, and rainy haze, which impact the image quality
from both near and far distances, resulting in a complex and intertwined
process of image degradation. However, current deraining techniques are limited
in their ability to address only one or two types of rainwater, which poses a
challenge in removing the mixture of rain (MOR). In this study, we propose an
effective image deraining paradigm for Mixture of rain REmoval, called
DEMore-Net, which takes full account of the MOR effect. Going beyond the
existing deraining wisdom, DEMore-Net is a joint learning paradigm that
integrates depth estimation and MOR removal tasks to achieve superior rain
removal. The depth information can offer additional meaningful guidance
information based on distance, thus better helping DEMore-Net remove different
types of rainwater. Moreover, this study explores normalization approaches in
image deraining tasks and introduces a new Hybrid Normalization Block (HNB) to
enhance the deraining performance of DEMore-Net. Extensive experiments
conducted on synthetic datasets and real-world MOR photos fully validate the
superiority of the proposed DEMore-Net. Code is available at
https://github.com/yz-wang/DEMore-Net.Comment: 11 pages, 7 figures, 5 table
Endoscopic video defogging using luminance blending.
Endoscopic video sequences provide surgeons with direct surgical field or visualisation on anatomical targets in the patient during robotic surgery. Unfortunately, these video images are unavoidably hazy or foggy to prevent surgeons from clear surgical vision due to typical surgical operations such as ablation and cauterisation during surgery. This Letter aims at removing fog or smoke on endoscopic video sequences to enhance and maintain a direct and clear visualisation of the operating field during robotic surgery. The authors propose a new luminance blending framework that integrates contrast enhancement with visibility restoration for foggy endoscopic video processing. The proposed method was validated on clinical endoscopic videos that were collected from robotic surgery. The experimental results demonstrate that their method provides a promising means to effectively remove fog or smoke on endoscopic video images. In particular, the visual quality of defogged endoscopic images was improved from 0.5088 to 0.6475
Intelligent Transportation Related Complex Systems and Sensors
Building around innovative services related to different modes of transport and traffic management, intelligent transport systems (ITS) are being widely adopted worldwide to improve the efficiency and safety of the transportation system. They enable users to be better informed and make safer, more coordinated, and smarter decisions on the use of transport networks. Current ITSs are complex systems, made up of several components/sub-systems characterized by time-dependent interactions among themselves. Some examples of these transportation-related complex systems include: road traffic sensors, autonomous/automated cars, smart cities, smart sensors, virtual sensors, traffic control systems, smart roads, logistics systems, smart mobility systems, and many others that are emerging from niche areas. The efficient operation of these complex systems requires: i) efficient solutions to the issues of sensors/actuators used to capture and control the physical parameters of these systems, as well as the quality of data collected from these systems; ii) tackling complexities using simulations and analytical modelling techniques; and iii) applying optimization techniques to improve the performance of these systems. It includes twenty-four papers, which cover scientific concepts, frameworks, architectures and various other ideas on analytics, trends and applications of transportation-related data
Artificial Intelligence in the Creative Industries: A Review
This paper reviews the current state of the art in Artificial Intelligence
(AI) technologies and applications in the context of the creative industries. A
brief background of AI, and specifically Machine Learning (ML) algorithms, is
provided including Convolutional Neural Network (CNNs), Generative Adversarial
Networks (GANs), Recurrent Neural Networks (RNNs) and Deep Reinforcement
Learning (DRL). We categorise creative applications into five groups related to
how AI technologies are used: i) content creation, ii) information analysis,
iii) content enhancement and post production workflows, iv) information
extraction and enhancement, and v) data compression. We critically examine the
successes and limitations of this rapidly advancing technology in each of these
areas. We further differentiate between the use of AI as a creative tool and
its potential as a creator in its own right. We foresee that, in the near
future, machine learning-based AI will be adopted widely as a tool or
collaborative assistant for creativity. In contrast, we observe that the
successes of machine learning in domains with fewer constraints, where AI is
the `creator', remain modest. The potential of AI (or its developers) to win
awards for its original creations in competition with human creatives is also
limited, based on contemporary technologies. We therefore conclude that, in the
context of creative industries, maximum benefit from AI will be derived where
its focus is human centric -- where it is designed to augment, rather than
replace, human creativity
- …