133,859 research outputs found
Edge-Aware Image Color Appearance and Difference Modeling
The perception of color is one of the most important aspects of human vision.
From an evolutionary perspective, the accurate perception of color is crucial
to distinguishing friend from foe, and food from fatal poison. As a result,
humans have developed a keen sense of color and are able to detect subtle
differences in appearance, while also robustly identifying colors across
illumination and viewing conditions. In this paper, we shall briefly review
methods for adapting traditional color appearance and difference models to
complex image stimuli, and propose mechanisms to improve their performance. In
particular, we find that applying contrast sensitivity functions and local
adaptation rules in an edge-aware manner improves image difference predictions
Nighttime Thermal Infrared Image Colorization with Feedback-based Object Appearance Learning
Stable imaging in adverse environments (e.g., total darkness) makes thermal
infrared (TIR) cameras a prevalent option for night scene perception. However,
the low contrast and lack of chromaticity of TIR images are detrimental to
human interpretation and subsequent deployment of RGB-based vision algorithms.
Therefore, it makes sense to colorize the nighttime TIR images by translating
them into the corresponding daytime color images (NTIR2DC). Despite the
impressive progress made in the NTIR2DC task, how to improve the translation
performance of small object classes is under-explored. To address this problem,
we propose a generative adversarial network incorporating feedback-based object
appearance learning (FoalGAN). Specifically, an occlusion-aware mixup module
and corresponding appearance consistency loss are proposed to reduce the
context dependence of object translation. As a representative example of small
objects in nighttime street scenes, we illustrate how to enhance the realism of
traffic light by designing a traffic light appearance loss. To further improve
the appearance learning of small objects, we devise a dual feedback learning
strategy to selectively adjust the learning frequency of different samples. In
addition, we provide pixel-level annotation for a subset of the Brno dataset,
which can facilitate the research of NTIR image understanding under multiple
weather conditions. Extensive experiments illustrate that the proposed FoalGAN
is not only effective for appearance learning of small objects, but also
outperforms other image translation methods in terms of semantic preservation
and edge consistency for the NTIR2DC task.Comment: 14 pages, 14 figures. arXiv admin note: text overlap with
arXiv:2208.0296
Segmentation-Aware Convolutional Networks Using Local Attention Masks
We introduce an approach to integrate segmentation information within a
convolutional neural network (CNN). This counter-acts the tendency of CNNs to
smooth information across regions and increases their spatial precision. To
obtain segmentation information, we set up a CNN to provide an embedding space
where region co-membership can be estimated based on Euclidean distance. We use
these embeddings to compute a local attention mask relative to every neuron
position. We incorporate such masks in CNNs and replace the convolution
operation with a "segmentation-aware" variant that allows a neuron to
selectively attend to inputs coming from its own region. We call the resulting
network a segmentation-aware CNN because it adapts its filters at each image
point according to local segmentation cues. We demonstrate the merit of our
method on two widely different dense prediction tasks, that involve
classification (semantic segmentation) and regression (optical flow). Our
results show that in semantic segmentation we can match the performance of
DenseCRFs while being faster and simpler, and in optical flow we obtain clearly
sharper responses than networks that do not use local attention masks. In both
cases, segmentation-aware convolution yields systematic improvements over
strong baselines. Source code for this work is available online at
http://cs.cmu.edu/~aharley/segaware
- …