240 research outputs found
Object-based attention mechanism for color calibration of UAV remote sensing images in precision agriculture.
Color calibration is a critical step for unmanned aerial vehicle (UAV) remote sensing, especially in precision agriculture, which relies mainly on correlating color changes to specific quality attributes, e.g. plant health, disease, and pest stresses. In UAV remote sensing, the exemplar-based color transfer is popularly used for color calibration, where the automatic search for the semantic correspondences is the key to ensuring the color transfer accuracy. However, the existing attention mechanisms encounter difficulties in building the precise semantic correspondences between the reference image and the target one, in which the normalized cross correlation is often computed for feature reassembling. As a result, the color transfer accuracy is inevitably decreased by the disturbance from the semantically unrelated pixels, leading to semantic mismatch due to the absence of semantic correspondences. In this article, we proposed an unsupervised object-based attention mechanism (OBAM) to suppress the disturbance of the semantically unrelated pixels, along with a further introduced weight-adjusted Adaptive Instance Normalization (AdaIN) (WAA) method to tackle the challenges caused by the absence of semantic correspondences. By embedding the proposed modules into a photorealistic style transfer method with progressive stylization, the color transfer accuracy can be improved while better preserving the structural details. We evaluated our approach on the UAV data of different crop types including rice, beans, and cotton. Extensive experiments demonstrate that our proposed method outperforms several state-of-the-art methods. As our approach requires no annotated labels, it can be easily embedded into the off-the-shelf color transfer approaches. Relevant codes and configurations will be available at https://github.com/huanghsheng/object-based-attention-mechanis
Ultrafast Photorealistic Style Transfer via Neural Architecture Search
The key challenge in photorealistic style transfer is that an algorithm
should faithfully transfer the style of a reference photo to a content photo
while the generated image should look like one captured by a camera. Although
several photorealistic style transfer algorithms have been proposed, they need
to rely on post- and/or pre-processing to make the generated images look
photorealistic. If we disable the additional processing, these algorithms would
fail to produce plausible photorealistic stylization in terms of detail
preservation and photorealism. In this work, we propose an effective solution
to these issues. Our method consists of a construction step (C-step) to build a
photorealistic stylization network and a pruning step (P-step) for
acceleration. In the C-step, we propose a dense auto-encoder named PhotoNet
based on a carefully designed pre-analysis. PhotoNet integrates a feature
aggregation module (BFA) and instance normalized skip links (INSL). To generate
faithful stylization, we introduce multiple style transfer modules in the
decoder and INSLs. PhotoNet significantly outperforms existing algorithms in
terms of both efficiency and effectiveness. In the P-step, we adopt a neural
architecture search method to accelerate PhotoNet. We propose an automatic
network pruning framework in the manner of teacher-student learning for
photorealistic stylization. The network architecture named PhotoNAS resulted
from the search achieves significant acceleration over PhotoNet while keeping
the stylization effects almost intact. We conduct extensive experiments on both
image and video transfer. The results show that our method can produce
favorable results while achieving 20-30 times acceleration in comparison with
the existing state-of-the-art approaches. It is worth noting that the proposed
algorithm accomplishes better performance without any pre- or post-processing
Inverting Adversarially Robust Networks for Image Synthesis
Recent research in adversarially robust classifiers suggests their
representations tend to be aligned with human perception, which makes them
attractive for image synthesis and restoration applications. Despite favorable
empirical results on a few downstream tasks, their advantages are limited to
slow and sensitive optimization-based techniques. Moreover, their use on
generative models remains unexplored. This work proposes the use of robust
representations as a perceptual primitive for feature inversion models, and
show its benefits with respect to standard non-robust image features. We
empirically show that adopting robust representations as an image prior
significantly improves the reconstruction accuracy of CNN-based feature
inversion models. Furthermore, it allows reconstructing images at multiple
scales out-of-the-box. Following these findings, we propose an
encoding-decoding network based on robust representations and show its
advantages for applications such as anomaly detection, style transfer and image
denoising
- …