460 research outputs found
Sparse representation-based SAR imaging
There is increasing interest in using synthetic aperture radar (SAR) images in automated target recognition and decision-making tasks. The success of such tasks depends on how well the reconstructed SAR images exhibit certain features of the underlying scene. Based on the observation that typical underlying scenes
usually exhibit sparsity in terms of such features, we develop an image formation method which formulates the SAR imaging problem as a sparse signal representation problem. Sparse signal representation, which has mostly been exploited in real-valued problems, has many capabilities such as superresolution and feature enhancement for various reconstruction and recognition tasks. However, for problems of complex-valued nature, such as SAR, a key challenge is how to choose the dictionary and the representation scheme for effective sparse representation. Since we are usually interested in features of the magnitude of the SAR reflectivity field, our new approach is designed to sparsely represent the magnitude of the complex-valued scattered field. This turns the image reconstruction problem into a joint optimization problem over the representation of magnitude and phase of the underlying field reflectivities. We develop the mathematical framework for this method and propose an iterative solution for the corresponding joint optimization problem. Our experimental results demonstrate the superiority of this method over previous approaches in terms of both producing high quality SAR images as well as exhibiting robustness to uncertain or limited data
Sparse representation-based synthetic aperture radar imaging
There is increasing interest in using synthetic aperture radar (SAR) images in automated target recognition and decision-making tasks. The success of such tasks depends on how well the reconstructed SAR images exhibit certain features of the underlying scene. Based on the observation that typical underlying scenes
usually exhibit sparsity in terms of such features, we develop an image formation method which formulates the SAR imaging problem as a sparse signal representation problem. Sparse signal representation, which has mostly been exploited in real-valued problems, has many capabilities such as superresolution and feature enhancement for various reconstruction and recognition tasks. However, for problems of complex-valued nature, such as SAR, a key challenge is how to choose the dictionary and the representation scheme for effective sparse representation. Since we are usually interested in features of the magnitude of the SAR reflectivity field, our new approach is designed to sparsely represent the magnitude of the complex-valued scattered field. This turns the image reconstruction problem into a joint optimization problem over the representation of magnitude and phase of the underlying field reflectivities. We develop the mathematical framework for this method and propose an iterative solution for the corresponding joint optimization problem. Our experimental results demonstrate the superiority of this method over previous approaches in terms of both producing high quality SAR images as well as exhibiting robustness to uncertain or limited data
Blind Quality Assessment for Image Superresolution Using Deep Two-Stream Convolutional Networks
Numerous image superresolution (SR) algorithms have been proposed for
reconstructing high-resolution (HR) images from input images with lower spatial
resolutions. However, effectively evaluating the perceptual quality of SR
images remains a challenging research problem. In this paper, we propose a
no-reference/blind deep neural network-based SR image quality assessor
(DeepSRQ). To learn more discriminative feature representations of various
distorted SR images, the proposed DeepSRQ is a two-stream convolutional network
including two subcomponents for distorted structure and texture SR images.
Different from traditional image distortions, the artifacts of SR images cause
both image structure and texture quality degradation. Therefore, we choose the
two-stream scheme that captures different properties of SR inputs instead of
directly learning features from one image stream. Considering the human visual
system (HVS) characteristics, the structure stream focuses on extracting
features in structural degradations, while the texture stream focuses on the
change in textural distributions. In addition, to augment the training data and
ensure the category balance, we propose a stride-based adaptive cropping
approach for further improvement. Experimental results on three publicly
available SR image quality databases demonstrate the effectiveness and
generalization ability of our proposed DeepSRQ method compared with
state-of-the-art image quality assessment algorithms
INR-V: A Continuous Representation Space for Video-based Generative Tasks
Generating videos is a complex task that is accomplished by generating a set
of temporally coherent images frame-by-frame. This limits the expressivity of
videos to only image-based operations on the individual video frames needing
network designs to obtain temporally coherent trajectories in the underlying
image space. We propose INR-V, a video representation network that learns a
continuous space for video-based generative tasks. INR-V parameterizes videos
using implicit neural representations (INRs), a multi-layered perceptron that
predicts an RGB value for each input pixel location of the video. The INR is
predicted using a meta-network which is a hypernetwork trained on neural
representations of multiple video instances. Later, the meta-network can be
sampled to generate diverse novel videos enabling many downstream video-based
generative tasks. Interestingly, we find that conditional regularization and
progressive weight initialization play a crucial role in obtaining INR-V. The
representation space learned by INR-V is more expressive than an image space
showcasing many interesting properties not possible with the existing works.
For instance, INR-V can smoothly interpolate intermediate videos between known
video instances (such as intermediate identities, expressions, and poses in
face videos). It can also in-paint missing portions in videos to recover
temporally coherent full videos. In this work, we evaluate the space learned by
INR-V on diverse generative tasks such as video interpolation, novel video
generation, video inversion, and video inpainting against the existing
baselines. INR-V significantly outperforms the baselines on several of these
demonstrated tasks, clearly showcasing the potential of the proposed
representation space.Comment: Published in Transactions on Machine Learning Research (10/2022);
https://openreview.net/forum?id=aIoEkwc2o
INR-V: A Continuous Representation Space for Video-based Generative Tasks
Generating videos is a complex task that is accomplished by generating a set of temporally coherent images frame-by-frame. This limits the expressivity of videos to only image-based operations on the individual video frames needing network designs to obtain temporally coherent trajectories in the underlying image space. We propose INR-V, a video representation network that learns a continuous space for video-based generative tasks. INR-V parameterizes videos using implicit neural representations (INRs), a multi-layered perceptron that predicts an RGB value for each input pixel location of the video. The INR is predicted using a meta-network which is a hypernetwork trained on neural representations of multiple video instances. Later, the meta-network can be sampled to generate diverse novel videos enabling many downstream video-based generative tasks. Interestingly, we find that conditional regularization and progressive weight initialization play a crucial role in obtaining INR-V. The representation space learned by INR-V is more expressive than an image space showcasing many interesting properties not possible with the existing works. For instance, INR-V can smoothly interpolate intermediate videos between known video instances (such as intermediate identities, expressions, and poses in face videos). It can also in-paint missing portions in videos to recover temporally coherent full videos. In this work, we evaluate the space learned by INR-V on diverse generative tasks such as video interpolation, novel video generation, video inversion, and video inpainting against the existing baselines. INR-V significantly outperforms the baselines on several of these demonstrated tasks, clearly showing the potential of the proposed representation space
Heavy Rain Face Image Restoration: Integrating Physical Degradation Model and Facial Component Guided Adversarial Learning
With the recent increase in intelligent CCTVs for visual surveillance, a new
image degradation that integrates resolution conversion and synthetic rain
models is required. For example, in heavy rain, face images captured by CCTV
from a distance have significant deterioration in both visibility and
resolution. Unlike traditional image degradation models (IDM), such as rain
removal and superresolution, this study addresses a new IDM referred to as a
scale-aware heavy rain model and proposes a method for restoring
high-resolution face images (HR-FIs) from low-resolution heavy rain face images
(LRHR-FI). To this end, a 2-stage network is presented. The first stage
generates low-resolution face images (LR-FIs), from which heavy rain has been
removed from the LRHR-FIs to improve visibility. To realize this, an
interpretable IDM-based network is constructed to predict physical parameters,
such as rain streaks, transmission maps, and atmospheric light. In addition,
the image reconstruction loss is evaluated to enhance the estimates of the
physical parameters. For the second stage, which aims to reconstruct the HR-FIs
from the LR-FIs outputted in the first stage, facial component guided
adversarial learning (FCGAL) is applied to boost facial structure expressions.
To focus on informative facial features and reinforce the authenticity of
facial components, such as the eyes and nose, a face-parsing-guided generator
and facial local discriminators are designed for FCGAL. The experimental
results verify that the proposed approach based on physical-based network
design and FCGAL can remove heavy rain and increase the resolution and
visibility simultaneously. Moreover, the proposed heavy-rain face image
restoration outperforms state-of-the-art models of heavy rain removal,
image-to-image translation, and superresolution
- …