57 research outputs found
A Joint Intensity and Depth Co-Sparse Analysis Model for Depth Map Super-Resolution
High-resolution depth maps can be inferred from low-resolution depth
measurements and an additional high-resolution intensity image of the same
scene. To that end, we introduce a bimodal co-sparse analysis model, which is
able to capture the interdependency of registered intensity and depth
information. This model is based on the assumption that the co-supports of
corresponding bimodal image structures are aligned when computed by a suitable
pair of analysis operators. No analytic form of such operators exist and we
propose a method for learning them from a set of registered training signals.
This learning process is done offline and returns a bimodal analysis operator
that is universally applicable to natural scenes. We use this to exploit the
bimodal co-sparse analysis model as a prior for solving inverse problems, which
leads to an efficient algorithm for depth map super-resolution.Comment: 13 pages, 4 figure
Generative Prior for Unsupervised Image Restoration
The challenge of restoring real world low-quality images is due to a lack of appropriate training data and difficulty in determining how the image was degraded. Recently, generative models have demonstrated great potential for creating high- quality images by utilizing the rich and diverse information contained within the model’s trained weights and learned latent representations. One popular type of generative model is the generative adversarial network (GAN). Many new methods have been developed to harness the information found in GANs for image manipulation. Our proposed approach is to utilize generative models for both understanding the degradation of an image and restoring it. We propose using a combination of cycle consistency losses and self-attention to enhance face images by first learning the degradation and then using this information to train a style-based neural network. We also aim to use the latent representation to achieve a high level of magnification for face images (x64). By incorporating the weights of a pre-trained StyleGAN into a restoration network with a vision transformer layer, we hope to improve the current state-of-the-art in face image restoration. Finally, we present a projection-based image-denoising algorithm named Noise2Code in the latent space of the VQGAN model with a fixed-point regularization strategy. The fixed-point condition follows the observation that the pre-trained VQGAN affects the clean and noisy images in a drastically different way. Unlike previous projection-based image restoration in the latent space, both the denoising network and VQGAN model parameters are jointly trained, although the latter is not needed during the testing. We report experimental results to demonstrate that the proposed Noise2Code approach is conceptually simple, computationally efficient, and generalizable to real-world degradation scenarios
Image-guided ToF depth upsampling: a survey
Recently, there has been remarkable growth of interest in the development and applications of time-of-flight (ToF) depth cameras. Despite the permanent improvement of their characteristics, the practical applicability of ToF cameras is still limited by low resolution and quality of depth measurements. This has motivated many researchers to combine ToF cameras with other sensors in order to enhance and upsample depth images. In this paper, we review the approaches that couple ToF depth images with high-resolution optical images. Other classes of upsampling methods are also briefly discussed. Finally, we provide an overview of performance evaluation tests presented in the related studies
Unsharp Mask Guided Filtering
The goal of this paper is guided image filtering, which emphasizes the
importance of structure transfer during filtering by means of an additional
guidance image. Where classical guided filters transfer structures using
hand-designed functions, recent guided filters have been considerably advanced
through parametric learning of deep networks. The state-of-the-art leverages
deep networks to estimate the two core coefficients of the guided filter. In
this work, we posit that simultaneously estimating both coefficients is
suboptimal, resulting in halo artifacts and structure inconsistencies. Inspired
by unsharp masking, a classical technique for edge enhancement that requires
only a single coefficient, we propose a new and simplified formulation of the
guided filter. Our formulation enjoys a filtering prior from a low-pass filter
and enables explicit structure transfer by estimating a single coefficient.
Based on our proposed formulation, we introduce a successive guided filtering
network, which provides multiple filtering results from a single network,
allowing for a trade-off between accuracy and efficiency. Extensive ablations,
comparisons and analysis show the effectiveness and efficiency of our
formulation and network, resulting in state-of-the-art results across filtering
tasks like upsampling, denoising, and cross-modality filtering. Code is
available at \url{https://github.com/shizenglin/Unsharp-Mask-Guided-Filtering}.Comment: IEEE Transactions on Image Processing, 202
Infrared Image Super-Resolution: Systematic Review, and Future Trends
Image Super-Resolution (SR) is essential for a wide range of computer vision
and image processing tasks. Investigating infrared (IR) image (or thermal
images) super-resolution is a continuing concern within the development of deep
learning. This survey aims to provide a comprehensive perspective of IR image
super-resolution, including its applications, hardware imaging system dilemmas,
and taxonomy of image processing methodologies. In addition, the datasets and
evaluation metrics in IR image super-resolution tasks are also discussed.
Furthermore, the deficiencies in current technologies and possible promising
directions for the community to explore are highlighted. To cope with the rapid
development in this field, we intend to regularly update the relevant excellent
work at \url{https://github.com/yongsongH/Infrared_Image_SR_SurveyComment: Submitted to IEEE TNNL
Explicit Edge Inconsistency Evaluation Model for Color-Guided Depth Map Enhancement
© 2016 IEEE. Color-guided depth enhancement is used to refine depth maps according to the assumption that the depth edges and the color edges at the corresponding locations are consistent. In methods on such low-level vision tasks, the Markov random field (MRF), including its variants, is one of the major approaches that have dominated this area for several years. However, the assumption above is not always true. To tackle the problem, the state-of-the-art solutions are to adjust the weighting coefficient inside the smoothness term of the MRF model. These methods lack an explicit evaluation model to quantitatively measure the inconsistency between the depth edge map and the color edge map, so they cannot adaptively control the efforts of the guidance from the color image for depth enhancement, leading to various defects such as texture-copy artifacts and blurring depth edges. In this paper, we propose a quantitative measurement on such inconsistency and explicitly embed it into the smoothness term. The proposed method demonstrates promising experimental results compared with the benchmark and state-of-the-art methods on the Middlebury ToF-Mark, and NYU data sets
- …