5,666 research outputs found

    Natural and Realistic Single Image Super-Resolution with Explicit Natural Manifold Discrimination

    Full text link
    Recently, many convolutional neural networks for single image super-resolution (SISR) have been proposed, which focus on reconstructing the high-resolution images in terms of objective distortion measures. However, the networks trained with objective loss functions generally fail to reconstruct the realistic fine textures and details that are essential for better perceptual quality. Recovering the realistic details remains a challenging problem, and only a few works have been proposed which aim at increasing the perceptual quality by generating enhanced textures. However, the generated fake details often make undesirable artifacts and the overall image looks somewhat unnatural. Therefore, in this paper, we present a new approach to reconstructing realistic super-resolved images with high perceptual quality, while maintaining the naturalness of the result. In particular, we focus on the domain prior properties of SISR problem. Specifically, we define the naturalness prior in the low-level domain and constrain the output image in the natural manifold, which eventually generates more natural and realistic images. Our results show better naturalness compared to the recent super-resolution algorithms including perception-oriented ones.Comment: Presented in CVPR 201

    Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions

    Full text link
    Generative Adversarial Networks (GANs) is a novel class of deep generative models which has recently gained significant attention. GANs learns complex and high-dimensional distributions implicitly over images, audio, and data. However, there exists major challenges in training of GANs, i.e., mode collapse, non-convergence and instability, due to inappropriate design of network architecture, use of objective function and selection of optimization algorithm. Recently, to address these challenges, several solutions for better design and optimization of GANs have been investigated based on techniques of re-engineered network architectures, new objective functions and alternative optimization algorithms. To the best of our knowledge, there is no existing survey that has particularly focused on broad and systematic developments of these solutions. In this study, we perform a comprehensive survey of the advancements in GANs design and optimization solutions proposed to handle GANs challenges. We first identify key research issues within each design and optimization technique and then propose a new taxonomy to structure solutions by key research issues. In accordance with the taxonomy, we provide a detailed discussion on different GANs variants proposed within each solution and their relationships. Finally, based on the insights gained, we present the promising research directions in this rapidly growing field.Comment: 42 pages, Figure 13, Table

    Generative adversarial networks: an overview

    No full text
    Generative adversarial networks (GANs) provide a way to learn deep representations without extensively annotated training data. They achieve this by deriving backpropagation signals through a competitive process involving a pair of networks. The representations that can be learned by GANs may be used in a variety of applications, including image synthesis, semantic image editing, style transfer, image superresolution, and classification. The aim of this review article is to provide an overview of GANs for the signal processing community, drawing on familiar analogies and concepts where possible. In addition to identifying different methods for training and constructing GANs, we also point to remaining challenges in their theory and application

    Diverse Image Generation with Very Low Resolution Conditioning

    Get PDF
    Traditionnellement, lorsqu’il s’agit de gĂ©nĂ©rer des images Ă  haute rĂ©solution (HR) Ă  partir d’images Ă  basse rĂ©solution (LR), les mĂ©thodes proposĂ©es jusqu’à maintenant se sont principalement concentrĂ©es sur les techniques de super-rĂ©solution qui visent Ă  rĂ©cupĂ©rer l’image la plus probable Ă  partir d’une image de basse qualitĂ©. En procĂ©dant de cette maniĂšre, on ignore le fait qu’il existe gĂ©nĂ©ralement de nombreuses versions valides d’images HR qui correspondent Ă  une image LR donnĂ©e. L’objectif de ce travail est d’obtenir diffĂ©rentes versions d’images HR Ă  partir d’une mĂȘme image LR en utilisant un modĂšle adversarial gĂ©nĂ©ratif. On aborde ce problĂšme sous deux angles diffĂ©rents. D’abord, on utilise des mĂ©thodes de super rĂ©solution, oĂč en plus de l’image LR, le gĂ©nĂ©rateur peut ĂȘtre paramĂ©trĂ© par une variable latente afin de produire diffĂ©rentes variations potentielles de l’image. Un tel conditionnement permet de moduler le gĂ©nĂ©rateur entre la rĂ©cupĂ©ration de l’image la plus proche de la vĂ©ritĂ© terrain et de variĂ©tĂ© d’images possibles. Les rĂ©sultats dĂ©montrent notre supĂ©rioritĂ© en termes de reconstruction et de variĂ©tĂ© d’images hallucinĂ©es plausible par rapport Ă  d’autres mĂ©thodes de l’état de l’art. La deuxiĂšme approche s’appuie sur les travaux de traduction d’image Ă  image, en proposant une nouvelle approche oĂč le modĂšle est conditionnĂ© sur une version LR du cible. Plus prĂ©cisĂ©ment, notre approche vise Ă  transfĂ©rer les dĂ©tails fins d’une image source HR pour les adapter la structure gĂ©nĂ©rale, selon la version LR de celle-ci. On gĂ©nĂšre donc des images HR qui partagent les caractĂ©ristiques distinctives de l’image HR et qui correspond Ă  l’image LR de la cible lors de la rĂ©duction d’échelle. Cette mĂ©thode diffĂšre des mĂ©thodes prĂ©cĂ©dentes qui se concentrent plutĂŽt sur la traduction d’un style d’image donnĂ© en un contenu cible. Les rĂ©sultats qualitatifs et quantitatifs dĂ©montrent des amĂ©liorations en termes de qualitĂ© visuelle, de diversitĂ© et de couverture par rapport aux mĂ©thodes de pointe telles que Stargan-v2.Traditionally, when it comes to generating high-resolution (HR) images from a low-resolution(LR) images, the methods proposed so far have mainly focused on super-resolution techniques that aim at recovering the most probable image from low-quality image. Doing so ignores the fact that there are usually many valid versions of HR images that match a given LR image. The objective of this work is to obtain different versions of HR images from the same LR imageusing a generative adversarial model. We approach this problem from two different angles. First, we use super-resolution methods, where in addition to the LR image, the generator can be parameterized by a latent variable to produce different potential variations of the image. Such a conditioning allows to modulate the generator between retrieving the closest image to the ground truth and a variety of possible images. The results demonstrate our superiority in terms of reconstruction and variety of plausible hallucinated images compared to other state-of-the-art methods. The second approach builds on the work of image-to-image translation, by proposing a new approach where the model is conditioned on a LR version of the target. More precisely, our approach aims at transferring the fine details of an HR source image to fit the general structure, according to the LR version of it. We therefore generate HR images that share the distinctive features of the HR image and match the LR image of the target duringdownscaling. This method differs from previous methods that focus instead on translatinga given image style into target content. Qualitative and quantitative results demonstrate improvements in visual quality, diversity, and coverage over state-of-the-art methods such asStargan-v2
    • 

    corecore