Diverse Image Generation with Very Low Resolution Conditioning

Traditionnellement, lorsqu’il s’agit de générer des images à haute résolution (HR) à partir d’images à basse résolution (LR), les méthodes proposées jusqu’à maintenant se sont principalement concentrées sur les techniques de super-résolution qui visent à récupérer l’image la plus probable à partir d’une image de basse qualité. En procédant de cette manière, on ignore le fait qu’il existe généralement de nombreuses versions valides d’images HR qui correspondent à une image LR donnée. L’objectif de ce travail est d’obtenir différentes versions d’images HR à partir d’une même image LR en utilisant un modèle adversarial génératif. On aborde ce problème sous deux angles différents. D’abord, on utilise des méthodes de super résolution, où en plus de l’image LR, le générateur peut être paramétré par une variable latente afin de produire différentes variations potentielles de l’image. Un tel conditionnement permet de moduler le générateur entre la récupération de l’image la plus proche de la vérité terrain et de variété d’images possibles. Les résultats démontrent notre supériorité en termes de reconstruction et de variété d’images hallucinées plausible par rapport à d’autres méthodes de l’état de l’art. La deuxième approche s’appuie sur les travaux de traduction d’image à image, en proposant une nouvelle approche où le modèle est conditionné sur une version LR du cible. Plus précisément, notre approche vise à transférer les détails fins d’une image source HR pour les adapter la structure générale, selon la version LR de celle-ci. On génère donc des images HR qui partagent les caractéristiques distinctives de l’image HR et qui correspond à l’image LR de la cible lors de la réduction d’échelle. Cette méthode diffère des méthodes précédentes qui se concentrent plutôt sur la traduction d’un style d’image donné en un contenu cible. Les résultats qualitatifs et quantitatifs démontrent des améliorations en termes de qualité visuelle, de diversité et de couverture par rapport aux méthodes de pointe telles que Stargan-v2.Traditionally, when it comes to generating high-resolution (HR) images from a low-resolution(LR) images, the methods proposed so far have mainly focused on super-resolution techniques that aim at recovering the most probable image from low-quality image. Doing so ignores the fact that there are usually many valid versions of HR images that match a given LR image. The objective of this work is to obtain different versions of HR images from the same LR imageusing a generative adversarial model. We approach this problem from two different angles. First, we use super-resolution methods, where in addition to the LR image, the generator can be parameterized by a latent variable to produce different potential variations of the image. Such a conditioning allows to modulate the generator between retrieving the closest image to the ground truth and a variety of possible images. The results demonstrate our superiority in terms of reconstruction and variety of plausible hallucinated images compared to other state-of-the-art methods. The second approach builds on the work of image-to-image translation, by proposing a new approach where the model is conditioned on a LR version of the target. More precisely, our approach aims at transferring the fine details of an HR source image to fit the general structure, according to the LR version of it. We therefore generate HR images that share the distinctive features of the HR image and match the LR image of the target duringdownscaling. This method differs from previous methods that focus instead on translatinga given image style into target content. Qualitative and quantitative results demonstrate improvements in visual quality, diversity, and coverage over state-of-the-art methods such asStargan-v2

SR-GAN for SR-gamma: photon super resolution at collider experiments

We study single-image super-resolution algorithms for photons at collider experiments based on generative adversarial networks. We treat the energy depositions of simulated electromagnetic showers of photons and neutral-pion decays in a toy electromagnetic calorimeter as 2D images and we train super-resolution networks to generate images with an artificially increased resolution by a factor of four in each dimension. The generated images are able to reproduce features of the electromagnetic showers that are not obvious from the images at nominal resolution. Using the artificially-enhanced images for the reconstruction of shower-shape variables and of the position of the shower center results in significant improvements. We additionally investigate the utilization of the generated images as a pre-processing step for deep-learning photon-identification algorithms and observe improvements in the case of low training statistics.Comment: 24 pages, 13 figure

Artificial Intelligence for Multimedia Signal Processing

Artificial intelligence technologies are also actively applied to broadcasting and multimedia processing technologies. A lot of research has been conducted in a wide variety of fields, such as content creation, transmission, and security, and these attempts have been made in the past two to three years to improve image, video, speech, and other data compression efficiency in areas related to MPEG media processing technology. Additionally, technologies such as media creation, processing, editing, and creating scenarios are very important areas of research in multimedia processing and engineering. This book contains a collection of some topics broadly across advanced computational intelligence algorithms and technologies for emerging multimedia signal processing as: Computer vision field, speech/sound/text processing, and content analysis/information mining

Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

On Martian Surface Exploration: Development of Automated 3D Reconstruction and Super-Resolution Restoration Techniques for Mars Orbital Images

Very high spatial resolution imaging and topographic (3D) data play an important role in modern Mars science research and engineering applications. This work describes a set of image processing and machine learning methods to produce the “best possible” high-resolution and high-quality 3D and imaging products from existing Mars orbital imaging datasets. The research work is described in nine chapters of which seven are based on separate published journal papers. These include a) a hybrid photogrammetric processing chain that combines the advantages of different stereo matching algorithms to compute stereo disparity with optimal completeness, fine-scale details, and minimised matching artefacts; b) image and 3D co-registration methods that correct a target image and/or 3D data to a reference image and/or 3D data to achieve robust cross-instrument multi-resolution 3D and image co-alignment; c) a deep learning network and processing chain to estimate pixel-scale surface topography from single-view imagery that outperforms traditional photogrammetric methods in terms of product quality and processing speed; d) a deep learning-based single-image super-resolution restoration (SRR) method to enhance the quality and effective resolution of Mars orbital imagery; e) a subpixel-scale 3D processing system using a combination of photogrammetric 3D reconstruction, SRR, and photoclinometric 3D refinement; and f) an optimised subpixel-scale 3D processing system using coupled deep learning based single-view SRR and deep learning based 3D estimation to derive the best possible (in terms of visual quality, effective resolution, and accuracy) 3D products out of present epoch Mars orbital images. The resultant 3D imaging products from the above listed new developments are qualitatively and quantitatively evaluated either in comparison with products from the official NASA planetary data system (PDS) and/or ESA planetary science archive (PSA) releases, and/or in comparison with products generated with different open-source systems. Examples of the scientific application of these novel 3D imaging products are discussed