Face Hallucination via Deep Neural Networks.

We firstly address aligned low-resolution (LR) face images (i.e. 16X16 pixels) by designing a discriminative generative network, named URDGN. URDGN is composed of two networks: a generative model and a discriminative model. We introduce a pixel-wise L2 regularization term to the generative model and exploit the feedback of the discriminative network to make the upsampled face images more similar to real ones. We present an end-to-end transformative discriminative neural network (TDN) devised for super-resolving unaligned tiny face images. TDN embeds spatial transformation layers to enforce local receptive fields to line-up with similar spatial supports. To upsample noisy unaligned LR face images, we propose decoder-encoder-decoder networks. A transformative discriminative decoder network is employed to upsample and denoise LR inputs simultaneously. Then we project the intermediate HR faces to aligned and noise-free LR faces by a transformative encoder network. Finally, high-quality hallucinated HR images are generated by our second decoder. Furthermore, we present an end-to-end multiscale transformative discriminative neural network (MTDN) to super-resolve unaligned LR face images of different resolutions in a unified framework. We propose a method that explicitly incorporates structural information of faces into the face super-resolution process by using a multi-task convolutional neural network (CNN). Our method not only uses low-level information (i.e. intensity similarity), but also middle-level information (i.e. face structure) to further explore spatial constraints of facial components from LR inputs images. We demonstrate that supplementing residual images or feature maps with additional facial attribute information can significantly reduce the ambiguity in face super-resolution. To explore this idea, we develop an attribute-embedded upsampling network. In this manner, our method is able to super-resolve LR faces by a large upscaling factor while reducing the uncertainty of one-to-many mappings remarkably. We further push the boundaries of hallucinating a tiny, non-frontal face image to understand how much of this is possible by leveraging the availability of large datasets and deep networks. To this end, we introduce a novel Transformative Adversarial Neural Network (TANN) to jointly frontalize very LR out-of-plane rotated face images (including profile views) and aggressively super-resolve them by 8X, regardless of their original poses and without using any 3D information. Besides recovering an HR face images from an LR version, this thesis also addresses the task of restoring realistic faces from stylized portrait images, which can also be regarded as face hallucination

A Comprehensive Review of Deep Learning-based Single Image Super-resolution

Image super-resolution (SR) is one of the vital image processing methods that improve the resolution of an image in the field of computer vision. In the last two decades, significant progress has been made in the field of super-resolution, especially by utilizing deep learning methods. This survey is an effort to provide a detailed survey of recent progress in single-image super-resolution in the perspective of deep learning while also informing about the initial classical methods used for image super-resolution. The survey classifies the image SR methods into four categories, i.e., classical methods, supervised learning-based methods, unsupervised learning-based methods, and domain-specific SR methods. We also introduce the problem of SR to provide intuition about image quality metrics, available reference datasets, and SR challenges. Deep learning-based approaches of SR are evaluated using a reference dataset. Some of the reviewed state-of-the-art image SR methods include the enhanced deep SR network (EDSR), cycle-in-cycle GAN (CinCGAN), multiscale residual network (MSRN), meta residual dense network (Meta-RDN), recurrent back-projection network (RBPN), second-order attention network (SAN), SR feedback network (SRFBN) and the wavelet-based residual attention network (WRAN). Finally, this survey is concluded with future directions and trends in SR and open problems in SR to be addressed by the researchers.Comment: 56 Pages, 11 Figures, 5 Table

Image Super-Resolution Based on Sparse Coding with Multi-Class Dictionaries

Sparse coding-based single image super-resolution has attracted much interest. In this paper, a super-resolution reconstruction algorithm based on sparse coding with multi-class dictionaries is put forward. We propose a novel method for image patch classification, using the phase congruency information. A sub-dictionary is learned from patches in each category. For a given image patch, the sub-dictionary that belongs to the same category is selected adaptively. Since the given patch has similar pattern with the selected sub-dictionary, it can be better represented. Finally, iterative back-projection is used to enforce global reconstruction constraint. Experiments demonstrate that our approach can produce comparable or even better super-resolution reconstruction results with some existing algorithms, in both subjective visual quality and numerical measures

Contributions to the Modelling of Auditory Hallucinations, Social robotics, and Multiagent Systems

165 p.The Thesis covers three diverse lines of work that have been tackled with the central endeavor of modeling and understanding the phenomena under consideration. Firstly, the Thesis works on the problem of finding brain connectivity biomarkers of auditory hallucinations, a rather frequent phenomena that can be related some pathologies, but which is also present in healthy population. We apply machine learning techniques to assess the significance of effective brain connections extracted by either dynamical causal modeling or Granger causality. Secondly, the Thesis deals with the usefulness of social robotics strorytelling as a therapeutic tools for children at risk of exclussion. The Thesis reports on the observations gathered in several therapeutic sessions carried out in Spain and Bulgaria, under the supervision of tutors and caregivers. Thirdly, the Thesis deals with the spatio-temporal dynamic modeling of social agents trying to explain the phenomena of opinion survival of the social minorities. The Thesis proposes a eco-social model endowed with spatial mobility of the agents. Such mobility and the spatial perception of the agents are found to be strong mechanisms explaining opinion propagation and survival