Variance Loss in Variational Autoencoders

In this article, we highlight what appears to be major issue of Variational Autoencoders, evinced from an extensive experimentation with different network architectures and datasets: the variance of generated data is significantly lower than that of training data. Since generative models are usually evaluated with metrics such as the Frechet Inception Distance (FID) that compare the distributions of (features of) real versus generated images, the variance loss typically results in degraded scores. This problem is particularly relevant in a two stage setting, where we use a second VAE to sample in the latent space of the first VAE. The minor variance creates a mismatch between the actual distribution of latent variables and those generated by the second VAE, that hinders the beneficial effects of the second stage. Renormalizing the output of the second VAE towards the expected normal spherical distribution, we obtain a sudden burst in the quality of generated samples, as also testified in terms of FID.Comment: Article accepted at the Sixth International Conference on Machine Learning, Optimization, and Data Science. July 19-23, 2020 - Certosa di Pontignano, Siena, Ital

Portrait Reification with Generative Diffusion Models

An application of Generative Diffusion Techniques for the reification of human portraits in artistic paintings is presented. By reification we intend the transformation of the painter's figurative abstraction into a real human face. The application exploits a recent embedding technique for Denoising Diffusion Implicit Models (DDIM) inverting the generative process and mapping the visible image into its latent representation. In this way, we can first embed the portrait into the latent space, and then use the reverse diffusion model, trained to generate real human faces, to produce the most likelihood real approximation of the portrait. The actual deployment of the application involves several additional techniques, mostly aimed to automatically identify, align and crop the relevant portion of the face, and to postprocess the generated reification in order to enhance its quality and to allow a smooth reinsertion in the original painting

Generazione di attributi facciali mediante Feature-wise Linear Modulation

L’aspetto dell’apprendimento automatico su cui si sta lavorando di più, negli ultimi anni, è quello della generazione di dati, come ad esempio suoni, testi e immagini. Un aspetto interessante nel campo della generazione è la possibilità di condizionare il modo in cui la rete neurale genera nuovi dati. Recentemente è stata introdotta la tecnica del Feature-wise Linear Modulation, abbreviato “FiLM”, usata per influenzare in modo adattivo l’output di una rete neurale basandosi su un input arbitrario, applicando una trasformazione affine sulle features intermedie della rete. Lo scopo dell’elaborato è mostrare l’integrazione di livelli FiLM all'interno di un modello Variational Autoencoder (VAE). Il modello così ottenuto verrà analizzato per le sue capacità di ricostruzione dell’input e di generazione di nuovi volti umani, sulla base di specifici attributi. Il modello verrà allenato sui volti presenti nel dataset CelebA e ne verrà valutata la capacità di ricostruzione e generazione attraverso la metrica della Fréchet Inception Distance (FID). Inoltre verrà condotto un piccolo esperimento per valutare la capacità del FID di discriminare alcuni attributi

Error Cause Analysis of Laboratory Results with the Help of AI

In the electronics laboratory, a large amount of different devices has to be tested, and the characterization of each of them generates a large amount of data. Classical root cause analysis is inherently inefficient because it requires manual inspection by experts, turning out to be costly and time consuming. Furthermore, automatic evaluations through sequences of conditions for the signals, ending up in hard-coded logical formulas, are still inappropriate due to still the necessity of experts, in order to design them specifically for each different device, and to the complexity of the data itself, which may lead to the infeasibility of such approach. For these reasons, in this thesis, first steps towards a machine learning (ML) approach are investigated, laying the foundation into the transition to a ML root cause analysis approach.In the electronics laboratory, a large amount of different devices has to be tested, and the characterization of each of them generates a large amount of data. Classical root cause analysis is inherently inefficient because it requires manual inspection by experts, turning out to be costly and time consuming. Furthermore, automatic evaluations through sequences of conditions for the signals, ending up in hard-coded logical formulas, are still inappropriate due to still the necessity of experts, in order to design them specifically for each different device, and to the complexity of the data itself, which may lead to the infeasibility of such approach. For these reasons, in this thesis, first steps towards a machine learning (ML) approach are investigated, laying the foundation into the transition to a ML root cause analysis approach

Comparison of Latent-Space Generative Models through Statistics and Mapping

Although data generation is a task with broad and exciting applications, samples created by generative models often fall victim to reduced variability and biases which, when coupled with the lack of explainability common to all neural networks, makes the evaluation of issues and limitations of these systems challenging. Much effort has been devoted to the exploration of the latent spaces of generative models in order to find more controllable editing directions and to the idea that better models would produce more disentangled representations. In this thesis we present a detailed and comparative analysis of latent-space generative models, beginning from their theoretical foundation and up to a number of statistical and empirical findings. We show that the original data is the sole factor truly impacting how different generative models learn, more than one may imagine: under the same dataset, even very different architectures distribute their latent spaces in essentially the same way. These results suggest new directions of research for representation learning, with the potential to transfer acquired knowledge between models and to understand the common mechanisms behind learning as a whole. Parts of the topics discussed in this thesis are a joint work, particularly those related to the mappings between models; they have already seen publication as a paper

Variance Loss in Variational Autoencoders

In this article, we highlight what appears to be major issue of Variational Autoencoders, evinced from an extensive experimentation with different network architectures and datasets: the variance of generated data is significantly lower than that of training data. Since generative models are usually evaluated with metrics such as the Frechet Inception Distance (FID) that compare the distributions of (features of) real versus generated images, the variance loss typically results in degraded scores. This problem is particularly relevant in a two stage setting, where we use a second VAE to sample in the latent space of the first VAE. The minor variance creates a mismatch between the actual distribution of latent variables and those generated by the second VAE, that hinders the beneficial effects of the second stage. Renormalizing the output of the second VAE towards the expected normal spherical distribution, we obtain a sudden burst in the quality of generated samples, as also testified in terms of FID