355 research outputs found
Jointly trained image and video generation using residual vectors
In this work, we propose a modeling technique for jointly training image and
video generation models by simultaneously learning to map latent variables with
a fixed prior onto real images and interpolate over images to generate videos.
The proposed approach models the variations in representations using residual
vectors encoding the change at each time step over a summary vector for the
entire video. We utilize the technique to jointly train an image generation
model with a fixed prior along with a video generation model lacking
constraints such as disentanglement. The joint training enables the image
generator to exploit temporal information while the video generation model
learns to flexibly share information across frames. Moreover, experimental
results verify our approach's compatibility with pre-training on videos or
images and training on datasets containing a mixture of both. A comprehensive
set of quantitative and qualitative evaluations reveal the improvements in
sample quality and diversity over both video generation and image generation
baselines. We further demonstrate the technique's capabilities of exploiting
similarity in features across frames by applying it to a model based on
decomposing the video into motion and content. The proposed model allows minor
variations in content across frames while maintaining the temporal dependence
through latent vectors encoding the pose or motion features.Comment: Accepted in 2020 Winter Conference on Applications of Computer Vision
(WACV '20
A Commentary on the Unsupervised Learning of Disentangled Representations
The goal of the unsupervised learning of disentangled representations is to
separate the independent explanatory factors of variation in the data without
access to supervision. In this paper, we summarize the results of Locatello et
al., 2019, and focus on their implications for practitioners. We discuss the
theoretical result showing that the unsupervised learning of disentangled
representations is fundamentally impossible without inductive biases and the
practical challenges it entails. Finally, we comment on our experimental
findings, highlighting the limitations of state-of-the-art approaches and
directions for future research
Flow Factorized Representation Learning
A prominent goal of representation learning research is to achieve
representations which are factorized in a useful manner with respect to the
ground truth factors of variation. The fields of disentangled and equivariant
representation learning have approached this ideal from a range of
complimentary perspectives; however, to date, most approaches have proven to
either be ill-specified or insufficiently flexible to effectively separate all
realistic factors of interest in a learned latent space. In this work, we
propose an alternative viewpoint on such structured representation learning
which we call Flow Factorized Representation Learning, and demonstrate it to
learn both more efficient and more usefully structured representations than
existing frameworks. Specifically, we introduce a generative model which
specifies a distinct set of latent probability paths that define different
input transformations. Each latent flow is generated by the gradient field of a
learned potential following dynamic optimal transport. Our novel setup brings
new understandings to both \textit{disentanglement} and \textit{equivariance}.
We show that our model achieves higher likelihoods on standard representation
learning benchmarks while simultaneously being closer to approximately
equivariant models. Furthermore, we demonstrate that the transformations
learned by our model are flexibly composable and can also extrapolate to new
data, implying a degree of robustness and generalizability approaching the
ultimate goal of usefully factorized representation learning.Comment: NeurIPS2
Learning Disentangled Representations with Latent Variation Predictability
Latent traversal is a popular approach to visualize the disentangled latent
representations. Given a bunch of variations in a single unit of the latent
representation, it is expected that there is a change in a single factor of
variation of the data while others are fixed. However, this impressive
experimental observation is rarely explicitly encoded in the objective function
of learning disentangled representations. This paper defines the variation
predictability of latent disentangled representations. Given image pairs
generated by latent codes varying in a single dimension, this varied dimension
could be closely correlated with these image pairs if the representation is
well disentangled. Within an adversarial generation process, we encourage
variation predictability by maximizing the mutual information between latent
variations and corresponding image pairs. We further develop an evaluation
metric that does not rely on the ground-truth generative factors to measure the
disentanglement of latent representations. The proposed variation
predictability is a general constraint that is applicable to the VAE and GAN
frameworks for boosting disentanglement of latent representations. Experiments
show that the proposed variation predictability correlates well with existing
ground-truth-required metrics and the proposed algorithm is effective for
disentanglement learning.Comment: 14 pages, ECCV2
Benchmarking bias mitigation algorithms in representation learning through fairness metrics
Le succès des modèles d’apprentissage en profondeur et leur adoption rapide dans de nombreux
domaines d’application ont soulevé d’importantes questions sur l’équité de ces modèles lorsqu’ils
sont déployés dans le monde réel. Des études récentes ont mis en évidence les biais encodés
par les algorithmes d’apprentissage des représentations et ont remis en cause la fiabilité de telles
approches pour prendre des décisions. En conséquence, il existe un intérêt croissant pour la
compréhension des sources de biais dans l’apprentissage des algorithmes et le développement de
stratégies d’atténuation des biais. L’objectif des algorithmes d’atténuation des biais est d’atténuer
l’influence des caractéristiques des données sensibles sur les décisions d’éligibilité prises. Les
caractéristiques sensibles sont des caractéristiques privées et protégées d’un ensemble de données
telles que le sexe ou la race, qui ne devraient pas affecter les décisions de sortie d’éligibilité, c’està -dire les critères qui rendent un individu qualifié ou non qualifié pour une tâche donnée, comme
l’octroi de prêts ou l’embauche. Les modèles d’atténuation des biais visent à prendre des décisions
d’éligibilité sur des échantillons d’ensembles de données sans biais envers les attributs sensibles
des données d’entrée. La difficulté des tâches d’atténuation des biais est souvent déterminée par
la distribution de l’ensemble de données, qui à son tour est fonction du déséquilibre potentiel de
l’étiquette et des caractéristiques, de la corrélation des caractéristiques potentiellement sensibles
avec d’autres caractéristiques des données, du décalage de la distribution de l’apprentissage vers
le phase de développement, etc. Sans l’évaluation des modèles d’atténuation des biais dans
diverses configurations difficiles, leurs mérites restent incertains. Par conséquent, une analyse
systématique qui comparerait différentes approches d’atténuation des biais sous la perspective de
différentes mesures d’équité pour assurer la réplication des résultats conclus est nécessaire. À
cette fin, nous proposons un cadre unifié pour comparer les approches d’atténuation des biais.
Nous évaluons différentes méthodes d’équité formées avec des réseaux de neurones profonds sur
un ensemble de données synthétiques commun et un ensemble de données du monde réel pour
obtenir de meilleures informations sur le fonctionnement de ces méthodes. En particulier, nous
formons environ 3000 modèles différents dans diverses configurations, y compris des configurations
de données déséquilibrées et corrélées, pour vérifier les limites des modèles actuels et mieux
comprendre dans quelles configurations ils sont sujets à des défaillances. Nos résultats montrent que
le biais des modèles augmente à mesure que les ensembles de données deviennent plus déséquilibrés
ou que les attributs des ensembles de données deviennent plus corrélés, le niveau de dominance
des caractéristiques des ensembles de données sensibles corrélées a un impact sur le biais, et
les informations sensibles restent dans la représentation latente même lorsque des algorithmes
d’atténuation des biais sont appliqués. Résumant nos contributions - nous présentons un ensemble
de données, proposons diverses configurations d’évaluation difficiles et évaluons rigoureusement
les récents algorithmes prometteurs d’atténuation des biais dans un cadre commun et publions
publiquement cette référence, en espérant que la communauté des chercheurs le considérerait
comme un point d’entrée commun pour un apprentissage en profondeur équitable.The rapid use and success of deep learning models in various application domains have raised
significant challenges about the fairness of these models when used in the real world. Recent
research has shown the biases incorporated within representation learning algorithms, raising
doubts about the dependability of such decision-making systems. As a result, there is a growing
interest in identifying the sources of bias in learning algorithms and developing bias-mitigation
techniques. The bias-mitigation algorithms aim to reduce the impact of sensitive data aspects on
eligibility choices. Sensitive features are private and protected features of a dataset, such as gender
of the person or race, that should not influence output eligibility decisions, i.e., the criteria that
determine whether or not an individual is qualified for a particular activity, such as lending or
hiring. Bias mitigation models are designed to make eligibility choices on dataset samples without
bias toward sensitive input data properties. The dataset distribution, which is a function of the
potential label and feature imbalance, the correlation of potentially sensitive features with other
features in the data, the distribution shift from training to the development phase, and other factors,
determines the difficulty of bias-mitigation tasks. Without evaluating bias-mitigation models in
various challenging setups, the merits of deep learning approaches to these tasks remain unclear.
As a result, a systematic analysis is required to compare different bias-mitigation procedures using
various fairness criteria to ensure that the final results are replicated. In order to do so, this thesis
offers a single paradigm for comparing bias-mitigation methods. To better understand how these
methods work, we compare alternative fairness algorithms trained with deep neural networks on a
common synthetic dataset and a real-world dataset. We train around 3000 distinct models in various
setups, including imbalanced and correlated data configurations, to validate the present models’
limits and better understand which setups are prone to failure. Our findings show that as datasets
become more imbalanced or dataset attributes become more correlated, model bias increases, the
dominance of correlated sensitive dataset features influence bias, and sensitive data remains in the
latent representation even after bias-mitigation algorithms are applied. In summary, we present a
dataset, propose multiple challenging assessment scenarios, rigorously analyse recent promising
bias-mitigation techniques in a common framework, and openly disclose this benchmark as an entry
point for fair deep learning
- …