308 research outputs found
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
The key idea behind the unsupervised learning of disentangled representations
is that real-world data is generated by a few explanatory factors of variation
which can be recovered by unsupervised learning algorithms. In this paper, we
provide a sober look at recent progress in the field and challenge some common
assumptions. We first theoretically show that the unsupervised learning of
disentangled representations is fundamentally impossible without inductive
biases on both the models and the data. Then, we train more than 12000 models
covering most prominent methods and evaluation metrics in a reproducible
large-scale experimental study on seven different data sets. We observe that
while the different methods successfully enforce properties ``encouraged'' by
the corresponding losses, well-disentangled models seemingly cannot be
identified without supervision. Furthermore, increased disentanglement does not
seem to lead to a decreased sample complexity of learning for downstream tasks.
Our results suggest that future work on disentanglement learning should be
explicit about the role of inductive biases and (implicit) supervision,
investigate concrete benefits of enforcing disentanglement of the learned
representations, and consider a reproducible experimental setup covering
several data sets
Improving Fairness of Graph Neural Networks: A Graph Counterfactual Perspective
Graph neural networks have shown great ability in representation (GNNs)
learning on graphs, facilitating various tasks. Despite their great performance
in modeling graphs, recent works show that GNNs tend to inherit and amplify the
bias from training data, causing concerns of the adoption of GNNs in high-stake
scenarios. Hence, many efforts have been taken for fairness-aware GNNs.
However, most existing fair GNNs learn fair node representations by adopting
statistical fairness notions, which may fail to alleviate bias in the presence
of statistical anomalies. Motivated by causal theory, there are several
attempts utilizing graph counterfactual fairness to mitigate root causes of
unfairness. However, these methods suffer from non-realistic counterfactuals
obtained by perturbation or generation. In this paper, we take a causal view on
fair graph learning problem. Guided by the casual analysis, we propose a novel
framework CAF, which can select counterfactuals from training data to avoid
non-realistic counterfactuals and adopt selected counterfactuals to learn fair
node representations for node classification task. Extensive experiments on
synthetic and real-world datasets show the effectiveness of CAF
Evaluating Disentanglement in Generative Models Without Knowledge of Latent Factors
Probabilistic generative models provide a flexible and systematic framework
for learning the underlying geometry of data. However, model selection in this
setting is challenging, particularly when selecting for ill-defined qualities
such as disentanglement or interpretability. In this work, we address this gap
by introducing a method for ranking generative models based on the training
dynamics exhibited during learning. Inspired by recent theoretical
characterizations of disentanglement, our method does not require supervision
of the underlying latent factors. We evaluate our approach by demonstrating the
need for disentanglement metrics which do not require labels\textemdash the
underlying generative factors. We additionally demonstrate that our approach
correlates with baseline supervised methods for evaluating disentanglement.
Finally, we show that our method can be used as an unsupervised indicator for
downstream performance on reinforcement learning and fairness-classification
problems
Rectifying Unfairness in Recommendation Feedback Loop
The issue of fairness in recommendation systems has recently become a matter of growing concern for both the academic and industrial sectors due to the potential for bias in machine learning models. One such bias is that of feedback loops, where the collection of data from an unfair online system hinders the accurate evaluation of the relevance scores between users and items. Given that recommendation systems often recommend popular content and vendors, the underlying relevance scores between users and items may not be accurately represented in the training data. Hence, this creates a feedback loop in which the user is not longer recommended based on their true relevance score but instead based on biased training data. To address this problem of feedback loops, we propose a two-stage representation learning framework, B-FAIR, aimed at rectifying the unfairness caused by biased historical data in recommendation systems. The framework disentangles the context data into sensitive and non-sensitive components using a variational autoencoder and then applies a novel Balanced Fairness Objective (BFO) to remove bias in the observational data when training a recommendation model. The efficacy of B-FAIR is demonstrated through experiments on both synthetic and real-world benchmarks, showing improved performance over state-of-the-art algorithms
A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification
One of the pursued objectives of deep learning is to provide tools that learn
abstract representations of reality from the observation of multiple contextual
situations. More precisely, one wishes to extract disentangled representations
which are (i) low dimensional and (ii) whose components are independent and
correspond to concepts capturing the essence of the objects under consideration
(Locatello et al., 2019b). One step towards this ambitious project consists in
learning disentangled representations with respect to a predefined (sensitive)
attribute, e.g., the gender or age of the writer. Perhaps one of the main
application for such disentangled representations is fair classification.
Existing methods extract the last layer of a neural network trained with a loss
that is composed of a cross-entropy objective and a disentanglement
regularizer. In this work, we adopt an information-theoretic view of this
problem which motivates a novel family of regularizers that minimizes the
mutual information between the latent representation and the sensitive
attribute conditional to the target. The resulting set of losses, called
CLINIC, is parameter free and thus, it is easier and faster to train. CLINIC
losses are studied through extensive numerical experiments by training over 2k
neural networks. We demonstrate that our methods offer a better
disentanglement/accuracy trade-off than previous techniques, and generalize
better than training with cross-entropy loss solely provided that the
disentanglement task is not too constraining.Comment: Findings AACL 202
On Fairness of Medical Image Classification with Multiple Sensitive Attributes via Learning Orthogonal Representations
Mitigating the discrimination of machine learning models has gained
increasing attention in medical image analysis. However, rare works focus on
fair treatments for patients with multiple sensitive demographic ones, which is
a crucial yet challenging problem for real-world clinical applications. In this
paper, we propose a novel method for fair representation learning with respect
to multi-sensitive attributes. We pursue the independence between target and
multi-sensitive representations by achieving orthogonality in the
representation space. Concretely, we enforce the column space orthogonality by
keeping target information on the complement of a low-rank sensitive space.
Furthermore, in the row space, we encourage feature dimensions between target
and sensitive representations to be orthogonal. The effectiveness of the
proposed method is demonstrated with extensive experiments on the CheXpert
dataset. To our best knowledge, this is the first work to mitigate unfairness
with respect to multiple sensitive attributes in the field of medical imaging
- …