35 research outputs found
Adversarial Removal of Demographic Attributes from Text Data
Recent advances in Representation Learning and Adversarial Training seem to
succeed in removing unwanted features from the learned representation. We show
that demographic information of authors is encoded in -- and can be recovered
from -- the intermediate representations learned by text-based neural
classifiers. The implication is that decisions of classifiers trained on
textual data are not agnostic to -- and likely condition on -- demographic
attributes. When attempting to remove such demographic information using
adversarial training, we find that while the adversarial component achieves
chance-level development-set accuracy during training, a post-hoc classifier,
trained on the encoded sentences from the first part, still manages to reach
substantially higher classification accuracies on the same data. This behavior
is consistent across several tasks, demographic properties and datasets. We
explore several techniques to improve the effectiveness of the adversarial
component. Our main conclusion is a cautionary one: do not rely on the
adversarial training to achieve invariant representation to sensitive features
Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations
Latent factor models for recommender systems represent users and items as low
dimensional vectors. Privacy risks of such systems have previously been studied
mostly in the context of recovery of personal information in the form of usage
records from the training data. However, the user representations themselves
may be used together with external data to recover private user information
such as gender and age. In this paper we show that user vectors calculated by a
common recommender system can be exploited in this way. We propose the
privacy-adversarial framework to eliminate such leakage of private information,
and study the trade-off between recommender performance and leakage both
theoretically and empirically using a benchmark dataset. An advantage of the
proposed method is that it also helps guarantee fairness of results, since all
implicit knowledge of a set of attributes is scrubbed from the representations
used by the model, and thus can't enter into the decision making. We discuss
further applications of this method towards the generation of deeper and more
insightful recommendations.Comment: International Conference on Pattern Recognition and Method
The Bias Amplification Paradox in Text-to-Image Generation
Bias amplification is a phenomenon in which models increase imbalances
present in the training data. In this paper, we study bias amplification in the
text-to-image domain using Stable Diffusion by comparing gender ratios in
training vs. generated images. We find that the model appears to amplify
gender-occupation biases found in the training data (LAION). However, we
discover that amplification can largely be attributed to discrepancies between
training captions and model prompts. For example, an inherent difference is
that captions from the training data often contain explicit gender information
while the prompts we use do not, which leads to a distribution shift and
consequently impacts bias measures. Once we account for various distributional
differences between texts used for training and generation, we observe that
amplification decreases considerably. Our findings illustrate the challenges of
comparing biases in models and the data they are trained on, and highlight
confounding factors that contribute to bias amplification
Unsupervised Distillation of Syntactic Information from Contextualized Word Representations
Contextualized word representations, such as ELMo and BERT, were shown to
perform well on various semantic and syntactic tasks. In this work, we tackle
the task of unsupervised disentanglement between semantics and structure in
neural language representations: we aim to learn a transformation of the
contextualized vectors, that discards the lexical semantics, but keeps the
structural information. To this end, we automatically generate groups of
sentences which are structurally similar but semantically different, and use
metric-learning approach to learn a transformation that emphasizes the
structural component that is encoded in the vectors. We demonstrate that our
transformation clusters vectors in space by structural properties, rather than
by lexical semantics. Finally, we demonstrate the utility of our distilled
representations by showing that they outperform the original contextualized
representations in a few-shot parsing setting.Comment: Accepted in BlackboxNLP@EMNLP202
Do Language Embeddings Capture Scales?
Pretrained Language Models (LMs) have been shown to possess significant
linguistic, common sense, and factual knowledge. One form of knowledge that has
not been studied yet in this context is information about the scalar magnitudes
of objects. We show that pretrained language models capture a significant
amount of this information but are short of the capability required for general
common-sense reasoning. We identify contextual information in pre-training and
numeracy as two key factors affecting their performance and show that a simple
method of canonicalizing numbers can have a significant effect on the results.Comment: Accepted at EMNLP Findings 2020 and EMNLP BlackboxNLP workshop 2020;
8 pages, 2 figures; Minor changes to the acknowledgment sectio
Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation
Few-shot fine-tuning and in-context learning are two alternative strategies
for task adaptation of pre-trained language models. Recently, in-context
learning has gained popularity over fine-tuning due to its simplicity and
improved out-of-domain generalization, and because extensive evidence shows
that fine-tuned models pick up on spurious correlations. Unfortunately,
previous comparisons of the two approaches were done using models of different
sizes. This raises the question of whether the observed weaker out-of-domain
generalization of fine-tuned models is an inherent property of fine-tuning or a
limitation of the experimental setup. In this paper, we compare the
generalization of few-shot fine-tuning and in-context learning to challenge
datasets, while controlling for the models used, the number of examples, and
the number of parameters, ranging from 125M to 30B. Our results show that
fine-tuned language models can in fact generalize well out-of-domain. We find
that both approaches generalize similarly; they exhibit large variation and
depend on properties such as model size and the number of examples,
highlighting that robust task adaptation remains a challenge.Comment: Accepted to Findings of ACL 202