2 research outputs found
Correcting the Common Discourse Bias in Linear Representation of Sentences using Conceptors
Distributed representations of words, better known as word embeddings, have
become important building blocks for natural language processing tasks.
Numerous studies are devoted to transferring the success of unsupervised word
embeddings to sentence embeddings. In this paper, we introduce a simple
representation of sentences in which a sentence embedding is represented as a
weighted average of word vectors followed by a soft projection. We demonstrate
the effectiveness of this proposed method on the clinical semantic textual
similarity task of the BioCreative/OHNLP Challenge 2018.Comment: Accepted by the BioCreative/OHNLP workshop of ACM-BCB 201
Conceptor Debiasing of Word Representations Evaluated on WEAT
Bias in word embeddings such as Word2Vec has been widely investigated, and
many efforts made to remove such bias. We show how to use conceptors debiasing
to post-process both traditional and contextualized word embeddings. Our
conceptor debiasing can simultaneously remove racial and gender biases and,
unlike standard debiasing methods, can make effect use of heterogeneous lists
of biased words. We show that conceptor debiasing diminishes racial and gender
bias of word representations as measured using the Word Embedding Association
Test (WEAT) of Caliskan et al. (2017)