28 research outputs found
Uncovering divergent linguistic information in word embeddings with lessons for intrinsic and extrinsic evaluation
Following the recent success of word embeddings, it has been argued that
there is no such thing as an ideal representation for words, as different
models tend to capture divergent and often mutually incompatible aspects like
semantics/syntax and similarity/relatedness. In this paper, we show that each
embedding model captures more information than directly apparent. A linear
transformation that adjusts the similarity order of the model without any
external resource can tailor it to achieve better results in those aspects,
providing a new perspective on how embeddings encode divergent linguistic
information. In addition, we explore the relation between intrinsic and
extrinsic evaluation, as the effect of our transformations in downstream tasks
is higher for unsupervised systems than for supervised ones.Comment: CoNLL 201