70 research outputs found
What a Nerd! Beating Students and Vector Cosine in the ESL and TOEFL Datasets
In this paper, we claim that Vector Cosine, which is generally considered one
of the most efficient unsupervised measures for identifying word similarity in
Vector Space Models, can be outperformed by a completely unsupervised measure
that evaluates the extent of the intersection among the most associated
contexts of two target words, weighting such intersection according to the rank
of the shared contexts in the dependency ranked lists. This claim comes from
the hypothesis that similar words do not simply occur in similar contexts, but
they share a larger portion of their most relevant contexts compared to other
related words. To prove it, we describe and evaluate APSyn, a variant of
Average Precision that, independently of the adopted parameters, outperforms
the Vector Cosine and the co-occurrence on the ESL and TOEFL test sets. In the
best setting, APSyn reaches 0.73 accuracy on the ESL dataset and 0.70 accuracy
in the TOEFL dataset, beating therefore the non-English US college applicants
(whose average, as reported in the literature, is 64.50%) and several
state-of-the-art approaches.Comment: in LREC 201
Nine Features in a Random Forest to Learn Taxonomical Semantic Relations
ROOT9 is a supervised system for the classification of hypernyms, co-hyponyms
and random words that is derived from the already introduced ROOT13 (Santus et
al., 2016). It relies on a Random Forest algorithm and nine unsupervised
corpus-based features. We evaluate it with a 10-fold cross validation on 9,600
pairs, equally distributed among the three classes and involving several
Parts-Of-Speech (i.e. adjectives, nouns and verbs). When all the classes are
present, ROOT9 achieves an F1 score of 90.7%, against a baseline of 57.2%
(vector cosine). When the classification is binary, ROOT9 achieves the
following results against the baseline: hypernyms-co-hyponyms 95.7% vs. 69.8%,
hypernyms-random 91.8% vs. 64.1% and co-hyponyms-random 97.8% vs. 79.4%. In
order to compare the performance with the state-of-the-art, we have also
evaluated ROOT9 in subsets of the Weeds et al. (2014) datasets, proving that it
is in fact competitive. Finally, we investigated whether the system learns the
semantic relation or it simply learns the prototypical hypernyms, as claimed by
Levy et al. (2015). The second possibility seems to be the most likely, even
though ROOT9 can be trained on negative examples (i.e., switched hypernyms) to
drastically reduce this bias.Comment: in LREC 201
Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection
Modeling hypernymy, such as poodle is-a dog, is an important generalization
aid to many NLP tasks, such as entailment, coreference, relation extraction,
and question answering. Supervised learning from labeled hypernym sources, such
as WordNet, limits the coverage of these models, which can be addressed by
learning hypernyms from unlabeled text. Existing unsupervised methods either do
not scale to large vocabularies or yield unacceptably poor accuracy. This paper
introduces distributional inclusion vector embedding (DIVE), a
simple-to-implement unsupervised method of hypernym discovery via per-word
non-negative vector embeddings which preserve the inclusion property of word
contexts in a low-dimensional and interpretable space. In experimental
evaluations more comprehensive than any previous literature of which we are
aware-evaluating on 11 datasets using multiple existing as well as newly
proposed scoring functions-we find that our method provides up to double the
precision of previous unsupervised embeddings, and the highest average
performance, using a much more compact word representation, and yielding many
new state-of-the-art results.Comment: NAACL 201
ROOT13: Spotting Hypernyms, Co-Hyponyms and Randoms
In this paper, we describe ROOT13, a supervised system for the classification
of hypernyms, co-hyponyms and random words. The system relies on a Random
Forest algorithm and 13 unsupervised corpus-based features. We evaluate it with
a 10-fold cross validation on 9,600 pairs, equally distributed among the three
classes and involving several Parts-Of-Speech (i.e. adjectives, nouns and
verbs). When all the classes are present, ROOT13 achieves an F1 score of 88.3%,
against a baseline of 57.6% (vector cosine). When the classification is binary,
ROOT13 achieves the following results: hypernyms-co-hyponyms (93.4% vs. 60.2%),
hypernymsrandom (92.3% vs. 65.5%) and co-hyponyms-random (97.3% vs. 81.5%). Our
results are competitive with stateof-the-art models.Comment: in AAAI 201
- …