466 research outputs found
Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network
Distinguishing between antonyms and synonyms is a key task to achieve high
performance in NLP systems. While they are notoriously difficult to distinguish
by distributional co-occurrence models, pattern-based methods have proven
effective to differentiate between the relations. In this paper, we present a
novel neural network model AntSynNET that exploits lexico-syntactic patterns
from syntactic parse trees. In addition to the lexical and syntactic
information, we successfully integrate the distance between the related words
along the syntactic path as a new pattern feature. The results from
classification experiments show that AntSynNET improves the performance over
prior pattern-based methods.Comment: EACL 2017, 10 page
Similarity Models in Distributional Semantics using Task Specific Information
In distributional semantics, the unsupervised learning approach has been widely used for a large number of tasks. On the other hand, supervised learning has less coverage.
In this dissertation, we investigate the supervised learning approach for semantic relatedness tasks in distributional semantics. The investigation considers mainly semantic similarity and semantic classification tasks. Existing and newly-constructed datasets are used as an input for the experiments. The new datasets are constructed from thesauruses like Eurovoc. The Eurovoc thesaurus is a multilingual thesaurus maintained by the Publications Office of the European Union. The meaning of the words in the dataset is represented by using a distributional semantic approach.
The distributional semantic approach collects co-occurrence information from large texts and represents the words in high-dimensional vectors. The English words are represented by using UkWaK corpus while German words are represented by using DeWaC corpus. After representing each word by the high dimensional vector, different supervised machine learning methods are used on the selected tasks. The outputs from the supervised machine learning methods are evaluated by comparing the tasks performance and accuracy with the state of the art unsupervised machine learning methods’ results. In addition, multi-relational matrix factorization is introduced as one supervised learning method in distributional semantics. This dissertation shows the multi-relational matrix factorization method as a good alternative method to integrate different sources of information of words in distributional semantics.
In the dissertation, some new applications are also introduced. One of the applications is an application which analyzes a German company’s website text, and provides information about the company with a concept cloud visualization. The other applications are automatic recognition/disambiguation of the library of congress subject headings and automatic identification of synonym relations in the Dutch Parliament thesaurus applications
Introducing various Semantic Models for Amharic: Experimentation and Evaluation with multiple Tasks and Datasets
The availability of different pre-trained semantic models enabled the quick
development of machine learning components for downstream applications. Despite
the availability of abundant text data for low resource languages, only a few
semantic models are publicly available. Publicly available pre-trained models
are usually built as a multilingual version of semantic models that can not fit
well for each language due to context variations. In this work, we introduce
different semantic models for Amharic. After we experiment with the existing
pre-trained semantic models, we trained and fine-tuned nine new different
models using a monolingual text corpus. The models are build using word2Vec
embeddings, distributional thesaurus (DT), contextual embeddings, and DT
embeddings obtained via network embedding algorithms. Moreover, we employ these
models for different NLP tasks and investigate their impact. We find that newly
trained models perform better than pre-trained multilingual models.
Furthermore, models based on contextual embeddings from RoBERTA perform better
than the word2Vec models
Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization
Semantic specialization is the process of fine-tuning pre-trained
distributional word vectors using external lexical knowledge (e.g., WordNet) to
accentuate a particular semantic relation in the specialized vector space.
While post-processing specialization methods are applicable to arbitrary
distributional vectors, they are limited to updating only the vectors of words
occurring in external lexicons (i.e., seen words), leaving the vectors of all
other words unchanged. We propose a novel approach to specializing the full
distributional vocabulary. Our adversarial post-specialization method
propagates the external lexical knowledge to the full distributional space. We
exploit words seen in the resources as training examples for learning a global
specialization function. This function is learned by combining a standard
L2-distance loss with an adversarial loss: the adversarial component produces
more realistic output vectors. We show the effectiveness and robustness of the
proposed method across three languages and on three tasks: word similarity,
dialog state tracking, and lexical simplification. We report consistent
improvements over distributional word vectors and vectors specialized by other
state-of-the-art specialization frameworks. Finally, we also propose a
cross-lingual transfer method for zero-shot specialization which successfully
specializes a full target distributional space without any lexical knowledge in
the target language and without any bilingual data.Comment: Accepted at EMNLP 201
- …