49 research outputs found
Polysemy Detection in Distributed Representation of Word Sense
In this paper, we propose a statistical test to determine whether a given
word is used as a polysemic word or not. The statistic of the word in this test
roughly corresponds to the fluctuation in the senses of the neighboring words a
nd the word itself. Even though the sense of a word corresponds to a single
vector, we discuss how polysemy of the words affects the position of vectors.
Finally, we also explain the method to detect this effect.Comment: The 9th International Conference on Knowledge and Smart Technology
(KST-2017
A gloss composition and context clustering based distributed word sense representation model
In recent years, there has been an increasing interest in learning a distributed representation of word sense. Traditional context clustering based models usually require careful tuning of model parameters, and typically perform worse on infrequent word senses. This paper presents a novel approach which addresses these limitations by first initializing the word sense embeddings through learning sentence-level embeddings from WordNet glosses using a convolutional neural networks. The initialized word sense embeddings are used by a context clustering based model to generate the distributed representations of word senses. Our learned representations outperform the publicly available embeddings on half of the metrics in the word similarity task, 6 out of 13 sub tasks in the analogical reasoning task, and gives the best overall accuracy in the word sense effect classification task, which shows the effectiveness of our proposed distributed distribution learning model
SECaps: A Sequence Enhanced Capsule Model for Charge Prediction
Automatic charge prediction aims to predict appropriate final charges
according to the fact descriptions for a given criminal case. Automatic charge
prediction plays a critical role in assisting judges and lawyers to improve the
efficiency of legal decisions, and thus has received much attention.
Nevertheless, most existing works on automatic charge prediction perform
adequately on high-frequency charges but are not yet capable of predicting
few-shot charges with limited cases. In this paper, we propose a Sequence
Enhanced Capsule model, dubbed as SECaps model, to relieve this problem.
Specifically, following the work of capsule networks, we propose the seq-caps
layer, which considers sequence information and spatial information of legal
texts simultaneously. Then we design a attention residual unit, which provides
auxiliary information for charge prediction. In addition, our SECaps model
introduces focal loss, which relieves the problem of imbalanced charges.
Comparing the state-of-the-art methods, our SECaps model obtains 4.5% and 6.4%
absolutely considerable improvements under Macro F1 in Criminal-S and
Criminal-L respectively. The experimental results consistently demonstrate the
superiorities and competitiveness of our proposed model.Comment: 13 pages, 3figures, 5 table
A Mixture Model for Learning Multi-Sense Word Embeddings
Word embeddings are now a standard technique for inducing meaning
representations for words. For getting good representations, it is important to
take into account different senses of a word. In this paper, we propose a
mixture model for learning multi-sense word embeddings. Our model generalizes
the previous works in that it allows to induce different weights of different
senses of a word. The experimental results show that our model outperforms
previous models on standard evaluation tasks.Comment: *SEM 201
MUSE: Modularizing Unsupervised Sense Embeddings
This paper proposes to address the word sense ambiguity issue in an
unsupervised manner, where word sense representations are learned along a word
sense selection mechanism given contexts. Prior work focused on designing a
single model to deliver both mechanisms, and thus suffered from either
coarse-grained representation learning or inefficient sense selection. The
proposed modular approach, MUSE, implements flexible modules to optimize
distinct mechanisms, achieving the first purely sense-level representation
learning system with linear-time sense selection. We leverage reinforcement
learning to enable joint training on the proposed modules, and introduce
various exploration techniques on sense selection for better robustness. The
experiments on benchmark data show that the proposed approach achieves the
state-of-the-art performance on synonym selection as well as on contextual word
similarities in terms of MaxSimC