101 research outputs found
Word Sense Disambiguation using a Bidirectional LSTM
In this paper we present a clean, yet effective, model for word sense
disambiguation. Our approach leverage a bidirectional long short-term memory
network which is shared between all words. This enables the model to share
statistical strength and to scale well with vocabulary size. The model is
trained end-to-end, directly from the raw text to sense labels, and makes
effective use of word order. We evaluate our approach on two standard datasets,
using identical hyperparameter settings, which are in turn tuned on a third set
of held out data. We employ no external resources (e.g. knowledge graphs,
part-of-speech tagging, etc), language specific features, or hand crafted
rules, but still achieve statistically equivalent results to the best
state-of-the-art systems, that employ no such limitations
Unsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation
Interpretability of a predictive model is a powerful feature that gains the
trust of users in the correctness of the predictions. In word sense
disambiguation (WSD), knowledge-based systems tend to be much more
interpretable than knowledge-free counterparts as they rely on the wealth of
manually-encoded elements representing word senses, such as hypernyms, usage
examples, and images. We present a WSD system that bridges the gap between
these two so far disconnected groups of methods. Namely, our system, providing
access to several state-of-the-art WSD models, aims to be interpretable as a
knowledge-based system while it remains completely unsupervised and
knowledge-free. The presented tool features a Web interface for all-word
disambiguation of texts that makes the sense predictions human readable by
providing interpretable word sense inventories, sense representations, and
disambiguation results. We provide a public API, enabling seamless integration.Comment: In Proceedings of the the Conference on Empirical Methods on Natural
Language Processing (EMNLP 2017). 2017. Copenhagen, Denmark. Association for
Computational Linguistic
BERTを利用した教師あり学習による語義曖昧性解消
Ibaraki UniversityIbaraki UniversityIbaraki UniversityIbaraki UniversityIbaraki University会議名: 言語資源活用ワークショップ2019, 開催地: 国立国語研究所, 会期: 2019年9月2日−4日, 主催: 国立国語研究所 コーパス開発センターBERTはTransformerで利用されるMulti-head attentionを12層(あるいは24層)積み重ねたモデルである。各層のMulti-head attentionは、基本的に、入力単語列に対応する単語埋め込み表現列を出力している。つまりBERTは入力文中の単語に対する埋め込み表現を出力しているが、その埋め込み表現がその単語の文脈に依存した形になっていることが大きな特徴である。この点からBERTから得られる多義語の埋め込み表現を、その多義語の語義曖昧性解消ための特徴ベクトルとして扱えると考えられる。実験では京都大学が公開している日本語版BERT事前学習モデルを利用して、上記の手法をSemEval-2の日本語辞書タスクに対する適用し、高い正解率を得た
- …