Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projection

The state-of-the-art named entity recognition (NER) systems are supervised machine learning models that require large amounts of manually annotated data to achieve high accuracy. However, annotating NER data by human is expensive and time-consuming, and can be quite difficult for a new language. In this paper, we present two weakly supervised approaches for cross-lingual NER with no human annotation in a target language. The first approach is to create automatically labeled NER data for a target language via annotation projection on comparable corpora, where we develop a heuristic scheme that effectively selects good-quality projection-labeled data from noisy data. The second approach is to project distributed representations of words (word embeddings) from a target language to a source language, so that the source-language NER system can be applied to the target language without re-training. We also design two co-decoding schemes that effectively combine the outputs of the two projection-based approaches. We evaluate the performance of the proposed approaches on both in-house and open NER data for several target languages. The results show that the combined systems outperform three other weakly supervised approaches on the CoNLL data.Comment: 11 pages, The 55th Annual Meeting of the Association for Computational Linguistics (ACL), 201

Learning Character-level Compositionality with Visual Features

Previous work has modeled the compositionality of words by creating character-level models of meaning, reducing problems of sparsity for rare words. However, in many writing systems compositionality has an effect even on the character-level: the meaning of a character is derived by the sum of its parts. In this paper, we model this effect by creating embeddings for characters based on their visual characteristics, creating an image for the character and running it through a convolutional neural network to produce a visual character embedding. Experiments on a text classification task demonstrate that such model allows for better processing of instances with rare characters in languages such as Chinese, Japanese, and Korean. Additionally, qualitative analyses demonstrate that our proposed model learns to focus on the parts of characters that carry semantic content, resulting in embeddings that are coherent in visual space.Comment: Accepted to ACL 201

A Syllable-based Technique for Word Embeddings of Korean Words

Word embedding has become a fundamental component to many NLP tasks such as named entity recognition and machine translation. However, popular models that learn such embeddings are unaware of the morphology of words, so it is not directly applicable to highly agglutinative languages such as Korean. We propose a syllable-based learning model for Korean using a convolutional neural network, in which word representation is composed of trained syllable vectors. Our model successfully produces morphologically meaningful representation of Korean words compared to the original Skip-gram embeddings. The results also show that it is quite robust to the Out-of-Vocabulary problem.Comment: 5 pages, 3 figures, 1 table. Accepted for EMNLP 2017 Workshop - The 1st Workshop on Subword and Character level models in NLP (SCLeM

A Sub-Character Architecture for Korean Language Processing

We introduce a novel sub-character architecture that exploits a unique compositional structure of the Korean language. Our method decomposes each character into a small set of primitive phonetic units called jamo letters from which character- and word-level representations are induced. The jamo letters divulge syntactic and semantic information that is difficult to access with conventional character-level units. They greatly alleviate the data sparsity problem, reducing the observation space to 1.6% of the original while increasing accuracy in our experiments. We apply our architecture to dependency parsing and achieve dramatic improvement over strong lexical baselines.Comment: EMNLP 201

개체명 인식을 위한 조정하는 표시법을 고려하는 뉴럴 모델

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2019. 2. 김태환.개체명 인식 (NER) 은 자연언어처리 임무들 중 중요한 임무입니다. 이 문제에 대해 기존 기술은 양방향 순환신경망 (BiRNN) 과 조건부 무작 위장 (CRF) 를 활용하는 방법입니다. 본 논문은 기계번역 분야에서 나온 attention이란 컨셉트에게서 영감을 받으며 모델을 이루었습니다. 이 모델은 트레이닝 할 때 동적으로 한 단어의 character-level 표시법과 단어 임베딩의 웨이트들을 결정하므로 모델의 효과를 증가시킵니다. 본 논문은 다언어 데이터셋 (영어, 스페인어, 네덜란드어) 에서 실험을 진행하고 F1 점수의 비교를 통해서 다른 최신 연구보다 정확도가 높아졌습니다. 또한, 논문은 다양한 모델 배치 방안을 분석해서 hidden layer수와 단어 임베딩이 이 모델에게 주는 영향, 모델의 실행 시간과 효율도 토론했습니다.Sequence tagging is an important task in Natural Language Processing (NLP), in which the Named Entity Recognition (NER) is the key issue. So far the most widely adopted model for NER in NLP is that of combining the neural network of bidirectional long short-term memory (BiLSTM) and the statistical sequence prediction method of Conditional Random Field (CRF). In this work, we improve the prediction accuracy of the BiLSTM model by supporting an aligned character and word-level representation mechanism. We have performed experiments on multilingual (English, Spanish and Dutch) datasets and confirmed that our proposed model out-performed the existing state-of-the-art models.1 Introduction 1.1 Study Background 1.2 Purpose of Research 2 The Proposed Model 2.1 Character-level BiLSTM 2.2 Attention Mechanism 2. 2.1 The concept of attention 2.2.2 Word embedding 2.2.3 Our application 2.3 Word-level BiLSTM-CRF 2.3.1 LSTM with Conditional Random Field 2.3.2 Highway layer 3 Experiment 3.1 datasets 3.2 Training 3.3 Performance 3.3.1 Evaluation criterion 3.3.2 NER results 3.3.3 Other results 4 ConclusionMaste