52 research outputs found
Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations
We propose a novel data augmentation for labeled sentences called contextual
augmentation. We assume an invariance that sentences are natural even if the
words in the sentences are replaced with other words with paradigmatic
relations. We stochastically replace words with other words that are predicted
by a bi-directional language model at the word positions. Words predicted
according to a context are numerous but appropriate for the augmentation of the
original words. Furthermore, we retrofit a language model with a
label-conditional architecture, which allows the model to augment sentences
without breaking the label-compatibility. Through the experiments for six
various different text classification tasks, we demonstrate that the proposed
method improves classifiers based on the convolutional or recurrent neural
networks.Comment: NAACL 201
Towards Semi-Supervised Learning for Deep Semantic Role Labeling
Neural models have shown several state-of-the-art performances on Semantic
Role Labeling (SRL). However, the neural models require an immense amount of
semantic-role corpora and are thus not well suited for low-resource languages
or domains. The paper proposes a semi-supervised semantic role labeling method
that outperforms the state-of-the-art in limited SRL training corpora. The
method is based on explicitly enforcing syntactic constraints by augmenting the
training objective with a syntactic-inconsistency loss component and uses
SRL-unlabeled instances to train a joint-objective LSTM. On CoNLL-2012 English
section, the proposed semi-supervised training with 1%, 10% SRL-labeled data
and varying amounts of SRL-unlabeled data achieves +1.58, +0.78 F1,
respectively, over the pre-trained models that were trained on SOTA
architecture with ELMo on the same SRL-labeled data. Additionally, by using the
syntactic-inconsistency loss on inference time, the proposed model achieves
+3.67, +2.1 F1 over pre-trained model on 1%, 10% SRL-labeled data,
respectively.Comment: EMNLP 201
The Development of a Temporal Information Dictionary for Social Media Analytics
Dictionaries have been used to analyze text even before the emergence of social media and the use of dictionaries for sentiment analysis there. While dictionaries have been used to understand the tonality of text, so far it has not been possible to automatically detect if the tonality refers to the present, past, or future. In this research, we develop a dictionary containing time-indicating words in a wordlist (T-wordlist). To test how the dictionary performs, we apply our T-wordlist on different disaster related social media datasets. Subsequently we will validate the wordlist and results by a manual content analysis. So far, in this research-in-progress, we were able to develop a first dictionary and will also provide some initial insight into the performance of our wordlist
Semi-supervised SRL system with Bayesian inference
International audienceWe propose a new approach to perform semi-supervised training of Semantic Role Labeling models with very few amount of initial labeled data. The proposed approach combines in a novel way supervised and unsupervised training, by forcing the supervised classifier to over-generate potential semantic candidates, and then letting unsupervised inference choose the best ones. Hence, the supervised classifier can be trained on a very small corpus and with coarse-grain features, because its precision does not need to be high: its role is mainly to constrain Bayesian inference to explore only a limited part of the full search space. This approach is evaluated on French and English. In both cases, it achieves very good performance and outperforms a strong supervised baseline when only a small number of annotated sentences is available and even without using any previously trained syntactic parser
Interpolated PLSI for Learning Plausible Verb Arguments
PACLIC 23 / City University of Hong Kong / 3-5 December 200
- …