6 research outputs found
Few-shot classification in Named Entity Recognition Task
For many natural language processing (NLP) tasks the amount of annotated data
is limited. This urges a need to apply semi-supervised learning techniques,
such as transfer learning or meta-learning. In this work we tackle Named Entity
Recognition (NER) task using Prototypical Network - a metric learning
technique. It learns intermediate representations of words which cluster well
into named entity classes. This property of the model allows classifying words
with extremely limited number of training examples, and can potentially be used
as a zero-shot learning method. By coupling this technique with transfer
learning we achieve well-performing classifiers trained on only 20 instances of
a target class.Comment: In proceedings of the 34th ACM/SIGAPP Symposium on Applied Computin
Template-free Prompt Tuning for Few-shot NER
Prompt-based methods have been successfully applied in sentence-level
few-shot learning tasks, mostly owing to the sophisticated design of templates
and label words. However, when applied to token-level labeling tasks such as
NER, it would be time-consuming to enumerate the template queries over all
potential entity spans. In this work, we propose a more elegant method to
reformulate NER tasks as LM problems without any templates. Specifically, we
discard the template construction process while maintaining the word prediction
paradigm of pre-training models to predict a class-related pivot word (or label
word) at the entity position. Meanwhile, we also explore principled ways to
automatically search for appropriate label words that the pre-trained models
can easily adapt to. While avoiding complicated template-based process, the
proposed LM objective also reduces the gap between different objectives used in
pre-training and fine-tuning, thus it can better benefit the few-shot
performance. Experimental results demonstrate the effectiveness of the proposed
method over bert-tagger and template-based method under few-shot setting.
Moreover, the decoding speed of the proposed method is up to 1930.12 times
faster than the template-based method.Comment: Work in Progres
A Survey on Semantic Processing Techniques
Semantic processing is a fundamental research domain in computational
linguistics. In the era of powerful pre-trained language models and large
language models, the advancement of research in this domain appears to be
decelerating. However, the study of semantics is multi-dimensional in
linguistics. The research depth and breadth of computational semantic
processing can be largely improved with new technologies. In this survey, we
analyzed five semantic processing tasks, e.g., word sense disambiguation,
anaphora resolution, named entity recognition, concept extraction, and
subjectivity detection. We study relevant theoretical research in these fields,
advanced methods, and downstream applications. We connect the surveyed tasks
with downstream applications because this may inspire future scholars to fuse
these low-level semantic processing tasks with high-level natural language
processing tasks. The review of theoretical research may also inspire new tasks
and technologies in the semantic processing domain. Finally, we compare the
different semantic processing techniques and summarize their technical trends,
application trends, and future directions.Comment: Published at Information Fusion, Volume 101, 2024, 101988, ISSN
1566-2535. The equal contribution mark is missed in the published version due
to the publication policies. Please contact Prof. Erik Cambria for detail