9,494 research outputs found
Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognition
Incorporating large-scale pre-trained models with the prototypical neural
networks is a de-facto paradigm in few-shot named entity recognition. Existing
methods, unfortunately, are not aware of the fact that embeddings from
pre-trained models contain a prominently large amount of information regarding
word frequencies, biasing prototypical neural networks against learning word
entities. This discrepancy constrains the two models' synergy. Thus, we propose
a one-line-code normalization method to reconcile such a mismatch with
empirical and theoretical grounds. Our experiments based on nine benchmark
datasets show the superiority of our method over the counterpart models and are
comparable to the state-of-the-art methods. In addition to the model
enhancement, our work also provides an analytical viewpoint for addressing the
general problems in few-shot name entity recognition or other tasks that rely
on pre-trained models or prototypical neural networks.Comment: Findings of EMNLP 202
- …