24 research outputs found
One-Shot Relational Learning for Knowledge Graphs
Knowledge graphs (KGs) are the key components of various natural language
processing applications. To further expand KGs' coverage, previous studies on
knowledge graph completion usually require a large number of training instances
for each relation. However, we observe that long-tail relations are actually
more common in KGs and those newly added relations often do not have many known
triples for training. In this work, we aim at predicting new facts under a
challenging setting where only one training instance is available. We propose a
one-shot relational learning framework, which utilizes the knowledge extracted
by embedding models and learns a matching metric by considering both the
learned embeddings and one-hop graph structures. Empirically, our model yields
considerable performance improvements over existing embedding models, and also
eliminates the need of re-training the embedding models when dealing with newly
added relations.Comment: EMNLP 201
Zero-Shot Cross-Lingual Transfer with Meta Learning
Learning what to share between tasks has been a topic of great importance
recently, as strategic sharing of knowledge has been shown to improve
downstream task performance. This is particularly important for multilingual
applications, as most languages in the world are under-resourced. Here, we
consider the setting of training models on multiple different languages at the
same time, when little or no data is available for languages other than
English. We show that this challenging setup can be approached using
meta-learning, where, in addition to training a source language model, another
model learns to select which training instances are the most beneficial to the
first. We experiment using standard supervised, zero-shot cross-lingual, as
well as few-shot cross-lingual settings for different natural language
understanding tasks (natural language inference, question answering). Our
extensive experimental setup demonstrates the consistent effectiveness of
meta-learning for a total of 15 languages. We improve upon the state-of-the-art
for zero-shot and few-shot NLI (on MultiNLI and XNLI) and QA (on the MLQA
dataset). A comprehensive error analysis indicates that the correlation of
typological features between languages can partly explain when parameter
sharing learned via meta-learning is beneficial.Comment: Accepted as long paper in EMNLP2020 main conferenc
Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks
Clinical Natural Language Processing (NLP) has become an emerging technology
in healthcare that leverages a large amount of free-text data in electronic
health records (EHRs) to improve patient care, support clinical decisions, and
facilitate clinical and translational science research. Recently, deep learning
has achieved state-of-the-art performance in many clinical NLP tasks. However,
training deep learning models usually requires large annotated datasets, which
are normally not publicly available and can be time-consuming to build in
clinical domains. Working with smaller annotated datasets is typical in
clinical NLP and therefore, ensuring that deep learning models perform well is
crucial for the models to be used in real-world applications. A widely adopted
approach is fine-tuning existing Pre-trained Language Models (PLMs), but these
attempts fall short when the training dataset contains only a few annotated
samples. Few-Shot Learning (FSL) has recently been investigated to tackle this
problem. Siamese Neural Network (SNN) has been widely utilized as an FSL
approach in computer vision, but has not been studied well in NLP. Furthermore,
the literature on its applications in clinical domains is scarce. In this
paper, we propose two SNN-based FSL approaches for clinical NLP, including
Pre-Trained SNN (PT-SNN) and SNN with Second-Order Embeddings (SOE-SNN). We
evaluated the proposed approaches on two clinical tasks, namely clinical text
classification and clinical named entity recognition. We tested three few-shot
settings including 4-shot, 8-shot, and 16-shot learning. Both clinical NLP
tasks were benchmarked using three PLMs, including BERT,BioBERT, and
BioClinicalBERT. The experimental results verified the effectiveness of the
proposed SNN-based FSL approaches in both NLP tasks
Exploiting the Matching Information in the Support Set for Few Shot Event Classification
The existing event classification (EC) work primarily focuseson the
traditional supervised learning setting in which models are unableto extract
event mentions of new/unseen event types. Few-shot learninghas not been
investigated in this area although it enables EC models toextend their
operation to unobserved event types. To fill in this gap, inthis work, we
investigate event classification under the few-shot learningsetting. We propose
a novel training method for this problem that exten-sively exploit the support
set during the training process of a few-shotlearning model. In particular, in
addition to matching the query exam-ple with those in the support set for
training, we seek to further matchthe examples within the support set
themselves. This method providesmore training signals for the models and can be
applied to every metric-learning-based few-shot learning methods. Our extensive
experiments ontwo benchmark EC datasets show that the proposed method can
improvethe best reported few-shot learning models by up to 10% on accuracyfor
event classificationComment: Pacific-Asia Conference on Knowledge Discovery and Data Mining
(PAKDD) 202