31,385 research outputs found
Labeled Memory Networks for Online Model Adaptation
Augmenting a neural network with memory that can grow without growing the
number of trained parameters is a recent powerful concept with many exciting
applications. We propose a design of memory augmented neural networks (MANNs)
called Labeled Memory Networks (LMNs) suited for tasks requiring online
adaptation in classification models. LMNs organize the memory with classes as
the primary key.The memory acts as a second boosted stage following a regular
neural network thereby allowing the memory and the primary network to play
complementary roles. Unlike existing MANNs that write to memory for every
instance and use LRU based memory replacement, LMNs write only for instances
with non-zero loss and use label-based memory replacement. We demonstrate
significant accuracy gains on various tasks including word-modelling and
few-shot learning. In this paper, we establish their potential in online
adapting a batch trained neural network to domain-relevant labeled data at
deployment time. We show that LMNs are better than other MANNs designed for
meta-learning. We also found them to be more accurate and faster than
state-of-the-art methods of retuning model parameters for adapting to
domain-specific labeled data.Comment: Accepted at AAAI 2018, 8 page
Zero-Shot Cross-Lingual Transfer with Meta Learning
Learning what to share between tasks has been a topic of great importance
recently, as strategic sharing of knowledge has been shown to improve
downstream task performance. This is particularly important for multilingual
applications, as most languages in the world are under-resourced. Here, we
consider the setting of training models on multiple different languages at the
same time, when little or no data is available for languages other than
English. We show that this challenging setup can be approached using
meta-learning, where, in addition to training a source language model, another
model learns to select which training instances are the most beneficial to the
first. We experiment using standard supervised, zero-shot cross-lingual, as
well as few-shot cross-lingual settings for different natural language
understanding tasks (natural language inference, question answering). Our
extensive experimental setup demonstrates the consistent effectiveness of
meta-learning for a total of 15 languages. We improve upon the state-of-the-art
for zero-shot and few-shot NLI (on MultiNLI and XNLI) and QA (on the MLQA
dataset). A comprehensive error analysis indicates that the correlation of
typological features between languages can partly explain when parameter
sharing learned via meta-learning is beneficial.Comment: Accepted as long paper in EMNLP2020 main conferenc
Recommended from our members
Learning models for semantic classification of insufficient plantar pressure images
Establishing a reliable and stable model to predict a target by using insufficient labeled samples is feasible and
effective, particularly, for a sensor-generated data-set. This paper has been inspired with insufficient data-set
learning algorithms, such as metric-based, prototype networks and meta-learning, and therefore we propose
an insufficient data-set transfer model learning method. Firstly, two basic models for transfer learning are
introduced. A classification system and calculation criteria are then subsequently introduced. Secondly, a dataset
of plantar pressure for comfort shoe design is acquired and preprocessed through foot scan system; and by
using a pre-trained convolution neural network employing AlexNet and convolution neural network (CNN)-
based transfer modeling, the classification accuracy of the plantar pressure images is over 93.5%. Finally,
the proposed method has been compared to the current classifiers VGG, ResNet, AlexNet and pre-trained
CNN. Also, our work is compared with known-scaling and shifting (SS) and unknown-plain slot (PS) partition
methods on the public test databases: SUN, CUB, AWA1, AWA2, and aPY with indices of precision (tr, ts, H)
and time (training and evaluation). The proposed method for the plantar pressure classification task shows high
performance in most indices when comparing with other methods. The transfer learning-based method can be
applied to other insufficient data-sets of sensor imaging fields
- …