3 research outputs found
Meta-Learning via Feature-Label Memory Network
Deep learning typically requires training a very capable architecture using
large datasets. However, many important learning problems demand an ability to
draw valid inferences from small size datasets, and such problems pose a
particular challenge for deep learning. In this regard, various researches on
"meta-learning" are being actively conducted. Recent work has suggested a
Memory Augmented Neural Network (MANN) for meta-learning. MANN is an
implementation of a Neural Turing Machine (NTM) with the ability to rapidly
assimilate new data in its memory, and use this data to make accurate
predictions. In models such as MANN, the input data samples and their
appropriate labels from previous step are bound together in the same memory
locations. This often leads to memory interference when performing a task as
these models have to retrieve a feature of an input from a certain memory
location and read only the label information bound to that location. In this
paper, we tried to address this issue by presenting a more robust MANN. We
revisited the idea of meta-learning and proposed a new memory augmented neural
network by explicitly splitting the external memory into feature and label
memories. The feature memory is used to store the features of input data
samples and the label memory stores their labels. Hence, when predicting the
label of a given input, our model uses its feature memory unit as a reference
to extract the stored feature of the input, and based on that feature, it
retrieves the label information of the input from the label memory unit. In
order for the network to function in this framework, a new memory-writingmodule
to encode label information into the label memory in accordance with the
meta-learning task structure is designed. Here, we demonstrate that our model
outperforms MANN by a large margin in supervised one-shot classification tasks
using Omniglot and MNIST datasets.Comment: https://github.com/Dawitmu/Meta-Learning-via-Feature-Label-Memory-Networ
Learning to Compare Relation: Semantic Alignment for Few-Shot Learning
Few-shot learning is a fundamental and challenging problem since it requires
recognizing novel categories from only a few examples. The objects for
recognition have multiple variants and can locate anywhere in images. Directly
comparing query images with example images can not handle content misalignment.
The representation and metric for comparison are critical but challenging to
learn due to the scarcity and wide variation of the samples in few-shot
learning. In this paper, we present a novel semantic alignment model to compare
relations, which is robust to content misalignment. We propose to add two key
ingredients to existing few-shot learning frameworks for better feature and
metric learning ability. First, we introduce a semantic alignment loss to align
the relation statistics of the features from samples that belong to the same
category. And second, local and global mutual information maximization is
introduced, allowing for representations that contain locally-consistent and
intra-class shared information across structural locations in an image.
Thirdly, we introduce a principled approach to weigh multiple loss functions by
considering the homoscedastic uncertainty of each stream. We conduct extensive
experiments on several few-shot learning datasets. Experimental results show
that the proposed method is capable of comparing relations with semantic
alignment strategies, and achieves state-of-the-art performance
Small Sample Learning in Big Data Era
As a promising area in artificial intelligence, a new learning paradigm,
called Small Sample Learning (SSL), has been attracting prominent research
attention in the recent years. In this paper, we aim to present a survey to
comprehensively introduce the current techniques proposed on this topic.
Specifically, current SSL techniques can be mainly divided into two categories.
The first category of SSL approaches can be called "concept learning", which
emphasizes learning new concepts from only few related observations. The
purpose is mainly to simulate human learning behaviors like recognition,
generation, imagination, synthesis and analysis. The second category is called
"experience learning", which usually co-exists with the large sample learning
manner of conventional machine learning. This category mainly focuses on
learning with insufficient samples, and can also be called small data learning
in some literatures. More extensive surveys on both categories of SSL
techniques are introduced and some neuroscience evidences are provided to
clarify the rationality of the entire SSL regime, and the relationship with
human learning process. Some discussions on the main challenges and possible
future research directions along this line are also presented.Comment: 76 pages, 15 figures, survey of small sample learnin