96,550 research outputs found
Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks
Text classification tasks often encounter few shot scenarios with limited
labeled data, and addressing data scarcity is crucial. Data augmentation with
mixup has shown to be effective on various text classification tasks. However,
most of the mixup methods do not consider the varying degree of learning
difficulty in different stages of training and generate new samples with one
hot labels, resulting in the model over confidence. In this paper, we propose a
self evolution learning (SE) based mixup approach for data augmentation in text
classification, which can generate more adaptive and model friendly pesudo
samples for the model training. SE focuses on the variation of the model's
learning ability. To alleviate the model confidence, we introduce a novel
instance specific label smoothing approach, which linearly interpolates the
model's output and one hot labels of the original samples to generate new soft
for label mixing up. Through experimental analysis, in addition to improving
classification accuracy, we demonstrate that SE also enhances the model's
generalize ability
Semi-supervised Learning with the EM Algorithm: A Comparative Study between Unstructured and Structured Prediction
Semi-supervised learning aims to learn prediction models from both labeled
and unlabeled samples. There has been extensive research in this area. Among
existing work, generative mixture models with Expectation-Maximization (EM) is
a popular method due to clear statistical properties. However, existing
literature on EM-based semi-supervised learning largely focuses on unstructured
prediction, assuming that samples are independent and identically distributed.
Studies on EM-based semi-supervised approach in structured prediction is
limited. This paper aims to fill the gap through a comparative study between
unstructured and structured methods in EM-based semi-supervised learning.
Specifically, we compare their theoretical properties and find that both
methods can be considered as a generalization of self-training with soft class
assignment of unlabeled samples, but the structured method additionally
considers structural constraint in soft class assignment. We conducted a case
study on real-world flood mapping datasets to compare the two methods. Results
show that structured EM is more robust to class confusion caused by noise and
obstacles in features in the context of the flood mapping application
Improved Training for Self-Training by Confidence Assessments
It is well known that for some tasks, labeled data sets may be hard to
gather. Therefore, we wished to tackle here the problem of having insufficient
training data. We examined learning methods from unlabeled data after an
initial training on a limited labeled data set. The suggested approach can be
used as an online learning method on the unlabeled test set. In the general
classification task, whenever we predict a label with high enough confidence,
we treat it as a true label and train the data accordingly. For the semantic
segmentation task, a classic example for an expensive data labeling process, we
do so pixel-wise. Our suggested approaches were applied on the MNIST data-set
as a proof of concept for a vision classification task and on the ADE20K
data-set in order to tackle the semi-supervised semantic segmentation problem
Fidelity-Weighted Learning
Training deep neural networks requires many training samples, but in practice
training labels are expensive to obtain and may be of varying quality, as some
may be from trusted expert labelers while others might be from heuristics or
other sources of weak supervision such as crowd-sourcing. This creates a
fundamental quality versus-quantity trade-off in the learning process. Do we
learn from the small amount of high-quality data or the potentially large
amount of weakly-labeled data? We argue that if the learner could somehow know
and take the label-quality into account when learning the data representation,
we could get the best of both worlds. To this end, we propose
"fidelity-weighted learning" (FWL), a semi-supervised student-teacher approach
for training deep neural networks using weakly-labeled data. FWL modulates the
parameter updates to a student network (trained on the task we care about) on a
per-sample basis according to the posterior confidence of its label-quality
estimated by a teacher (who has access to the high-quality labels). Both
student and teacher are learned from the data. We evaluate FWL on two tasks in
information retrieval and natural language processing where we outperform
state-of-the-art alternative semi-supervised methods, indicating that our
approach makes better use of strong and weak labels, and leads to better
task-dependent data representations.Comment: Published as a conference paper at ICLR 201
- …