20 research outputs found
Meta Soft Label Generation for Noisy Labels
The existence of noisy labels in the dataset causes significant performance
degradation for deep neural networks (DNNs). To address this problem, we
propose a Meta Soft Label Generation algorithm called MSLG, which can jointly
generate soft labels using meta-learning techniques and learn DNN parameters in
an end-to-end fashion. Our approach adapts the meta-learning paradigm to
estimate optimal label distribution by checking gradient directions on both
noisy training data and noise-free meta-data. In order to iteratively update
soft labels, meta-gradient descent step is performed on estimated labels, which
would minimize the loss of noise-free meta samples. In each iteration, the base
classifier is trained on estimated meta labels. MSLG is model-agnostic and can
be added on top of any existing model at hand with ease. We performed extensive
experiments on CIFAR10, Clothing1M and Food101N datasets. Results show that our
approach outperforms other state-of-the-art methods by a large margin.Comment: Accepted by ICPR 202
Decoupling Representation and Classifier for Noisy Label Learning
Since convolutional neural networks (ConvNets) can easily memorize noisy
labels, which are ubiquitous in visual classification tasks, it has been a
great challenge to train ConvNets against them robustly. Various solutions,
e.g., sample selection, label correction, and robustifying loss functions, have
been proposed for this challenge, and most of them stick to the end-to-end
training of the representation (feature extractor) and classifier. In this
paper, by a deep rethinking and careful re-examining on learning behaviors of
the representation and classifier, we discover that the representation is much
more fragile in the presence of noisy labels than the classifier. Thus, we are
motivated to design a new method, i.e., REED, to leverage above discoveries to
learn from noisy labels robustly. The proposed method contains three stages,
i.e., obtaining the representation by self-supervised learning without any
labels, transferring the noisy label learning problem into a semisupervised one
by the classifier directly and reliably trained with noisy labels, and joint
semi-supervised retraining of both the representation and classifier. Extensive
experiments are performed on both synthetic and real benchmark datasets.
Results demonstrate that the proposed method can beat the state-of-the-art ones
by a large margin, especially under high noise level
PI-GNN: A Novel Perspective on Semi-Supervised Node Classification against Noisy Labels
Semi-supervised node classification, as a fundamental problem in graph
learning, leverages unlabeled nodes along with a small portion of labeled nodes
for training. Existing methods rely heavily on high-quality labels, which,
however, are expensive to obtain in real-world applications since certain
noises are inevitably involved during the labeling process. It hence poses an
unavoidable challenge for the learning algorithm to generalize well. In this
paper, we propose a novel robust learning objective dubbed pairwise
interactions (PI) for the model, such as Graph Neural Network (GNN) to combat
noisy labels. Unlike classic robust training approaches that operate on the
pointwise interactions between node and class label pairs, PI explicitly forces
the embeddings for node pairs that hold a positive PI label to be close to each
other, which can be applied to both labeled and unlabeled nodes. We design
several instantiations for PI labels based on the graph structure and the node
class labels, and further propose a new uncertainty-aware training technique to
mitigate the negative effect of the sub-optimal PI labels. Extensive
experiments on different datasets and GNN architectures demonstrate the
effectiveness of PI, yielding a promising improvement over the state-of-the-art
methods.Comment: 16 pages, 3 figure