39 research outputs found
Deep Bilevel Learning
We present a novel regularization approach to train neural networks that
enjoys better generalization and test error than standard stochastic gradient
descent. Our approach is based on the principles of cross-validation, where a
validation set is used to limit the model overfitting. We formulate such
principles as a bilevel optimization problem. This formulation allows us to
define the optimization of a cost on the validation set subject to another
optimization on the training set. The overfitting is controlled by introducing
weights on each mini-batch in the training set and by choosing their values so
that they minimize the error on the validation set. In practice, these weights
define mini-batch learning rates in a gradient descent update equation that
favor gradients with better generalization capabilities. Because of its
simplicity, this approach can be integrated with other regularization methods
and training schemes. We evaluate extensively our proposed algorithm on several
neural network architectures and datasets, and find that it consistently
improves the generalization of the model, especially when labels are noisy.Comment: ECCV 201
Cluster consistency: Simple yet effect robust learning algorithm on large-scale photoplethysmography for atrial fibrillation detection in the presence of real-world label noise
Obtaining large-scale well-annotated is always a daunting challenge,
especially in the medical research domain because of the shortage of domain
expert. Instead of human annotation, in this work, we use the alarm information
generated from bed-side monitor to get the pseudo label for the co-current
photoplethysmography (PPG) signal. Based on this strategy, we end up with over
8 million 30-second PPG segment. To solve the label noise caused by false
alarms, we propose the cluster consistency, which use an unsupervised
auto-encoder (hence not subject to label noise) approach to cluster training
samples into a finite number of clusters. Then the learned cluster membership
is used in the subsequent supervised learning phase to force the distance in
the latent space of samples in the same cluster to be small while that of
samples in different clusters to be big. In the experiment, we compare with the
state-of-the-art algorithms and test on external datasets. The results show the
superiority of our method in both classification performance and efficiency
Elucidating and Overcoming the Challenges of Label Noise in Supervised Contrastive Learning
Image classification datasets exhibit a non-negligible fraction of mislabeled
examples, often due to human error when one class superficially resembles
another. This issue poses challenges in supervised contrastive learning (SCL),
where the goal is to cluster together data points of the same class in the
embedding space while distancing those of disparate classes. While such methods
outperform those based on cross-entropy, they are not immune to labeling
errors. However, while the detrimental effects of noisy labels in supervised
learning are well-researched, their influence on SCL remains largely
unexplored. Hence, we analyse the effect of label errors and examine how they
disrupt the SCL algorithm's ability to distinguish between positive and
negative sample pairs. Our analysis reveals that human labeling errors manifest
as easy positive samples in around 99% of cases. We, therefore, propose D-SCL,
a novel Debiased Supervised Contrastive Learning objective designed to mitigate
the bias introduced by labeling errors. We demonstrate that D-SCL consistently
outperforms state-of-the-art techniques for representation learning across
diverse vision benchmarks, offering improved robustness to label errors
A Benchmark of Long-tailed Instance Segmentation with Noisy Labels (Short Version)
In this paper, we consider the instance segmentation task on a long-tailed
dataset, which contains label noise, i.e., some of the annotations are
incorrect. There are two main reasons making this case realistic. First,
datasets collected from real world usually obey a long-tailed distribution.
Second, for instance segmentation datasets, as there are many instances in one
image and some of them are tiny, it is easier to introduce noise into the
annotations. Specifically, we propose a new dataset, which is a large
vocabulary long-tailed dataset containing label noise for instance
segmentation. Furthermore, we evaluate previous proposed instance segmentation
algorithms on this dataset. The results indicate that the noise in the training
dataset will hamper the model in learning rare categories and decrease the
overall performance, and inspire us to explore more effective approaches to
address this practical challenge. The code and dataset are available in
https://github.com/GuanlinLee/Noisy-LVIS