1,926 research outputs found
Transcript of Origins of the Ukranian Fleet
This story is an excerpt from a longer interview that was collected as part of the Launching through the Surf: The Dory Fleet of Pacific City project. In this story, Noel Knopf and his son, Albert Knopf, recount the origins of the Ukranian Fleet, a group of California teachers and their children who dory fished in Pacific City during the summers
The Potential of Synergistic Static, Dynamic and Speculative Loop Nest Optimizations for Automatic Parallelization
Research in automatic parallelization of loop-centric programs started with
static analysis, then broadened its arsenal to include dynamic
inspection-execution and speculative execution, the best results involving
hybrid static-dynamic schemes. Beyond the detection of parallelism in a
sequential program, scalable parallelization on many-core processors involves
hard and interesting parallelism adaptation and mapping challenges. These
challenges include tailoring data locality to the memory hierarchy, structuring
independent tasks hierarchically to exploit multiple levels of parallelism,
tuning the synchronization grain, balancing the execution load, decoupling the
execution into thread-level pipelines, and leveraging heterogeneous hardware
with specialized accelerators. The polyhedral framework allows to model,
construct and apply very complex loop nest transformations addressing most of
the parallelism adaptation and mapping challenges. But apart from
hardware-specific, back-end oriented transformations (if-conversion, trace
scheduling, value prediction), loop nest optimization has essentially ignored
dynamic and speculative techniques. Research in polyhedral compilation recently
reached a significant milestone towards the support of dynamic, data-dependent
control flow. This opens a large avenue for blending dynamic analyses and
speculative techniques with advanced loop nest optimizations. Selecting
real-world examples from SPEC benchmarks and numerical kernels, we make a case
for the design of synergistic static, dynamic and speculative loop
transformation techniques. We also sketch the embedding of dynamic information,
including speculative assumptions, in the heart of affine transformation search
spaces
Embedding contrastive unsupervised features to cluster in- and out-of-distribution noise in corrupted image datasets
Using search engines for web image retrieval is a tempting alternative to
manual curation when creating an image dataset, but their main drawback remains
the proportion of incorrect (noisy) samples retrieved. These noisy samples have
been evidenced by previous works to be a mixture of in-distribution (ID)
samples, assigned to the incorrect category but presenting similar visual
semantics to other classes in the dataset, and out-of-distribution (OOD)
images, which share no semantic correlation with any category from the dataset.
The latter are, in practice, the dominant type of noisy images retrieved. To
tackle this noise duality, we propose a two stage algorithm starting with a
detection step where we use unsupervised contrastive feature learning to
represent images in a feature space. We find that the alignment and uniformity
principles of contrastive learning allow OOD samples to be linearly separated
from ID samples on the unit hypersphere. We then spectrally embed the
unsupervised representations using a fixed neighborhood size and apply an
outlier sensitive clustering at the class level to detect the clean and OOD
clusters as well as ID noisy outliers. We finally train a noise robust neural
network that corrects ID noise to the correct category and utilizes OOD samples
in a guided contrastive objective, clustering them to improve low-level
features. Our algorithm improves the state-of-the-art results on synthetic
noise image datasets as well as real-world web-crawled data. Our work is fully
reproducible github.com/PaulAlbert31/SNCF.Comment: Accepted at ECCV 202
Reliable Label Bootstrapping for Semi-Supervised Learning
Reducing the amount of labels required to train convolutional neural networks
without performance degradation is key to effectively reduce human annotation
efforts. We propose Reliable Label Bootstrapping (ReLaB), an unsupervised
preprossessing algorithm which improves the performance of semi-supervised
algorithms in extremely low supervision settings. Given a dataset with few
labeled samples, we first learn meaningful self-supervised, latent features for
the data. Second, a label propagation algorithm propagates the known labels on
the unsupervised features, effectively labeling the full dataset in an
automatic fashion. Third, we select a subset of correctly labeled (reliable)
samples using a label noise detection algorithm. Finally, we train a
semi-supervised algorithm on the extended subset. We show that the selection of
the network architecture and the self-supervised algorithm are important
factors to achieve successful label propagation and demonstrate that ReLaB
substantially improves semi-supervised learning in scenarios of very limited
supervision on CIFAR-10, CIFAR-100 and mini-ImageNet. We reach average error
rates of with 1 random labeled sample per class on
CIFAR-10 and lower this error to when the labeled sample in
each class is highly representative. Our work is fully reproducible:
https://github.com/PaulAlbert31/ReLaB.Comment: 10 pages, 3 figure
Towards Robust Learning with Different Label Noise Distributions
Noisy labels are an unavoidable consequence of labeling processes and
detecting them is an important step towards preventing performance degradations
in Convolutional Neural Networks. Discarding noisy labels avoids a harmful
memorization, while the associated image content can still be exploited in a
semi-supervised learning (SSL) setup. Clean samples are usually identified
using the small loss trick, i.e. they exhibit a low loss. However, we show that
different noise distributions make the application of this trick less
straightforward and propose to continuously relabel all images to reveal a
discriminative loss against multiple distributions. SSL is then applied twice,
once to improve the clean-noisy detection and again for training the final
model. We design an experimental setup based on ImageNet32/64 for better
understanding the consequences of representation learning with differing label
noise distributions and find that non-uniform out-of-distribution noise better
resembles real-world noise and that in most cases intermediate features are not
affected by label noise corruption. Experiments in CIFAR-10/100, ImageNet32/64
and WebVision (real-world noise) demonstrate that the proposed label noise
Distribution Robust Pseudo-Labeling (DRPL) approach gives substantial
improvements over recent state-of-the-art. Code is available at
https://git.io/JJ0PV
Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning
Semi-supervised learning, i.e. jointly learning from labeled and unlabeled
samples, is an active research topic due to its key role on relaxing human
supervision. In the context of image classification, recent advances to learn
from unlabeled samples are mainly focused on consistency regularization methods
that encourage invariant predictions for different perturbations of unlabeled
samples. We, conversely, propose to learn from unlabeled data by generating
soft pseudo-labels using the network predictions. We show that a naive
pseudo-labeling overfits to incorrect pseudo-labels due to the so-called
confirmation bias and demonstrate that mixup augmentation and setting a minimum
number of labeled samples per mini-batch are effective regularization
techniques for reducing it. The proposed approach achieves state-of-the-art
results in CIFAR-10/100, SVHN, and Mini-ImageNet despite being much simpler
than other methods. These results demonstrate that pseudo-labeling alone can
outperform consistency regularization methods, while the opposite was supposed
in previous work. Source code is available at https://git.io/fjQsC
Roll Calibration for CryoSat-2: a comprehensive approach
International audienceCryoSat-2 is the first satellite mission carrying a high pulse repetition frequency radar altimeter with interferometric capability on board. Across track interferometry allows the angle to the point of closest approach to be determined by combining echoes received by two antennas and knowledge of their orientation. Accurate information of the platform mispointing angles, in particular of the roll, is crucial to determine the angle of arrival in the across-track direction with sufficient accuracy. As a consequence, different methods were designed in the CryoSat-2 calibration plan in order to estimate interferometer performance along with the mission and to assess the roll’s contribution to the accuracy of the angle of arrival. In this paper, we present the comprehensive approach used in the CryoSat-2 Mission to calibrate the roll mispointing angle, combining analysis from external calibration of both man-made targets, i.e., transponder and natural targets. The roll calibration approach for CryoSat-2 is proven to guarantee that the interferometric measurements are exceeding the expected performance
Unsupervised label noise modeling and loss correction
Despite being robust to small amounts of label noise, convolutional neural networks trained with stochastic gradient methods have been shown to easily fit random labels. When there are a mixture of correct and mislabelled targets, networks
tend to fit the former before the latter. This suggests using a suitable two-component mixture model as an unsupervised generative model of sample loss values during training to allow online estimation of the probability that a sample is mislabelled. Specifically, we propose a beta mixture to estimate this probability and correct the loss by relying on the network prediction (the so-called bootstrapping loss). We further adapt mixup augmentation to drive our approach a step further. Experiments on CIFAR-10/100 and TinyImageNet demonstrate a robustness to label noise that substantially outperforms recent state-of-the-art. Source code is available at https://git.io/fjsvE and Appendix at https://arxiv.org/abs/1904.11238
- …