4 research outputs found
Adaptive Semantic Segmentation with a Strategic Curriculum of Proxy Labels
Training deep networks for semantic segmentation requires annotation of large
amounts of data, which can be time-consuming and expensive. Unfortunately,
these trained networks still generalize poorly when tested in domains not
consistent with the training data. In this paper, we show that by carefully
presenting a mixture of labeled source domain and proxy-labeled target domain
data to a network, we can achieve state-of-the-art unsupervised domain
adaptation results. With our design, the network progressively learns features
specific to the target domain using annotation from only the source domain. We
generate proxy labels for the target domain using the network's own
predictions. Our architecture then allows selective mining of easy samples from
this set of proxy labels, and hard samples from the annotated source domain. We
conduct a series of experiments with the GTA5, Cityscapes and BDD100k datasets
on synthetic-to-real domain adaptation and geographic domain adaptation,
showing the advantages of our method over baselines and existing approaches
Training Data Subset Search with Ensemble Active Learning
Deep Neural Networks (DNNs) often rely on very large datasets for training.
Given the large size of such datasets, it is conceivable that they contain
certain samples that either do not contribute or negatively impact the DNN's
optimization. Modifying the training distribution in a way that excludes such
samples could provide an effective solution to both improve performance and
reduce training time. In this paper, we propose to scale up ensemble Active
Learning (AL) methods to perform acquisition at a large scale (10k to 500k
samples at a time). We do this with ensembles of hundreds of models, obtained
at a minimal computational cost by reusing intermediate training checkpoints.
This allows us to automatically and efficiently perform a training data subset
search for large labeled datasets. We observe that our approach obtains
favorable subsets of training data, which can be used to train more accurate
DNNs than training with the entire dataset. We perform an extensive
experimental study of this phenomenon on three image classification benchmarks
(CIFAR-10, CIFAR-100 and ImageNet), as well as an internal object detection
benchmark for prototyping perception models for autonomous driving. Unlike
existing studies, our experiments on object detection are at the scale required
for production-ready autonomous driving systems. We provide insights on the
impact of different initialization schemes, acquisition functions and ensemble
configurations at this scale. Our results provide strong empirical evidence
that optimizing the training data distribution can provide significant benefits
on large scale vision tasks
Large-Scale Visual Active Learning with Deep Probabilistic Ensembles
Annotating the right data for training deep neural networks is an important
challenge. Active learning using uncertainty estimates from Bayesian Neural
Networks (BNNs) could provide an effective solution to this. Despite being
theoretically principled, BNNs require approximations to be applied to
large-scale problems, where both performance and uncertainty estimation are
crucial. In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a
scalable technique that uses a regularized ensemble to approximate a deep BNN.
We conduct a series of large-scale visual active learning experiments to
evaluate DPEs on classification with the CIFAR-10, CIFAR-100 and ImageNet
datasets, and semantic segmentation with the BDD100k dataset. Our models
require significantly less training data to achieve competitive performances,
and steadily improve upon strong active learning baselines as the annotation
budget is increased.Comment: arXiv admin note: text overlap with arXiv:1811.0264
Self-Training and Adversarial Background Regularization for Unsupervised Domain Adaptive One-Stage Object Detection
Deep learning-based object detectors have shown remarkable improvements.
However, supervised learning-based methods perform poorly when the train data
and the test data have different distributions. To address the issue, domain
adaptation transfers knowledge from the label-sufficient domain (source domain)
to the label-scarce domain (target domain). Self-training is one of the
powerful ways to achieve domain adaptation since it helps class-wise domain
adaptation. Unfortunately, a naive approach that utilizes pseudo-labels as
ground-truth degenerates the performance due to incorrect pseudo-labels. In
this paper, we introduce a weak self-training (WST) method and adversarial
background score regularization (BSR) for domain adaptive one-stage object
detection. WST diminishes the adverse effects of inaccurate pseudo-labels to
stabilize the learning procedure. BSR helps the network extract discriminative
features for target backgrounds to reduce the domain shift. Two components are
complementary to each other as BSR enhances discrimination between foregrounds
and backgrounds, whereas WST strengthen class-wise discrimination. Experimental
results show that our approach effectively improves the performance of the
one-stage object detection in unsupervised domain adaptation setting.Comment: ICCV 2019 (oral