3,032 research outputs found
CAST: Cluster-Aware Self-Training for Tabular Data
Self-training has gained attraction because of its simplicity and
versatility, yet it is vulnerable to noisy pseudo-labels. Several studies have
proposed successful approaches to tackle this issue, but they have diminished
the advantages of self-training because they require specific modifications in
self-training algorithms or model architectures. Furthermore, most of them are
incompatible with gradient boosting decision trees, which dominate the tabular
domain. To address this, we revisit the cluster assumption, which states that
data samples that are close to each other tend to belong to the same class.
Inspired by the assumption, we propose Cluster-Aware Self-Training (CAST) for
tabular data. CAST is a simple and universally adaptable approach for enhancing
existing self-training algorithms without significant modifications.
Concretely, our method regularizes the confidence of the classifier, which
represents the value of the pseudo-label, forcing the pseudo-labels in
low-density regions to have lower confidence by leveraging prior knowledge for
each class within the training data. Extensive empirical evaluations on up to
20 real-world datasets confirm not only the superior performance of CAST but
also its robustness in various setups in self-training contexts.Comment: 17 pages with appendi
Improving Multi-task Learning via Seeking Task-based Flat Regions
Multi-Task Learning (MTL) is a widely-used and powerful learning paradigm for
training deep neural networks that allows learning more than one objective by a
single backbone. Compared to training tasks separately, MTL significantly
reduces computational costs, improves data efficiency, and potentially enhances
model performance by leveraging knowledge across tasks. Hence, it has been
adopted in a variety of applications, ranging from computer vision to natural
language processing and speech recognition. Among them, there is an emerging
line of work in MTL that focuses on manipulating the task gradient to derive an
ultimate gradient descent direction to benefit all tasks. Despite achieving
impressive results on many benchmarks, directly applying these approaches
without using appropriate regularization techniques might lead to suboptimal
solutions on real-world problems. In particular, standard training that
minimizes the empirical loss on the training data can easily suffer from
overfitting to low-resource tasks or be spoiled by noisy-labeled ones, which
can cause negative transfer between tasks and overall performance drop. To
alleviate such problems, we propose to leverage a recently introduced training
method, named Sharpness-aware Minimization, which can enhance model
generalization ability on single-task learning. Accordingly, we present a novel
MTL training methodology, encouraging the model to find task-based flat minima
for coherently improving its generalization capability on all tasks. Finally,
we conduct comprehensive experiments on a variety of applications to
demonstrate the merit of our proposed approach to existing gradient-based MTL
methods, as suggested by our developed theory.Comment: 29 pages, 11 figures, 6 table
Toward Explainable Fashion Recommendation
Many studies have been conducted so far to build systems for recommending
fashion items and outfits. Although they achieve good performances in their
respective tasks, most of them cannot explain their judgments to the users,
which compromises their usefulness. Toward explainable fashion recommendation,
this study proposes a system that is able not only to provide a goodness score
for an outfit but also to explain the score by providing reason behind it. For
this purpose, we propose a method for quantifying how influential each feature
of each item is to the score. Using this influence value, we can identify which
item and what feature make the outfit good or bad. We represent the image of
each item with a combination of human-interpretable features, and thereby the
identification of the most influential item-feature pair gives useful
explanation of the output score. To evaluate the performance of this approach,
we design an experiment that can be performed without human annotation; we
replace a single item-feature pair in an outfit so that the score will
decrease, and then we test if the proposed method can detect the replaced item
correctly using the above influence values. The experimental results show that
the proposed method can accurately detect bad items in outfits lowering their
scores
Confidence-Calibrated Face and Kinship Verification
In this paper, we investigate the problem of prediction confidence in face
and kinship verification. Most existing face and kinship verification methods
focus on accuracy performance while ignoring confidence estimation for their
prediction results. However, confidence estimation is essential for modeling
reliability and trustworthiness in such high-risk tasks. To address this, we
introduce an effective confidence measure that allows verification models to
convert a similarity score into a confidence score for any given face pair. We
further propose a confidence-calibrated approach, termed Angular Scaling
Calibration (ASC). ASC is easy to implement and can be readily applied to
existing verification models without model modifications, yielding
accuracy-preserving and confidence-calibrated probabilistic verification
models. In addition, we introduce the uncertainty in the calibrated confidence
to boost the reliability and trustworthiness of the verification models in the
presence of noisy data. To the best of our knowledge, our work presents the
first comprehensive confidence-calibrated solution for modern face and kinship
verification tasks. We conduct extensive experiments on four widely used face
and kinship verification datasets, and the results demonstrate the
effectiveness of our proposed approach. Code and models are available at
https://github.com/cnulab/ASC.Comment: 14 pages, 10 figures, and 9 tables, IEEE Transactions on Information
Forensics and Securit
Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation
We address the problem of semantic nighttime image segmentation and improve
the state-of-the-art, by adapting daytime models to nighttime without using
nighttime annotations. Moreover, we design a new evaluation framework to
address the substantial uncertainty of semantics in nighttime images. Our
central contributions are: 1) a curriculum framework to gradually adapt
semantic segmentation models from day to night through progressively darker
times of day, exploiting cross-time-of-day correspondences between daytime
images from a reference map and dark images to guide the label inference in the
dark domains; 2) a novel uncertainty-aware annotation and evaluation framework
and metric for semantic segmentation, including image regions beyond human
recognition capability in the evaluation in a principled fashion; 3) the Dark
Zurich dataset, comprising 2416 unlabeled nighttime and 2920 unlabeled twilight
images with correspondences to their daytime counterparts plus a set of 201
nighttime images with fine pixel-level annotations created with our protocol,
which serves as a first benchmark for our novel evaluation. Experiments show
that our map-guided curriculum adaptation significantly outperforms
state-of-the-art methods on nighttime sets both for standard metrics and our
uncertainty-aware metric. Furthermore, our uncertainty-aware evaluation reveals
that selective invalidation of predictions can improve results on data with
ambiguous content such as our benchmark and profit safety-oriented applications
involving invalid inputs.Comment: IEEE T-PAMI 202
Predicting Out-of-Domain Generalization with Local Manifold Smoothness
Understanding how machine learning models generalize to new environments is a
critical part of their safe deployment. Recent work has proposed a variety of
complexity measures that directly predict or theoretically bound the
generalization capacity of a model. However, these methods rely on a strong set
of assumptions that in practice are not always satisfied. Motivated by the
limited settings in which existing measures can be applied, we propose a novel
complexity measure based on the local manifold smoothness of a classifier. We
define local manifold smoothness as a classifier's output sensitivity to
perturbations in the manifold neighborhood around a given test point.
Intuitively, a classifier that is less sensitive to these perturbations should
generalize better. To estimate smoothness we sample points using data
augmentation and measure the fraction of these points classified into the
majority class. Our method only requires selecting a data augmentation method
and makes no other assumptions about the model or data distributions, meaning
it can be applied even in out-of-domain (OOD) settings where existing methods
cannot. In experiments on robustness benchmarks in image classification,
sentiment analysis, and natural language inference, we demonstrate a strong and
robust correlation between our manifold smoothness measure and actual OOD
generalization on over 3,000 models evaluated on over 100 train/test domain
pairs.Comment: 18 pages, 3 figure
- …