2 research outputs found
A Primal-Dual Subgradient Approachfor Fair Meta Learning
The problem of learning to generalize to unseen classes during training,
known as few-shot classification, has attracted considerable attention.
Initialization based methods, such as the gradient-based model agnostic
meta-learning (MAML), tackle the few-shot learning problem by "learning to
fine-tune". The goal of these approaches is to learn proper model
initialization, so that the classifiers for new classes can be learned from a
few labeled examples with a small number of gradient update steps. Few shot
meta-learning is well-known with its fast-adapted capability and accuracy
generalization onto unseen tasks. Learning fairly with unbiased outcomes is
another significant hallmark of human intelligence, which is rarely touched in
few-shot meta-learning. In this work, we propose a Primal-Dual Fair
Meta-learning framework, namely PDFM, which learns to train fair machine
learning models using only a few examples based on data from related tasks. The
key idea is to learn a good initialization of a fair model's primal and dual
parameters so that it can adapt to a new fair learning task via a few gradient
update steps. Instead of manually tuning the dual parameters as hyperparameters
via a grid search, PDFM optimizes the initialization of the primal and dual
parameters jointly for fair meta-learning via a subgradient primal-dual
approach. We further instantiate examples of bias controlling using mean
difference and decision boundary covariance as fairness constraints to each
task for supervised regression and classification, respectively. We demonstrate
the versatility of our proposed approach by applying our approach to various
real-world datasets. Our experiments show substantial improvements over the
best prior work for this setting.Comment: 20th IEEE International Conference on Data Mining (ICDM 2020
Learning Adaptive Embedding Considering Incremental Class
Class-Incremental Learning (CIL) aims to train a reliable model with the
streaming data, which emerges unknown classes sequentially. Different from
traditional closed set learning, CIL has two main challenges: 1) Novel class
detection. The initial training data only contains incomplete classes, and
streaming test data will accept unknown classes. Therefore, the model needs to
not only accurately classify known classes, but also effectively detect unknown
classes; 2) Model expansion. After the novel classes are detected, the model
needs to be updated without re-training using entire previous data. However,
traditional CIL methods have not fully considered these two challenges, first,
they are always restricted to single novel class detection each phase and
embedding confusion caused by unknown classes. Besides, they also ignore the
catastrophic forgetting of known categories in model update. To this end, we
propose a Class-Incremental Learning without Forgetting (CILF) framework, which
aims to learn adaptive embedding for processing novel class detection and model
update in a unified framework. In detail, CILF designs to regularize
classification with decoupled prototype based loss, which can improve the
intra-class and inter-class structure significantly, and acquire a compact
embedding representation for novel class detection in result. Then, CILF
employs a learnable curriculum clustering operator to estimate the number of
semantic clusters via fine-tuning the learned network, in which curriculum
operator can adaptively learn the embedding in self-taught form. Therefore,
CILF can detect multiple novel classes and mitigate the embedding confusion
problem. Last, with the labeled streaming test data, CILF can update the
network with robust regularization to mitigate the catastrophic forgetting.
Consequently, CILF is able to iteratively perform novel class detection and
model update.Comment: 15 pages, 11 figure