28 research outputs found
Heterogeneous Forgetting Compensation for Class-Incremental Learning
Class-incremental learning (CIL) has achieved remarkable successes in
learning new classes consecutively while overcoming catastrophic forgetting on
old categories. However, most existing CIL methods unreasonably assume that all
old categories have the same forgetting pace, and neglect negative influence of
forgetting heterogeneity among different old classes on forgetting
compensation. To surmount the above challenges, we develop a novel
Heterogeneous Forgetting Compensation (HFC) model, which can resolve
heterogeneous forgetting of easy-to-forget and hard-to-forget old categories
from both representation and gradient aspects. Specifically, we design a
task-semantic aggregation block to alleviate heterogeneous forgetting from
representation aspect. It aggregates local category information within each
task to learn task-shared global representations. Moreover, we develop two
novel plug-and-play losses: a gradient-balanced forgetting compensation loss
and a gradient-balanced relation distillation loss to alleviate forgetting from
gradient aspect. They consider gradient-balanced compensation to rectify
forgetting heterogeneity of old categories and heterogeneous relation
consistency. Experiments on several representative datasets illustrate
effectiveness of our HFC model. The code is available at
https://github.com/JiahuaDong/HFC.Comment: Accepted to ICCV202
Self-paced Weight Consolidation for Continual Learning
Continual learning algorithms which keep the parameters of new tasks close to
that of previous tasks, are popular in preventing catastrophic forgetting in
sequential task learning settings. However, 1) the performance for the new
continual learner will be degraded without distinguishing the contributions of
previously learned tasks; 2) the computational cost will be greatly increased
with the number of tasks, since most existing algorithms need to regularize all
previous tasks when learning new tasks. To address the above challenges, we
propose a self-paced Weight Consolidation (spWC) framework to attain robust
continual learning via evaluating the discriminative contributions of previous
tasks. To be specific, we develop a self-paced regularization to reflect the
priorities of past tasks via measuring difficulty based on key performance
indicator (i.e., accuracy). When encountering a new task, all previous tasks
are sorted from "difficult" to "easy" based on the priorities. Then the
parameters of the new continual learner will be learned via selectively
maintaining the knowledge amongst more difficult past tasks, which could well
overcome catastrophic forgetting with less computational cost. We adopt an
alternative convex search to iteratively update the model parameters and
priority weights in the bi-convex formulation. The proposed spWC framework is
plug-and-play, which is applicable to most continual learning algorithms (e.g.,
EWC, MAS and RCIL) in different directions (e.g., classification and
segmentation). Experimental results on several public benchmark datasets
demonstrate that our proposed framework can effectively improve performance
when compared with other popular continual learning algorithms
I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting
3D object classification has attracted appealing attentions in academic
researches and industrial applications. However, most existing methods need to
access the training data of past 3D object classes when facing the common
real-world scenario: new classes of 3D objects arrive in a sequence. Moreover,
the performance of advanced approaches degrades dramatically for past learned
classes (i.e., catastrophic forgetting), due to the irregular and redundant
geometric structures of 3D point cloud data. To address these challenges, we
propose a new Incremental 3D Object Learning (i.e., I3DOL) model, which is the
first exploration to learn new classes of 3D object continually. Specifically,
an adaptive-geometric centroid module is designed to construct discriminative
local geometric structures, which can better characterize the irregular point
cloud representation for 3D object. Afterwards, to prevent the catastrophic
forgetting brought by redundant geometric information, a geometric-aware
attention mechanism is developed to quantify the contributions of local
geometric structures, and explore unique 3D geometric characteristics with high
contributions for classes incremental learning. Meanwhile, a score fairness
compensation strategy is proposed to further alleviate the catastrophic
forgetting caused by unbalanced data between past and new classes of 3D object,
by compensating biased prediction for new classes in the validation phase.
Experiments on 3D representative datasets validate the superiority of our I3DOL
framework.Comment: Accepted by Association for the Advancement of Artificial
Intelligence 2021 (AAAI 2021
Gradient-Semantic Compensation for Incremental Semantic Segmentation
Incremental semantic segmentation aims to continually learn the segmentation
of new coming classes without accessing the training data of previously learned
classes. However, most current methods fail to address catastrophic forgetting
and background shift since they 1) treat all previous classes equally without
considering different forgetting paces caused by imbalanced gradient
back-propagation; 2) lack strong semantic guidance between classes. To tackle
the above challenges, in this paper, we propose a Gradient-Semantic
Compensation (GSC) model, which surmounts incremental semantic segmentation
from both gradient and semantic perspectives. Specifically, to address
catastrophic forgetting from the gradient aspect, we develop a step-aware
gradient compensation that can balance forgetting paces of previously seen
classes via re-weighting gradient backpropagation. Meanwhile, we propose a
soft-sharp semantic relation distillation to distill consistent inter-class
semantic relations via soft labels for alleviating catastrophic forgetting from
the semantic aspect. In addition, we develop a prototypical pseudo re-labeling
that provides strong semantic guidance to mitigate background shift. It
produces high-quality pseudo labels for old classes in the background by
measuring distances between pixels and class-wise prototypes. Extensive
experiments on three public datasets, i.e., Pascal VOC 2012, ADE20K, and
Cityscapes, demonstrate the effectiveness of our proposed GSC model
Create Your World: Lifelong Text-to-Image Diffusion
Text-to-image generative models can produce diverse high-quality images of
concepts with a text prompt, which have demonstrated excellent ability in image
generation, image translation, etc. We in this work study the problem of
synthesizing instantiations of a use's own concepts in a never-ending manner,
i.e., create your world, where the new concepts from user are quickly learned
with a few examples. To achieve this goal, we propose a Lifelong text-to-image
Diffusion Model (L2DM), which intends to overcome knowledge "catastrophic
forgetting" for the past encountered concepts, and semantic "catastrophic
neglecting" for one or more concepts in the text prompt. In respect of
knowledge "catastrophic forgetting", our L2DM framework devises a task-aware
memory enhancement module and a elastic-concept distillation module, which
could respectively safeguard the knowledge of both prior concepts and each past
personalized concept. When generating images with a user text prompt, the
solution to semantic "catastrophic neglecting" is that a concept attention
artist module can alleviate the semantic neglecting from concept aspect, and an
orthogonal attention module can reduce the semantic binding from attribute
aspect. To the end, our model can generate more faithful image across a range
of continual text prompts in terms of both qualitative and quantitative
metrics, when comparing with the related state-of-the-art models. The code will
be released at https://wenqiliang.github.io/.Comment: 15 pages,10 figure
No One Left Behind: Real-World Federated Class-Incremental Learning
Federated learning (FL) is a hot collaborative training framework via
aggregating model parameters of decentralized local clients. However, most FL
methods unreasonably assume data categories of FL framework are known and fixed
in advance. Moreover, some new local clients that collect novel categories
unseen by other clients may be introduced to FL training irregularly. These
issues render global model to undergo catastrophic forgetting on old
categories, when local clients receive new categories consecutively under
limited memory of storing old categories. To tackle the above issues, we
propose a novel Local-Global Anti-forgetting (LGA) model. It ensures no local
clients are left behind as they learn new classes continually, by addressing
local and global catastrophic forgetting. Specifically, considering tackling
class imbalance of local client to surmount local forgetting, we develop a
category-balanced gradient-adaptive compensation loss and a category
gradient-induced semantic distillation loss. They can balance heterogeneous
forgetting speeds of hard-to-forget and easy-to-forget old categories, while
ensure consistent class-relations within different tasks. Moreover, a proxy
server is designed to tackle global forgetting caused by Non-IID class
imbalance between different clients. It augments perturbed prototype images of
new categories collected from local clients via self-supervised prototype
augmentation, thus improving robustness to choose the best old global model for
local-side semantic distillation loss. Experiments on representative datasets
verify superior performance of our model against comparison methods. The code
is available at https://github.com/JiahuaDong/LGA.Comment: 17 pages, 8 figure
Incorporating Dynamic Traffic Distribution into Pavement Maintenance Optimization Model
An optimal pavement maintenance strategy can keep the pavement performance at a high level under budget constraint. However, the impact of changes in traffic distribution caused by maintenance actions on user costs is rarely investigated in existing approaches. This research aims to solve the optimization of pavement maintenance strategy using a multi-stage dynamic programming model combined with the stochastic user equilibrium model, which can simulate the dynamic traffic distribution in the life cycle. To deal with the proposed model, a heuristic iterative algorithm is put forward. Ultimately, a hypothetical network is established to test the model and algorithm. The testing results prove that the proposed framework has an advantage in assessing user costs comprehensively and can provide an effective and optimal pavement maintenance strategy in a 30-year life cycle, which improves the efficiency of budget and pavement conditions. Additionally, this research provides quantitative evidence of interdependency in a road network, i.e., pavement maintenance actions on links can interfere with the user costs and traffic flow distribution in the whole network, which should be taken into account in pavement maintenance decision-making