5 research outputs found
Graph Relation Aware Continual Learning
Continual graph learning (CGL) studies the problem of learning from an
infinite stream of graph data, consolidating historical knowledge, and
generalizing it to the future task. At once, only current graph data are
available. Although some recent attempts have been made to handle this task, we
still face two potential challenges: 1) most of existing works only manipulate
on the intermediate graph embedding and ignore intrinsic properties of graphs.
It is non-trivial to differentiate the transferred information across graphs.
2) recent attempts take a parameter-sharing policy to transfer knowledge across
time steps or progressively expand new architecture given shifted graph
distribution. Learning a single model could loss discriminative information for
each graph task while the model expansion scheme suffers from high model
complexity. In this paper, we point out that latent relations behind graph
edges can be attributed as an invariant factor for the evolving graphs and the
statistical information of latent relations evolves. Motivated by this, we
design a relation-aware adaptive model, dubbed as RAM-CG, that consists of a
relation-discovery modular to explore latent relations behind edges and a
task-awareness masking classifier to accounts for the shifted. Extensive
experiments show that RAM-CG provides significant 2.2%, 6.9% and 6.6% accuracy
improvements over the state-of-the-art results on CitationNet, OGBN-arxiv and
TWITCH dataset, respective
T-SaS: Toward Shift-aware Dynamic Adaptation for Streaming Data
In many real-world scenarios, distribution shifts exist in the streaming data
across time steps. Many complex sequential data can be effectively divided into
distinct regimes that exhibit persistent dynamics. Discovering the shifted
behaviors and the evolving patterns underlying the streaming data are important
to understand the dynamic system. Existing methods typically train one robust
model to work for the evolving data of distinct distributions or sequentially
adapt the model utilizing explicitly given regime boundaries. However, there
are two challenges: (1) shifts in data streams could happen drastically and
abruptly without precursors. Boundaries of distribution shifts are usually
unavailable, and (2) training a shared model for all domains could fail to
capture varying patterns. This paper aims to solve the problem of sequential
data modeling in the presence of sudden distribution shifts that occur without
any precursors. Specifically, we design a Bayesian framework, dubbed as T-SaS,
with a discrete distribution-modeling variable to capture abrupt shifts of
data. Then, we design a model that enable adaptation with dynamic network
selection conditioned on that discrete variable. The proposed method learns
specific model parameters for each distribution by learning which neurons
should be activated in the full network. A dynamic masking strategy is adopted
here to support inter-distribution transfer through the overlapping of a set of
sparse networks. Extensive experiments show that our proposed method is
superior in both accurately detecting shift boundaries to get segments of
varying distributions and effectively adapting to downstream forecast or
classification tasks.Comment: CIKM 202
Mitigating Popularity Bias in Recommendation with Unbalanced Interactions: A Gradient Perspective
Recommender systems learn from historical user-item interactions to identify
preferred items for target users. These observed interactions are usually
unbalanced following a long-tailed distribution. Such long-tailed data lead to
popularity bias to recommend popular but not personalized items to users. We
present a gradient perspective to understand two negative impacts of popularity
bias in recommendation model optimization: (i) the gradient direction of
popular item embeddings is closer to that of positive interactions, and (ii)
the magnitude of positive gradient for popular items are much greater than that
of unpopular items. To address these issues, we propose a simple yet efficient
framework to mitigate popularity bias from a gradient perspective.
Specifically, we first normalize each user embedding and record accumulated
gradients of users and items via popularity bias measures in model training. To
address the popularity bias issues, we develop a gradient-based embedding
adjustment approach used in model testing. This strategy is generic,
model-agnostic, and can be seamlessly integrated into most existing recommender
systems. Our extensive experiments on two classic recommendation models and
four real-world datasets demonstrate the effectiveness of our method over
state-of-the-art debiasing baselines.Comment: Recommendation System, Popularity Bia