671 research outputs found
Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction
User response prediction, which models the user preference w.r.t. the
presented items, plays a key role in online services. With two-decade rapid
development, nowadays the cumulated user behavior sequences on mature Internet
service platforms have become extremely long since the user's first
registration. Each user not only has intrinsic tastes, but also keeps changing
her personal interests during lifetime. Hence, it is challenging to handle such
lifelong sequential modeling for each individual user. Existing methodologies
for sequential modeling are only capable of dealing with relatively recent user
behaviors, which leaves huge space for modeling long-term especially lifelong
sequential patterns to facilitate user modeling. Moreover, one user's behavior
may be accounted for various previous behaviors within her whole online
activity history, i.e., long-term dependency with multi-scale sequential
patterns. In order to tackle these challenges, in this paper, we propose a
Hierarchical Periodic Memory Network for lifelong sequential modeling with
personalized memorization of sequential patterns for each user. The model also
adopts a hierarchical and periodical updating mechanism to capture multi-scale
sequential patterns of user interests while supporting the evolving user
behavior logs. The experimental results over three large-scale real-world
datasets have demonstrated the advantages of our proposed model with
significant improvement in user response prediction performance against the
state-of-the-arts.Comment: SIGIR 2019. Reproducible codes and datasets:
https://github.com/alimamarankgroup/HPM
User Multi-Interest Modeling for Behavioral Cognition
Representation modeling based on user behavior sequences is an important
direction in user cognition. In this study, we propose a novel framework called
Multi-Interest User Representation Model. Specifically, the model consists of
two sub-models. The first sub-module is used to encode user behaviors in any
period into a super-high dimensional sparse vector. Then, we design a
self-supervised network to map vectors in the first module to low-dimensional
dense user representations by contrastive learning. With the help of a novel
attention module which can learn multi-interests of user, the second sub-module
achieves almost lossless dimensionality reduction. Experiments on several
benchmark datasets show that our approach works well and outperforms
state-of-the-art unsupervised representation methods in different downstream
tasks.Comment: during peer revie
ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation
With large language models (LLMs) achieving remarkable breakthroughs in
natural language processing (NLP) domains, LLM-enhanced recommender systems
have received much attention and have been actively explored currently. In this
paper, we focus on adapting and empowering a pure large language model for
zero-shot and few-shot recommendation tasks. First and foremost, we identify
and formulate the lifelong sequential behavior incomprehension problem for LLMs
in recommendation domains, i.e., LLMs fail to extract useful information from a
textual context of long user behavior sequence, even if the length of context
is far from reaching the context limitation of LLMs. To address such an issue
and improve the recommendation performance of LLMs, we propose a novel
framework, namely Retrieval-enhanced Large Language models (ReLLa) for
recommendation tasks in both zero-shot and few-shot settings. For zero-shot
recommendation, we perform semantic user behavior retrieval (SUBR) to improve
the data quality of testing samples, which greatly reduces the difficulty for
LLMs to extract the essential knowledge from user behavior sequences. As for
few-shot recommendation, we further design retrieval-enhanced instruction
tuning (ReiT) by adopting SUBR as a data augmentation technique for training
samples. Specifically, we develop a mixed training dataset consisting of both
the original data samples and their retrieval-enhanced counterparts. We conduct
extensive experiments on a real-world public dataset (i.e., MovieLens-1M) to
demonstrate the superiority of ReLLa compared with existing baseline models, as
well as its capability for lifelong sequential behavior comprehension.Comment: Under Revie
Graph Based Long-Term And Short-Term Interest Model for Click-Through Rate Prediction
Click-through rate (CTR) prediction aims to predict the probability that the
user will click an item, which has been one of the key tasks in online
recommender and advertising systems. In such systems, rich user behavior (viz.
long- and short-term) has been proved to be of great value in capturing user
interests. Both industry and academy have paid much attention to this topic and
propose different approaches to modeling with long-term and short-term user
behavior data. But there are still some unresolved issues. More specially, (1)
rule and truncation based methods to extract information from long-term
behavior are easy to cause information loss, and (2) single feedback behavior
regardless of scenario to extract information from short-term behavior lead to
information confusion and noise. To fill this gap, we propose a Graph based
Long-term and Short-term interest Model, termed GLSM. It consists of a
multi-interest graph structure for capturing long-term user behavior, a
multi-scenario heterogeneous sequence model for modeling short-term
information, then an adaptive fusion mechanism to fused information from
long-term and short-term behaviors. Comprehensive experiments on real-world
datasets, GLSM achieved SOTA score on offline metrics. At the same time, the
GLSM algorithm has been deployed in our industrial application, bringing 4.9%
CTR and 4.3% GMV lift, which is significant to the business.Comment: CIKM 2022 accepte
TBIN: Modeling Long Textual Behavior Data for CTR Prediction
Click-through rate (CTR) prediction plays a pivotal role in the success of
recommendations. Inspired by the recent thriving of language models (LMs), a
surge of works improve prediction by organizing user behavior data in a
\textbf{textual} format and using LMs to understand user interest at a semantic
level. While promising, these works have to truncate the textual data to reduce
the quadratic computational overhead of self-attention in LMs. However, it has
been studied that long user behavior data can significantly benefit CTR
prediction. In addition, these works typically condense user diverse interests
into a single feature vector, which hinders the expressive capability of the
model. In this paper, we propose a \textbf{T}extual \textbf{B}ehavior-based
\textbf{I}nterest Chunking \textbf{N}etwork (TBIN), which tackles the above
limitations by combining an efficient locality-sensitive hashing algorithm and
a shifted chunk-based self-attention. The resulting user diverse interests are
dynamically activated, producing user interest representation towards the
target item. Finally, the results of both offline and online experiments on
real-world food recommendation platform demonstrate the effectiveness of TBIN
Multi-Granularity Click Confidence Learning via Self-Distillation in Recommendation
Recommendation systems rely on historical clicks to learn user interests and
provide appropriate items. However, current studies tend to treat clicks
equally, which may ignore the assorted intensities of user interests in
different clicks. In this paper, we aim to achieve multi-granularity Click
confidence Learning via Self-Distillation in recommendation (CLSD). Due to the
lack of supervised signals in click confidence, we first apply self-supervised
learning to obtain click confidence scores via a global self-distillation
method. After that, we define a local confidence function to adapt confidence
scores at the user group level, since the confidence distributions can be
varied among user groups. With the combination of multi-granularity confidence
learning, we can distinguish the quality of clicks and model user interests
more accurately without involving extra data and model structures. The
significant improvements over different backbones on industrial offline and
online experiments in a real-world recommender system prove the effectiveness
of our model. Recently, CLSD has been deployed on a large-scale recommender
system, affecting over 400 million users
ATBRG: Adaptive Target-Behavior Relational Graph Network for Effective Recommendation
Recommender system (RS) devotes to predicting user preference to a given item
and has been widely deployed in most web-scale applications. Recently,
knowledge graph (KG) attracts much attention in RS due to its abundant
connective information. Existing methods either explore independent meta-paths
for user-item pairs over KG, or employ graph neural network (GNN) on whole KG
to produce representations for users and items separately. Despite
effectiveness, the former type of methods fails to fully capture structural
information implied in KG, while the latter ignores the mutual effect between
target user and item during the embedding propagation. In this work, we propose
a new framework named Adaptive Target-Behavior Relational Graph network (ATBRG
for short) to effectively capture structural relations of target user-item
pairs over KG. Specifically, to associate the given target item with user
behaviors over KG, we propose the graph connect and graph prune techniques to
construct adaptive target-behavior relational graph. To fully distill
structural information from the sub-graph connected by rich relations in an
end-to-end fashion, we elaborate on the model design of ATBRG, equipped with
relation-aware extractor layer and representation activation layer. We perform
extensive experiments on both industrial and benchmark datasets. Empirical
results show that ATBRG consistently and significantly outperforms
state-of-the-art methods. Moreover, ATBRG has also achieved a performance
improvement of 5.1% on CTR metric after successful deployment in one popular
recommendation scenario of Taobao APP.Comment: Accepted by SIGIR 2020, full paper with 10 pages and 5 figure
- …