2,207 research outputs found
Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction
User response prediction, which models the user preference w.r.t. the
presented items, plays a key role in online services. With two-decade rapid
development, nowadays the cumulated user behavior sequences on mature Internet
service platforms have become extremely long since the user's first
registration. Each user not only has intrinsic tastes, but also keeps changing
her personal interests during lifetime. Hence, it is challenging to handle such
lifelong sequential modeling for each individual user. Existing methodologies
for sequential modeling are only capable of dealing with relatively recent user
behaviors, which leaves huge space for modeling long-term especially lifelong
sequential patterns to facilitate user modeling. Moreover, one user's behavior
may be accounted for various previous behaviors within her whole online
activity history, i.e., long-term dependency with multi-scale sequential
patterns. In order to tackle these challenges, in this paper, we propose a
Hierarchical Periodic Memory Network for lifelong sequential modeling with
personalized memorization of sequential patterns for each user. The model also
adopts a hierarchical and periodical updating mechanism to capture multi-scale
sequential patterns of user interests while supporting the evolving user
behavior logs. The experimental results over three large-scale real-world
datasets have demonstrated the advantages of our proposed model with
significant improvement in user response prediction performance against the
state-of-the-arts.Comment: SIGIR 2019. Reproducible codes and datasets:
https://github.com/alimamarankgroup/HPM
ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation
With large language models (LLMs) achieving remarkable breakthroughs in
natural language processing (NLP) domains, LLM-enhanced recommender systems
have received much attention and have been actively explored currently. In this
paper, we focus on adapting and empowering a pure large language model for
zero-shot and few-shot recommendation tasks. First and foremost, we identify
and formulate the lifelong sequential behavior incomprehension problem for LLMs
in recommendation domains, i.e., LLMs fail to extract useful information from a
textual context of long user behavior sequence, even if the length of context
is far from reaching the context limitation of LLMs. To address such an issue
and improve the recommendation performance of LLMs, we propose a novel
framework, namely Retrieval-enhanced Large Language models (ReLLa) for
recommendation tasks in both zero-shot and few-shot settings. For zero-shot
recommendation, we perform semantic user behavior retrieval (SUBR) to improve
the data quality of testing samples, which greatly reduces the difficulty for
LLMs to extract the essential knowledge from user behavior sequences. As for
few-shot recommendation, we further design retrieval-enhanced instruction
tuning (ReiT) by adopting SUBR as a data augmentation technique for training
samples. Specifically, we develop a mixed training dataset consisting of both
the original data samples and their retrieval-enhanced counterparts. We conduct
extensive experiments on a real-world public dataset (i.e., MovieLens-1M) to
demonstrate the superiority of ReLLa compared with existing baseline models, as
well as its capability for lifelong sequential behavior comprehension.Comment: Under Revie
TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou
Life-long user behavior modeling, i.e., extracting a user's hidden interests
from rich historical behaviors in months or even years, plays a central role in
modern CTR prediction systems. Conventional algorithms mostly follow two
cascading stages: a simple General Search Unit (GSU) for fast and coarse search
over tens of thousands of long-term behaviors and an Exact Search Unit (ESU)
for effective Target Attention (TA) over the small number of finalists from
GSU. Although efficient, existing algorithms mostly suffer from a crucial
limitation: the \textit{inconsistent} target-behavior relevance metrics between
GSU and ESU. As a result, their GSU usually misses highly relevant behaviors
but retrieves ones considered irrelevant by ESU. In such case, the TA in ESU,
no matter how attention is allocated, mostly deviates from the real user
interests and thus degrades the overall CTR prediction accuracy. To address
such inconsistency, we propose \textbf{TWo-stage Interest Network (TWIN)},
where our Consistency-Preserved GSU (CP-GSU) adopts the identical
target-behavior relevance metric as the TA in ESU, making the two stages twins.
Specifically, to break TA's computational bottleneck and extend it from ESU to
GSU, or namely from behavior length to length , we build a
novel attention mechanism by behavior feature splitting. For the video inherent
features of a behavior, we calculate their linear projection by efficient
pre-computing \& caching strategies. And for the user-item cross features, we
compress each into a one-dimentional bias term in the attention score
calculation to save the computational cost. The consistency between two stages,
together with the effective TA-based relevance metric in CP-GSU, contributes to
significant performance gain in CTR prediction.Comment: Accepted by KDD 202
AutoAttention: Automatic Field Pair Selection for Attention in User Behavior Modeling
In Click-through rate (CTR) prediction models, a user's interest is usually
represented as a fixed-length vector based on her history behaviors. Recently,
several methods are proposed to learn an attentive weight for each user
behavior and conduct weighted sum pooling. However, these methods only manually
select several fields from the target item side as the query to interact with
the behaviors, neglecting the other target item fields, as well as user and
context fields. Directly including all these fields in the attention may
introduce noise and deteriorate the performance. In this paper, we propose a
novel model named AutoAttention, which includes all item/user/context side
fields as the query, and assigns a learnable weight for each field pair between
behavior fields and query fields. Pruning on these field pairs via these
learnable weights lead to automatic field pair selection, so as to identify and
remove noisy field pairs. Though including more fields, the computation cost of
AutoAttention is still low due to using a simple attention function and field
pair selection. Extensive experiments on the public dataset and Tencent's
production dataset demonstrate the effectiveness of the proposed approach.Comment: Accepted by ICDM 202
User Multi-Interest Modeling for Behavioral Cognition
Representation modeling based on user behavior sequences is an important
direction in user cognition. In this study, we propose a novel framework called
Multi-Interest User Representation Model. Specifically, the model consists of
two sub-models. The first sub-module is used to encode user behaviors in any
period into a super-high dimensional sparse vector. Then, we design a
self-supervised network to map vectors in the first module to low-dimensional
dense user representations by contrastive learning. With the help of a novel
attention module which can learn multi-interests of user, the second sub-module
achieves almost lossless dimensionality reduction. Experiments on several
benchmark datasets show that our approach works well and outperforms
state-of-the-art unsupervised representation methods in different downstream
tasks.Comment: during peer revie
TBIN: Modeling Long Textual Behavior Data for CTR Prediction
Click-through rate (CTR) prediction plays a pivotal role in the success of
recommendations. Inspired by the recent thriving of language models (LMs), a
surge of works improve prediction by organizing user behavior data in a
\textbf{textual} format and using LMs to understand user interest at a semantic
level. While promising, these works have to truncate the textual data to reduce
the quadratic computational overhead of self-attention in LMs. However, it has
been studied that long user behavior data can significantly benefit CTR
prediction. In addition, these works typically condense user diverse interests
into a single feature vector, which hinders the expressive capability of the
model. In this paper, we propose a \textbf{T}extual \textbf{B}ehavior-based
\textbf{I}nterest Chunking \textbf{N}etwork (TBIN), which tackles the above
limitations by combining an efficient locality-sensitive hashing algorithm and
a shifted chunk-based self-attention. The resulting user diverse interests are
dynamically activated, producing user interest representation towards the
target item. Finally, the results of both offline and online experiments on
real-world food recommendation platform demonstrate the effectiveness of TBIN
Temporal Interest Network for Click-Through Rate Prediction
The history of user behaviors constitutes one of the most significant
characteristics in predicting the click-through rate (CTR), owing to their
strong semantic and temporal correlation with the target item. While the
literature has individually examined each of these correlations, research has
yet to analyze them in combination, that is, the quadruple correlation of
(behavior semantics, target semantics, behavior temporal, and target temporal).
The effect of this correlation on performance and the extent to which existing
methods learn it remain unknown. To address this gap, we empirically measure
the quadruple correlation and observe intuitive yet robust quadruple patterns.
We measure the learned correlation of several representative user behavior
methods, but to our surprise, none of them learn such a pattern, especially the
temporal one.
In this paper, we propose the Temporal Interest Network (TIN) to capture the
quadruple semantic and temporal correlation between behaviors and the target.
We achieve this by incorporating target-aware temporal encoding, in addition to
semantic embedding, to represent behaviors and the target. Furthermore, we
deploy target-aware attention, along with target-aware representation, to
explicitly conduct the 4-way interaction. We performed comprehensive
evaluations on the Amazon and Alibaba datasets. Our proposed TIN outperforms
the best-performing baselines by 0.43\% and 0.29\% on two datasets,
respectively. Comprehensive analysis and visualization show that TIN is indeed
capable of learning the quadruple correlation effectively, while all existing
methods fail to do so. We provide our implementation of TIN in Tensorflow
Modeling Occasion Evolution in Frequency Domain for Promotion-Aware Click-Through Rate Prediction
Promotions are becoming more important and prevalent in e-commerce to attract
customers and boost sales, leading to frequent changes of occasions, which
drives users to behave differently. In such situations, most existing
Click-Through Rate (CTR) models can't generalize well to online serving due to
distribution uncertainty of the upcoming occasion. In this paper, we propose a
novel CTR model named MOEF for recommendations under frequent changes of
occasions. Firstly, we design a time series that consists of occasion signals
generated from the online business scenario. Since occasion signals are more
discriminative in the frequency domain, we apply Fourier Transformation to
sliding time windows upon the time series, obtaining a sequence of frequency
spectrum which is then processed by Occasion Evolution Layer (OEL). In this
way, a high-order occasion representation can be learned to handle the online
distribution uncertainty. Moreover, we adopt multiple experts to learn feature
representations from multiple aspects, which are guided by the occasion
representation via an attention mechanism. Accordingly, a mixture of feature
representations is obtained adaptively for different occasions to predict the
final CTR. Experimental results on real-world datasets validate the superiority
of MOEF and online A/B tests also show MOEF outperforms representative CTR
models significantly
ATBRG: Adaptive Target-Behavior Relational Graph Network for Effective Recommendation
Recommender system (RS) devotes to predicting user preference to a given item
and has been widely deployed in most web-scale applications. Recently,
knowledge graph (KG) attracts much attention in RS due to its abundant
connective information. Existing methods either explore independent meta-paths
for user-item pairs over KG, or employ graph neural network (GNN) on whole KG
to produce representations for users and items separately. Despite
effectiveness, the former type of methods fails to fully capture structural
information implied in KG, while the latter ignores the mutual effect between
target user and item during the embedding propagation. In this work, we propose
a new framework named Adaptive Target-Behavior Relational Graph network (ATBRG
for short) to effectively capture structural relations of target user-item
pairs over KG. Specifically, to associate the given target item with user
behaviors over KG, we propose the graph connect and graph prune techniques to
construct adaptive target-behavior relational graph. To fully distill
structural information from the sub-graph connected by rich relations in an
end-to-end fashion, we elaborate on the model design of ATBRG, equipped with
relation-aware extractor layer and representation activation layer. We perform
extensive experiments on both industrial and benchmark datasets. Empirical
results show that ATBRG consistently and significantly outperforms
state-of-the-art methods. Moreover, ATBRG has also achieved a performance
improvement of 5.1% on CTR metric after successful deployment in one popular
recommendation scenario of Taobao APP.Comment: Accepted by SIGIR 2020, full paper with 10 pages and 5 figure
- …