25 research outputs found
GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretation
Recent advancements in Natural Language Processing (NLP) have led to the
development of NLP-based recommender systems that have shown superior
performance. However, current models commonly treat items as mere IDs and adopt
discriminative modeling, resulting in limitations of (1) fully leveraging the
content information of items and the language modeling capabilities of NLP
models; (2) interpreting user interests to improve relevance and diversity; and
(3) adapting practical circumstances such as growing item inventories. To
address these limitations, we present GPT4Rec, a novel and flexible generative
framework inspired by search engines. It first generates hypothetical "search
queries" given item titles in a user's history, and then retrieves items for
recommendation by searching these queries. The framework overcomes previous
limitations by learning both user and item embeddings in the language space. To
well-capture user interests with different aspects and granularity for
improving relevance and diversity, we propose a multi-query generation
technique with beam search. The generated queries naturally serve as
interpretable representations of user interests and can be searched to
recommend cold-start items. With GPT-2 language model and BM25 search engine,
our framework outperforms state-of-the-art methods by and in
Recall@K on two public datasets. Experiments further revealed that multi-query
generation with beam search improves both the diversity of retrieved items and
the coverage of a user's multi-interests. The adaptiveness and interpretability
of generated queries are discussed with qualitative case studies
Latent User Intent Modeling for Sequential Recommenders
Sequential recommender models are essential components of modern industrial
recommender systems. These models learn to predict the next items a user is
likely to interact with based on his/her interaction history on the platform.
Most sequential recommenders however lack a higher-level understanding of user
intents, which often drive user behaviors online. Intent modeling is thus
critical for understanding users and optimizing long-term user experience. We
propose a probabilistic modeling approach and formulate user intent as latent
variables, which are inferred based on user behavior signals using variational
autoencoders (VAE). The recommendation policy is then adjusted accordingly
given the inferred user intent. We demonstrate the effectiveness of the latent
user intent modeling via offline analyses as well as live experiments on a
large-scale industrial recommendation platform.Comment: The Web Conference 2023, Industry Trac
Multi-granularity Item-based Contrastive Recommendation
Contrastive learning (CL) has shown its power in recommendation. However,
most CL-based recommendation models build their CL tasks merely focusing on the
user's aspects, ignoring the rich diverse information in items. In this work,
we propose a novel Multi-granularity item-based contrastive learning (MicRec)
framework for the matching stage (i.e., candidate generation) in
recommendation, which systematically introduces multi-aspect item-related
information to representation learning with CL. Specifically, we build three
item-based CL tasks as a set of plug-and-play auxiliary objectives to capture
item correlations in feature, semantic and session levels. The feature-level
item CL aims to learn the fine-grained feature-level item correlations via
items and their augmentations. The semantic-level item CL focuses on the
coarse-grained semantic correlations between semantically related items. The
session-level item CL highlights the global behavioral correlations of items
from users' sequential behaviors in all sessions. In experiments, we conduct
both offline and online evaluations on real-world datasets, verifying the
effectiveness and universality of three proposed CL tasks. Currently, MicRec
has been deployed on a real-world recommender system, affecting millions of
users. The source code will be released in the future.Comment: 17 pages, under revie
Trinity: Syncretizing Multi-/Long-tail/Long-term Interests All in One
Interest modeling in recommender system has been a constant topic for
improving user experience, and typical interest modeling tasks (e.g.
multi-interest, long-tail interest and long-term interest) have been
investigated in many existing works. However, most of them only consider one
interest in isolation, while neglecting their interrelationships. In this
paper, we argue that these tasks suffer from a common "interest amnesia"
problem, and a solution exists to mitigate it simultaneously. We figure that
long-term cues can be the cornerstone since they reveal multi-interest and
clarify long-tail interest. Inspired by the observation, we propose a novel and
unified framework in the retrieval stage, "Trinity", to solve interest amnesia
problem and improve multiple interest modeling tasks. We construct a real-time
clustering system that enables us to project items into enumerable clusters,
and calculate statistical interest histograms over these clusters. Based on
these histograms, Trinity recognizes underdelivered themes and remains stable
when facing emerging hot topics. Trinity is more appropriate for large-scale
industry scenarios because of its modest computational overheads. Its derived
retrievers have been deployed on the recommender system of Douyin,
significantly improving user experience and retention. We believe that such
practical experience can be well generalized to other scenarios
CROLoss: Towards a Customizable Loss for Retrieval Models in Recommender Systems
In large-scale recommender systems, retrieving top N relevant candidates
accurately with resource constrain is crucial. To evaluate the performance of
such retrieval models, Recall@N, the frequency of positive samples being
retrieved in the top N ranking, is widely used. However, most of the
conventional loss functions for retrieval models such as softmax cross-entropy
and pairwise comparison methods do not directly optimize Recall@N. Moreover,
those conventional loss functions cannot be customized for the specific
retrieval size N required by each application and thus may lead to sub-optimal
performance. In this paper, we proposed the Customizable Recall@N Optimization
Loss (CROLoss), a loss function that can directly optimize the Recall@N metrics
and is customizable for different choices of N. This proposed CROLoss
formulation defines a more generalized loss function space, covering most of
the conventional loss functions as special cases. Furthermore, we develop the
Lambda method, a gradient-based method that invites more flexibility and can
further boost the system performance. We evaluate the proposed CROLoss on two
public benchmark datasets. The results show that CROLoss achieves SOTA results
over conventional loss functions for both datasets with various choices of
retrieval size N. CROLoss has been deployed onto our online E-commerce
advertising platform, where a fourteen-day online A/B test demonstrated that
CROLoss contributes to a significant business revenue growth of 4.75%.Comment: 9 pages, 5 figures. Accepted by by CIKM 202
DiffuRec: A Diffusion Model for Sequential Recommendation
Mainstream solutions to Sequential Recommendation (SR) represent items with
fixed vectors. These vectors have limited capability in capturing items' latent
aspects and users' diverse preferences. As a new generative paradigm, Diffusion
models have achieved excellent performance in areas like computer vision and
natural language processing. To our understanding, its unique merit in
representation generation well fits the problem setting of sequential
recommendation. In this paper, we make the very first attempt to adapt
Diffusion model to SR and propose DiffuRec, for item representation
construction and uncertainty injection. Rather than modeling item
representations as fixed vectors, we represent them as distributions in
DiffuRec, which reflect user's multiple interests and item's various aspects
adaptively. In diffusion phase, DiffuRec corrupts the target item embedding
into a Gaussian distribution via noise adding, which is further applied for
sequential item distribution representation generation and uncertainty
injection. Afterwards, the item representation is fed into an Approximator for
target item representation reconstruction. In reversion phase, based on user's
historical interaction behaviors, we reverse a Gaussian noise into the target
item representation, then apply rounding operation for target item prediction.
Experiments over four datasets show that DiffuRec outperforms strong baselines
by a large margin
Attribute Simulation for Item Embedding Enhancement in Multi-interest Recommendation
Although multi-interest recommenders have achieved significant progress in
the matching stage, our research reveals that existing models tend to exhibit
an under-clustered item embedding space, which leads to a low discernibility
between items and hampers item retrieval. This highlights the necessity for
item embedding enhancement. However, item attributes, which serve as effective
and straightforward side information for enhancement, are either unavailable or
incomplete in many public datasets due to the labor-intensive nature of manual
annotation tasks. This dilemma raises two meaningful questions: 1. Can we
bypass manual annotation and directly simulate complete attribute information
from the interaction data? And 2. If feasible, how to simulate attributes with
high accuracy and low complexity in the matching stage?
In this paper, we first establish an inspiring theoretical feasibility that
the item-attribute correlation matrix can be approximated through elementary
transformations on the item co-occurrence matrix. Then based on formula
derivation, we propose a simple yet effective module, SimEmb (Item Embedding
Enhancement via Simulated Attribute), in the multi-interest recommendation of
the matching stage to implement our findings. By simulating attributes with the
co-occurrence matrix, SimEmb discards the item ID-based embedding and employs
the attribute-weighted summation for item embedding enhancement. Comprehensive
experiments on four benchmark datasets demonstrate that our approach notably
enhances the clustering of item embedding and significantly outperforms SOTA
models with an average improvement of 25.59% on [email protected]: This paper has been accepted by the 17th ACM International Conference
on Web Search and Data Mining (WSDM 2024). The camera-ready version will be
available in the conference proceeding
FairSync: Ensuring Amortized Group Exposure in Distributed Recommendation Retrieval
In pursuit of fairness and balanced development, recommender systems (RS)
often prioritize group fairness, ensuring that specific groups maintain a
minimum level of exposure over a given period. For example, RS platforms aim to
ensure adequate exposure for new providers or specific categories of items
according to their needs. Modern industry RS usually adopts a two-stage
pipeline: stage-1 (retrieval stage) retrieves hundreds of candidates from
millions of items distributed across various servers, and stage-2 (ranking
stage) focuses on presenting a small-size but accurate selection from items
chosen in stage-1. Existing efforts for ensuring amortized group exposures
focus on stage-2, however, stage-1 is also critical for the task. Without a
high-quality set of candidates, the stage-2 ranker cannot ensure the required
exposure of groups. Previous fairness-aware works designed for stage-2
typically require accessing and traversing all items. In stage-1, however,
millions of items are distributively stored in servers, making it infeasible to
traverse all of them. How to ensure group exposures in the distributed
retrieval process is a challenging question. To address this issue, we
introduce a model named FairSync, which transforms the problem into a
constrained distributed optimization problem. Specifically, FairSync resolves
the issue by moving it to the dual space, where a central node aggregates
historical fairness data into a vector and distributes it to all servers. To
trade off the efficiency and accuracy, the gradient descent technique is used
to periodically update the parameter of the dual vector. The experiment results
on two public recommender retrieval datasets showcased that FairSync
outperformed all the baselines, achieving the desired minimum level of
exposures while maintaining a high level of retrieval accuracy.Comment: Accepted in WWW'2
Understanding and Modeling Passive-Negative Feedback for Short-video Sequential Recommendation
Sequential recommendation is one of the most important tasks in recommender
systems, which aims to recommend the next interacted item with historical
behaviors as input. Traditional sequential recommendation always mainly
considers the collected positive feedback such as click, purchase, etc.
However, in short-video platforms such as TikTok, video viewing behavior may
not always represent positive feedback. Specifically, the videos are played
automatically, and users passively receive the recommended videos. In this new
scenario, users passively express negative feedback by skipping over videos
they do not like, which provides valuable information about their preferences.
Different from the negative feedback studied in traditional recommender
systems, this passive-negative feedback can reflect users' interests and serve
as an important supervision signal in extracting users' preferences. Therefore,
it is essential to carefully design and utilize it in this novel recommendation
scenario. In this work, we first conduct analyses based on a large-scale
real-world short-video behavior dataset and illustrate the significance of
leveraging passive feedback. We then propose a novel method that deploys the
sub-interest encoder, which incorporates positive feedback and passive-negative
feedback as supervision signals to learn the user's current active
sub-interest. Moreover, we introduce an adaptive fusion layer to integrate
various sub-interests effectively. To enhance the robustness of our model, we
then introduce a multi-task learning module to simultaneously optimize two
kinds of feedback -- passive-negative feedback and traditional randomly-sampled
negative feedback. The experiments on two large-scale datasets verify that the
proposed method can significantly outperform state-of-the-art approaches. The
code is released at https://github.com/tsinghua-fib-lab/RecSys2023-SINE.Comment: Accepted by RecSys'2