2,581 research outputs found
Recommender Systems with Generative Retrieval
Modern recommender systems perform large-scale retrieval by first embedding
queries and item candidates in the same unified space, followed by approximate
nearest neighbor search to select top candidates given a query embedding. In
this paper, we propose a novel generative retrieval approach, where the
retrieval model autoregressively decodes the identifiers of the target
candidates. To that end, we create semantically meaningful tuple of codewords
to serve as a Semantic ID for each item. Given Semantic IDs for items in a user
session, a Transformer-based sequence-to-sequence model is trained to predict
the Semantic ID of the next item that the user will interact with. To the best
of our knowledge, this is the first Semantic ID-based generative model for
recommendation tasks. We show that recommender systems trained with the
proposed paradigm significantly outperform the current SOTA models on various
datasets. In addition, we show that incorporating Semantic IDs into the
sequence-to-sequence model enhances its ability to generalize, as evidenced by
the improved retrieval performance observed for items with no prior interaction
history.Comment: Preprint versio
Leveraging Large Language Models in Conversational Recommender Systems
A Conversational Recommender System (CRS) offers increased transparency and
control to users by enabling them to engage with the system through a real-time
multi-turn dialogue. Recently, Large Language Models (LLMs) have exhibited an
unprecedented ability to converse naturally and incorporate world knowledge and
common-sense reasoning into language understanding, unlocking the potential of
this paradigm. However, effectively leveraging LLMs within a CRS introduces new
technical challenges, including properly understanding and controlling a
complex conversation and retrieving from external sources of information. These
issues are exacerbated by a large, evolving item corpus and a lack of
conversational data for training. In this paper, we provide a roadmap for
building an end-to-end large-scale CRS using LLMs. In particular, we propose
new implementations for user preference understanding, flexible dialogue
management and explainable recommendations as part of an integrated
architecture powered by LLMs. For improved personalization, we describe how an
LLM can consume interpretable natural language user profiles and use them to
modulate session-level context. To overcome conversational data limitations in
the absence of an existing production CRS, we propose techniques for building a
controllable LLM-based user simulator to generate synthetic conversations. As a
proof of concept we introduce RecLLM, a large-scale CRS for YouTube videos
built on LaMDA, and demonstrate its fluency and diverse functionality through
some illustrative example conversations
Mixture of Virtual-Kernel Experts for Multi-Objective User Profile Modeling
In many industrial applications like online advertising and recommendation
systems, diverse and accurate user profiles can greatly help improve
personalization. For building user profiles, deep learning is widely used to
mine expressive tags to describe users' preferences from their historical
actions. For example, tags mined from users' click-action history can represent
the categories of ads that users are interested in, and they are likely to
continue being clicked in the future. Traditional solutions usually introduce
multiple independent Two-Tower models to mine tags from different actions,
e.g., click, conversion. However, the models cannot learn complementarily and
support effective training for data-sparse actions. Besides, limited by the
lack of information fusion between the two towers, the model learning is
insufficient to represent users' preferences on various topics well. This paper
introduces a novel multi-task model called Mixture of Virtual-Kernel Experts
(MVKE) to learn multiple topic-related user preferences based on different
actions unitedly. In MVKE, we propose a concept of Virtual-Kernel Expert, which
focuses on modeling one particular facet of the user's preference, and all of
them learn coordinately. Besides, the gate-based structure used in MVKE builds
an information fusion bridge between two towers, improving the model's
capability much and maintaining high efficiency. We apply the model in Tencent
Advertising System, where both online and offline evaluations show that our
method has a significant improvement compared with the existing ones and brings
about an obvious lift to actual advertising revenue.Comment: 10 pages, under revie
Handling Large-scale Cardinality in building recommendation systems
Effective recommendation systems rely on capturing user preferences, often
requiring incorporating numerous features such as universally unique
identifiers (UUIDs) of entities. However, the exceptionally high cardinality of
UUIDs poses a significant challenge in terms of model degradation and increased
model size due to sparsity. This paper presents two innovative techniques to
address the challenge of high cardinality in recommendation systems.
Specifically, we propose a bag-of-words approach, combined with layer sharing,
to substantially decrease the model size while improving performance. Our
techniques were evaluated through offline and online experiments on Uber use
cases, resulting in promising results demonstrating our approach's
effectiveness in optimizing recommendation systems and enhancing their overall
performance
Multi-granularity Item-based Contrastive Recommendation
Contrastive learning (CL) has shown its power in recommendation. However,
most CL-based recommendation models build their CL tasks merely focusing on the
user's aspects, ignoring the rich diverse information in items. In this work,
we propose a novel Multi-granularity item-based contrastive learning (MicRec)
framework for the matching stage (i.e., candidate generation) in
recommendation, which systematically introduces multi-aspect item-related
information to representation learning with CL. Specifically, we build three
item-based CL tasks as a set of plug-and-play auxiliary objectives to capture
item correlations in feature, semantic and session levels. The feature-level
item CL aims to learn the fine-grained feature-level item correlations via
items and their augmentations. The semantic-level item CL focuses on the
coarse-grained semantic correlations between semantically related items. The
session-level item CL highlights the global behavioral correlations of items
from users' sequential behaviors in all sessions. In experiments, we conduct
both offline and online evaluations on real-world datasets, verifying the
effectiveness and universality of three proposed CL tasks. Currently, MicRec
has been deployed on a real-world recommender system, affecting millions of
users. The source code will be released in the future.Comment: 17 pages, under revie
Multi-Objective Personalized Product Retrieval in Taobao Search
In large-scale e-commerce platforms like Taobao, it is a big challenge to
retrieve products that satisfy users from billions of candidates. This has been
a common concern of academia and industry. Recently, plenty of works in this
domain have achieved significant improvements by enhancing embedding-based
retrieval (EBR) methods, including the Multi-Grained Deep Semantic Product
Retrieval (MGDSPR) model [16] in Taobao search engine. However, we find that
MGDSPR still has problems of poor relevance and weak personalization compared
to other retrieval methods in our online system, such as lexical matching and
collaborative filtering. These problems promote us to further strengthen the
capabilities of our EBR model in both relevance estimation and personalized
retrieval. In this paper, we propose a novel Multi-Objective Personalized
Product Retrieval (MOPPR) model with four hierarchical optimization objectives:
relevance, exposure, click and purchase. We construct entire-space
multi-positive samples to train MOPPR, rather than the single-positive samples
for existing EBR models.We adopt a modified softmax loss for optimizing
multiple objectives. Results of extensive offline and online experiments show
that MOPPR outperforms the baseline MGDSPR on evaluation metrics of relevance
estimation and personalized retrieval. MOPPR achieves 0.96% transaction and
1.29% GMV improvements in a 28-day online A/B test. Since the Double-11
shopping festival of 2021, MOPPR has been fully deployed in mobile Taobao
search, replacing the previous MGDSPR. Finally, we discuss several advanced
topics of our deeper explorations on multi-objective retrieval and ranking to
contribute to the community.Comment: 9 pages, 4 figures, submitted to the 28th ACM SIGKDD Conference on
Knowledge Discovery & Data Minin
Impressions in Recommender Systems: Present and Future
Impressions are a novel data source providing researchers and practitioners with more details about user interactions and their context. In particular, an impression contain the items shown on screen to users, alongside users' interactions toward such items. In recent years, interest in impressions has thrived, and more papers use impressions in recommender systems. Despite this, the literature does not contain a comprehensive review of the current topics and future directions. This work summarizes impressions in recommender systems under three perspectives: recommendation models, datasets with impressions, and evaluation methodologies. Then, we propose several future directions with an emphasis on novel approaches. This work is part of an ongoing review of impressions in recommender systems
Attribute Simulation for Item Embedding Enhancement in Multi-interest Recommendation
Although multi-interest recommenders have achieved significant progress in
the matching stage, our research reveals that existing models tend to exhibit
an under-clustered item embedding space, which leads to a low discernibility
between items and hampers item retrieval. This highlights the necessity for
item embedding enhancement. However, item attributes, which serve as effective
and straightforward side information for enhancement, are either unavailable or
incomplete in many public datasets due to the labor-intensive nature of manual
annotation tasks. This dilemma raises two meaningful questions: 1. Can we
bypass manual annotation and directly simulate complete attribute information
from the interaction data? And 2. If feasible, how to simulate attributes with
high accuracy and low complexity in the matching stage?
In this paper, we first establish an inspiring theoretical feasibility that
the item-attribute correlation matrix can be approximated through elementary
transformations on the item co-occurrence matrix. Then based on formula
derivation, we propose a simple yet effective module, SimEmb (Item Embedding
Enhancement via Simulated Attribute), in the multi-interest recommendation of
the matching stage to implement our findings. By simulating attributes with the
co-occurrence matrix, SimEmb discards the item ID-based embedding and employs
the attribute-weighted summation for item embedding enhancement. Comprehensive
experiments on four benchmark datasets demonstrate that our approach notably
enhances the clustering of item embedding and significantly outperforms SOTA
models with an average improvement of 25.59% on [email protected]: This paper has been accepted by the 17th ACM International Conference
on Web Search and Data Mining (WSDM 2024). The camera-ready version will be
available in the conference proceeding
- …