264 research outputs found
Whole-Chain Recommendations
With the recent prevalence of Reinforcement Learning (RL), there have been
tremendous interests in developing RL-based recommender systems. In practical
recommendation sessions, users will sequentially access multiple scenarios,
such as the entrance pages and the item detail pages, and each scenario has its
specific characteristics. However, the majority of existing RL-based
recommender systems focus on optimizing one strategy for all scenarios or
separately optimizing each strategy, which could lead to sub-optimal overall
performance. In this paper, we study the recommendation problem with multiple
(consecutive) scenarios, i.e., whole-chain recommendations. We propose a
multi-agent RL-based approach (DeepChain), which can capture the sequential
correlation among different scenarios and jointly optimize multiple
recommendation strategies. To be specific, all recommender agents (RAs) share
the same memory of users' historical behaviors, and they work collaboratively
to maximize the overall reward of a session. Note that optimizing multiple
recommendation strategies jointly faces two challenges in the existing
model-free RL model - (i) it requires huge amounts of user behavior data, and
(ii) the distribution of reward (users' feedback) are extremely unbalanced. In
this paper, we introduce model-based RL techniques to reduce the training data
requirement and execute more accurate strategy updates. The experimental
results based on a real e-commerce platform demonstrate the effectiveness of
the proposed framework.Comment: 29th ACM International Conference on Information and Knowledge
Managemen
Reinforcement Knowledge Graph Reasoning for Explainable Recommendation
Recent advances in personalized recommendation have sparked great interest in
the exploitation of rich structured information provided by knowledge graphs.
Unlike most existing approaches that only focus on leveraging knowledge graphs
for more accurate recommendation, we perform explicit reasoning with knowledge
for decision making so that the recommendations are generated and supported by
an interpretable causal inference procedure. To this end, we propose a method
called Policy-Guided Path Reasoning (PGPR), which couples recommendation and
interpretability by providing actual paths in a knowledge graph. Our
contributions include four aspects. We first highlight the significance of
incorporating knowledge graphs into recommendation to formally define and
interpret the reasoning process. Second, we propose a reinforcement learning
(RL) approach featuring an innovative soft reward strategy, user-conditional
action pruning and a multi-hop scoring function. Third, we design a
policy-guided graph search algorithm to efficiently and effectively sample
reasoning paths for recommendation. Finally, we extensively evaluate our method
on several large-scale real-world benchmark datasets, obtaining favorable
results compared with state-of-the-art methods.Comment: Accepted in SIGIR 201
Intrinsically Motivated Reinforcement Learning based Recommendation with Counterfactual Data Augmentation
Deep reinforcement learning (DRL) has been proven its efficiency in capturing
users' dynamic interests in recent literature. However, training a DRL agent is
challenging, because of the sparse environment in recommender systems (RS), DRL
agents could spend times either exploring informative user-item interaction
trajectories or using existing trajectories for policy learning. It is also
known as the exploration and exploitation trade-off which affects the
recommendation performance significantly when the environment is sparse. It is
more challenging to balance the exploration and exploitation in DRL RS where RS
agent need to deeply explore the informative trajectories and exploit them
efficiently in the context of recommender systems. As a step to address this
issue, We design a novel intrinsically ,otivated reinforcement learning method
to increase the capability of exploring informative interaction trajectories in
the sparse environment, which are further enriched via a counterfactual
augmentation strategy for more efficient exploitation. The extensive
experiments on six offline datasets and three online simulation platforms
demonstrate the superiority of our model to a set of existing state-of-the-art
methods
Explainability in Music Recommender Systems
The most common way to listen to recorded music nowadays is via streaming
platforms which provide access to tens of millions of tracks. To assist users
in effectively browsing these large catalogs, the integration of Music
Recommender Systems (MRSs) has become essential. Current real-world MRSs are
often quite complex and optimized for recommendation accuracy. They combine
several building blocks based on collaborative filtering and content-based
recommendation. This complexity can hinder the ability to explain
recommendations to end users, which is particularly important for
recommendations perceived as unexpected or inappropriate. While pure
recommendation performance often correlates with user satisfaction,
explainability has a positive impact on other factors such as trust and
forgiveness, which are ultimately essential to maintain user loyalty.
In this article, we discuss how explainability can be addressed in the
context of MRSs. We provide perspectives on how explainability could improve
music recommendation algorithms and enhance user experience. First, we review
common dimensions and goals of recommenders' explainability and in general of
eXplainable Artificial Intelligence (XAI), and elaborate on the extent to which
these apply -- or need to be adapted -- to the specific characteristics of
music consumption and recommendation. Then, we show how explainability
components can be integrated within a MRS and in what form explanations can be
provided. Since the evaluation of explanation quality is decoupled from pure
accuracy-based evaluation criteria, we also discuss requirements and strategies
for evaluating explanations of music recommendations. Finally, we describe the
current challenges for introducing explainability within a large-scale
industrial music recommender system and provide research perspectives.Comment: To appear in AI Magazine, Special Topic on Recommender Systems 202
Adversarial Training Towards Robust Multimedia Recommender System
With the prevalence of multimedia content on the Web, developing recommender
solutions that can effectively leverage the rich signal in multimedia data is
in urgent need. Owing to the success of deep neural networks in representation
learning, recent advance on multimedia recommendation has largely focused on
exploring deep learning methods to improve the recommendation accuracy. To
date, however, there has been little effort to investigate the robustness of
multimedia representation and its impact on the performance of multimedia
recommendation.
In this paper, we shed light on the robustness of multimedia recommender
system. Using the state-of-the-art recommendation framework and deep image
features, we demonstrate that the overall system is not robust, such that a
small (but purposeful) perturbation on the input image will severely decrease
the recommendation accuracy. This implies the possible weakness of multimedia
recommender system in predicting user preference, and more importantly, the
potential of improvement by enhancing its robustness. To this end, we propose a
novel solution named Adversarial Multimedia Recommendation (AMR), which can
lead to a more robust multimedia recommender model by using adversarial
learning. The idea is to train the model to defend an adversary, which adds
perturbations to the target image with the purpose of decreasing the model's
accuracy. We conduct experiments on two representative multimedia
recommendation tasks, namely, image recommendation and visually-aware product
recommendation. Extensive results verify the positive effect of adversarial
learning and demonstrate the effectiveness of our AMR method. Source codes are
available in https://github.com/duxy-me/AMR.Comment: TKD
KuaiSim: A Comprehensive Simulator for Recommender Systems
Reinforcement Learning (RL)-based recommender systems (RSs) have garnered
considerable attention due to their ability to learn optimal recommendation
policies and maximize long-term user rewards. However, deploying RL models
directly in online environments and generating authentic data through A/B tests
can pose challenges and require substantial resources. Simulators offer an
alternative approach by providing training and evaluation environments for RS
models, reducing reliance on real-world data. Existing simulators have shown
promising results but also have limitations such as simplified user feedback,
lacking consistency with real-world data, the challenge of simulator
evaluation, and difficulties in migration and expansion across RSs. To address
these challenges, we propose KuaiSim, a comprehensive user environment that
provides user feedback with multi-behavior and cross-session responses. The
resulting simulator can support three levels of recommendation problems: the
request level list-wise recommendation task, the whole-session level sequential
recommendation task, and the cross-session level retention optimization task.
For each task, KuaiSim also provides evaluation protocols and baseline
recommendation algorithms that further serve as benchmarks for future research.
We also restructure existing competitive simulators on the KuaiRand Dataset and
compare them against KuaiSim to future assess their performance and behavioral
differences. Furthermore, to showcase KuaiSim's flexibility in accommodating
different datasets, we demonstrate its versatility and robustness when
deploying it on the ML-1m dataset
- …