98 research outputs found
Carousel Personalization in Music Streaming Apps with Contextual Bandits
Media services providers, such as music streaming platforms, frequently
leverage swipeable carousels to recommend personalized content to their users.
However, selecting the most relevant items (albums, artists, playlists...) to
display in these carousels is a challenging task, as items are numerous and as
users have different preferences. In this paper, we model carousel
personalization as a contextual multi-armed bandit problem with multiple plays,
cascade-based updates and delayed batch feedback. We empirically show the
effectiveness of our framework at capturing characteristics of real-world
carousels by addressing a large-scale playlist recommendation task on a global
music streaming mobile app. Along with this paper, we publicly release
industrial data from our experiments, as well as an open-source environment to
simulate comparable carousel personalization learning problems.Comment: 14th ACM Conference on Recommender Systems (RecSys 2020, Best Short
Paper Candidate
On Learning to Rank Long Sequences with Contextual Bandits
Motivated by problems of learning to rank long item sequences, we introduce a
variant of the cascading bandit model that considers flexible length sequences
with varying rewards and losses. We formulate two generative models for this
problem within the generalized linear setting, and design and analyze upper
confidence algorithms for it. Our analysis delivers tight regret bounds which,
when specialized to vanilla cascading bandits, results in sharper guarantees
than previously available in the literature. We evaluate our algorithms on a
number of real-world datasets, and show significantly improved empirical
performance as compared to known cascading bandit baselines
Online Corrupted User Detection and Regret Minimization
In real-world online web systems, multiple users usually arrive sequentially
into the system. For applications like click fraud and fake reviews, some users
can maliciously perform corrupted (disrupted) behaviors to trick the system.
Therefore, it is crucial to design efficient online learning algorithms to
robustly learn from potentially corrupted user behaviors and accurately
identify the corrupted users in an online manner. Existing works propose bandit
algorithms robust to adversarial corruption. However, these algorithms are
designed for a single user, and cannot leverage the implicit social relations
among multiple users for more efficient learning. Moreover, none of them
consider how to detect corrupted users online in the multiple-user scenario. In
this paper, we present an important online learning problem named LOCUD to
learn and utilize unknown user relations from disrupted behaviors to speed up
learning, and identify the corrupted users in an online setting. To robustly
learn and utilize the unknown relations among potentially corrupted users, we
propose a novel bandit algorithm RCLUB-WCU. To detect the corrupted users, we
devise a novel online detection algorithm OCCUD based on RCLUB-WCU's inferred
user relations. We prove a regret upper bound for RCLUB-WCU, which
asymptotically matches the lower bound with respect to up to logarithmic
factors, and matches the state-of-the-art results in degenerate cases. We also
give a theoretical guarantee for the detection accuracy of OCCUD. With
extensive experiments, our methods achieve superior performance over previous
bandit algorithms and high corrupted user detection accuracy
Master-slave Deep Architecture for Top-K Multi-armed Bandits with Non-linear Bandit Feedback and Diversity Constraints
We propose a novel master-slave architecture to solve the top-
combinatorial multi-armed bandits problem with non-linear bandit feedback and
diversity constraints, which, to the best of our knowledge, is the first
combinatorial bandits setting considering diversity constraints under bandit
feedback. Specifically, to efficiently explore the combinatorial and
constrained action space, we introduce six slave models with distinguished
merits to generate diversified samples well balancing rewards and constraints
as well as efficiency. Moreover, we propose teacher learning based optimization
and the policy co-training technique to boost the performance of the multiple
slave models. The master model then collects the elite samples provided by the
slave models and selects the best sample estimated by a neural contextual
UCB-based network to make a decision with a trade-off between exploration and
exploitation. Thanks to the elaborate design of slave models, the co-training
mechanism among slave models, and the novel interactions between the master and
slave models, our approach significantly surpasses existing state-of-the-art
algorithms in both synthetic and real datasets for recommendation tasks. The
code is available at:
\url{https://github.com/huanghanchi/Master-slave-Algorithm-for-Top-K-Bandits}.Comment: IEEE Transactions on Neural Networks and Learning System
Online Clustering of Bandits with Misspecified User Models
The contextual linear bandit is an important online learning problem where
given arm features, a learning agent selects an arm at each round to maximize
the cumulative rewards in the long run. A line of works, called the clustering
of bandits (CB), utilize the collaborative effect over user preferences and
have shown significant improvements over classic linear bandit algorithms.
However, existing CB algorithms require well-specified linear user models and
can fail when this critical assumption does not hold. Whether robust CB
algorithms can be designed for more practical scenarios with misspecified user
models remains an open problem. In this paper, we are the first to present the
important problem of clustering of bandits with misspecified user models
(CBMUM), where the expected rewards in user models can be perturbed away from
perfect linear models. We devise two robust CB algorithms, RCLUMB and RSCLUMB
(representing the learned clustering structure with dynamic graph and sets,
respectively), that can accommodate the inaccurate user preference estimations
and erroneous clustering caused by model misspecifications. We prove regret
upper bounds of for our
algorithms under milder assumptions than previous CB works (notably, we move
past a restrictive technical assumption on the distribution of the arms), which
match the lower bound asymptotically in up to logarithmic factors, and also
match the state-of-the-art results in several degenerate cases. The techniques
in proving the regret caused by misclustering users are quite general and may
be of independent interest. Experiments on both synthetic and real-world data
show our outperformance over previous algorithms
Clustered Linear Contextual Bandits with Knapsacks
In this work, we study clustered contextual bandits where rewards and
resource consumption are the outcomes of cluster-specific linear models. The
arms are divided in clusters, with the cluster memberships being unknown to an
algorithm. Pulling an arm in a time period results in a reward and in
consumption for each one of multiple resources, and with the total consumption
of any resource exceeding a constraint implying the termination of the
algorithm. Thus, maximizing the total reward requires learning not only models
about the reward and the resource consumption, but also cluster memberships. We
provide an algorithm that achieves regret sublinear in the number of time
periods, without requiring access to all of the arms. In particular, we show
that it suffices to perform clustering only once to a randomly selected subset
of the arms. To achieve this result, we provide a sophisticated combination of
techniques from the literature of econometrics and of bandits with constraints
- …