178,545 research outputs found
Seamlessly Unifying Attributes and Items: Conversational Recommendation for Cold-Start Users
Static recommendation methods like collaborative filtering suffer from the
inherent limitation of performing real-time personalization for cold-start
users. Online recommendation, e.g., multi-armed bandit approach, addresses this
limitation by interactively exploring user preference online and pursuing the
exploration-exploitation (EE) trade-off. However, existing bandit-based methods
model recommendation actions homogeneously. Specifically, they only consider
the items as the arms, being incapable of handling the item attributes, which
naturally provide interpretable information of user's current demands and can
effectively filter out undesired items. In this work, we consider the
conversational recommendation for cold-start users, where a system can both ask
the attributes from and recommend items to a user interactively. This important
scenario was studied in a recent work. However, it employs a hand-crafted
function to decide when to ask attributes or make recommendations. Such
separate modeling of attributes and items makes the effectiveness of the system
highly rely on the choice of the hand-crafted function, thus introducing
fragility to the system. To address this limitation, we seamlessly unify
attributes and items in the same arm space and achieve their EE trade-offs
automatically using the framework of Thompson Sampling. Our Conversational
Thompson Sampling (ConTS) model holistically solves all questions in
conversational recommendation by choosing the arm with the maximal reward to
play. Extensive experiments on three benchmark datasets show that ConTS
outperforms the state-of-the-art methods Conversational UCB (ConUCB) and
Estimation-Action-Reflection model in both metrics of success rate and average
number of conversation turns.Comment: TOIS 202
R-UCB: a Contextual Bandit Algorithm for Risk-Aware Recommender Systems
Mobile Context-Aware Recommender Systems can be naturally modelled as an
exploration/exploitation trade-off (exr/exp) problem, where the system has to
choose between maximizing its expected rewards dealing with its current
knowledge (exploitation) and learning more about the unknown user's preferences
to improve its knowledge (exploration). This problem has been addressed by the
reinforcement learning community but they do not consider the risk level of the
current user's situation, where it may be dangerous to recommend items the user
may not desire in her current situation if the risk level is high. We introduce
in this paper an algorithm named R-UCB that considers the risk level of the
user's situation to adaptively balance between exr and exp. The detailed
analysis of the experimental results reveals several important discoveries in
the exr/exp behaviour
Bandits Warm-up Cold Recommender Systems
We address the cold start problem in recommendation systems assuming no
contextual information is available neither about users, nor items. We consider
the case in which we only have access to a set of ratings of items by users.
Most of the existing works consider a batch setting, and use cross-validation
to tune parameters. The classical method consists in minimizing the root mean
square error over a training subset of the ratings which provides a
factorization of the matrix of ratings, interpreted as a latent representation
of items and users. Our contribution in this paper is 5-fold. First, we
explicit the issues raised by this kind of batch setting for users or items
with very few ratings. Then, we propose an online setting closer to the actual
use of recommender systems; this setting is inspired by the bandit framework.
The proposed methodology can be used to turn any recommender system dataset
(such as Netflix, MovieLens,...) into a sequential dataset. Then, we explicit a
strong and insightful link between contextual bandit algorithms and matrix
factorization; this leads us to a new algorithm that tackles the
exploration/exploitation dilemma associated to the cold start problem in a
strikingly new perspective. Finally, experimental evidence confirm that our
algorithm is effective in dealing with the cold start problem on publicly
available datasets. Overall, the goal of this paper is to bridge the gap
between recommender systems based on matrix factorizations and those based on
contextual bandits
Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
Contextual bandit algorithms have become popular for online recommendation
systems such as Digg, Yahoo! Buzz, and news recommendation in general.
\emph{Offline} evaluation of the effectiveness of new algorithms in these
applications is critical for protecting online user experiences but very
challenging due to their "partial-label" nature. Common practice is to create a
simulator which simulates the online environment for the problem at hand and
then run an algorithm against this simulator. However, creating simulator
itself is often difficult and modeling bias is usually unavoidably introduced.
In this paper, we introduce a \emph{replay} methodology for contextual bandit
algorithm evaluation. Different from simulator-based approaches, our method is
completely data-driven and very easy to adapt to different applications. More
importantly, our method can provide provably unbiased evaluations. Our
empirical results on a large-scale news article recommendation dataset
collected from Yahoo! Front Page conform well with our theoretical results.
Furthermore, comparisons between our offline replay and online bucket
evaluation of several contextual bandit algorithms show accuracy and
effectiveness of our offline evaluation method.Comment: 10 pages, 7 figures, revised from the published version at the WSDM
2011 conferenc
Incentivizing Exploration with Selective Data Disclosure
We study the design of rating systems that incentivize (more) efficient
social learning among self-interested agents. Agents arrive sequentially and
are presented with a set of possible actions, each of which yields a positive
reward with an unknown probability. A disclosure policy sends messages about
the rewards of previously-chosen actions to arriving agents. These messages can
alter agents' incentives towards exploration, taking potentially sub-optimal
actions for the sake of learning more about their rewards. Prior work achieves
much progress with disclosure policies that merely recommend an action to each
user, but relies heavily on standard, yet very strong rationality assumptions.
We study a particular class of disclosure policies that use messages, called
unbiased subhistories, consisting of the actions and rewards from a subsequence
of past agents. Each subsequence is chosen ahead of time, according to a
predetermined partial order on the rounds. We posit a flexible model of
frequentist agent response, which we argue is plausible for this class of
"order-based" disclosure policies. We measure the success of a policy by its
regret, i.e., the difference, over all rounds, between the expected reward of
the best action and the reward induced by the policy. A disclosure policy that
reveals full history in each round risks inducing herding behavior among the
agents, and typically has regret linear in the time horizon . Our main
result is an order-based disclosure policy that obtains regret
. This regret is known to be optimal in the worst case
over reward distributions, even absent incentives. We also exhibit simpler
order-based policies with higher, but still sublinear, regret. These policies
can be interpreted as dividing a sublinear number of agents into constant-sized
focus groups, whose histories are then revealed to future agents
- …