3 research outputs found
Adversarial Attacks on Online Learning to Rank with Stochastic Click Models
We propose the first study of adversarial attacks on online learning to rank.
The goal of the adversary is to misguide the online learning to rank algorithm
to place the target item on top of the ranking list linear times to time
horizon with a sublinear attack cost. We propose generalized list poisoning
attacks that perturb the ranking list presented to the user. This strategy can
efficiently attack any no-regret ranker in general stochastic click models.
Furthermore, we propose a click poisoning-based strategy named attack-then-quit
that can efficiently attack two representative OLTR algorithms for stochastic
click models. We theoretically analyze the success and cost upper bound of the
two proposed methods. Experimental results based on synthetic and real-world
data further validate the effectiveness and cost-efficiency of the proposed
attack strategies
Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective
Off-policy Learning to Rank (LTR) aims to optimize a ranker from data
collected by a deployed logging policy. However, existing off-policy learning
to rank methods often make strong assumptions about how users generate the
click data, i.e., the click model, and hence need to tailor their methods
specifically under different click models. In this paper, we unified the
ranking process under general stochastic click models as a Markov Decision
Process (MDP), and the optimal ranking could be learned with offline
reinforcement learning (RL) directly. Building upon this, we leverage offline
RL techniques for off-policy LTR and propose the Click Model-Agnostic Unified
Off-policy Learning to Rank (CUOLR) method, which could be easily applied to a
wide range of click models. Through a dedicated formulation of the MDP, we show
that offline RL algorithms can adapt to various click models without complex
debiasing techniques and prior knowledge of the model. Results on various
large-scale datasets demonstrate that CUOLR consistently outperforms the
state-of-the-art off-policy learning to rank algorithms while maintaining
consistency and robustness under different click models
Recommended from our members
Adversarial Attacks and Robustness of Combinatorial Multi-Armed Bandits
We study reward poisoning attacks on Combinatorial Multi-armed Bandits (CMAB). We first provide a sufficient and necessary condition for the attackability of CMAB, a notion to capture the vulnerability and robustness of CMAB. The attackability condition depends on the intrinsic properties of the corresponding CMAB instance such as the reward distributions of super arms and outcome distributions of base arms. Additionally, we devise an attack algorithm for attackable CMAB instances. Contrary to prior understanding of multi-armed bandits, our work reveals a surprising fact that the attackability of a specific CMAB instance also depends on whether the bandit instance is known or unknown to the adversary. This finding indicates that adversarial attacks on CMAB are difficult in practice and a general attack strategy for any CMAB instance does not exist since the environment is mostly unknown to the adversary. We validate our theoretical findings via extensive experiments on real-world CMAB applications including probabilistic maximum covering problem, online minimum spanning tree, cascading bandits for online ranking, and online shortest path