668 research outputs found
Efficient and Robust Algorithms for Adversarial Linear Contextual Bandits
We consider an adversarial variant of the classic -armed linear contextual
bandit problem where the sequence of loss functions associated with each arm
are allowed to change without restriction over time. Under the assumption that
the -dimensional contexts are generated i.i.d.~at random from a known
distributions, we develop computationally efficient algorithms based on the
classic Exp3 algorithm. Our first algorithm, RealLinExp3, is shown to achieve a
regret guarantee of over rounds, which matches
the best available bound for this problem. Our second algorithm, RobustLinExp3,
is shown to be robust to misspecification, in that it achieves a regret bound
of if the true
reward function is linear up to an additive nonlinear error uniformly bounded
in absolute value by . To our knowledge, our performance
guarantees constitute the very first results on this problem setting
A Neural Networks Committee for the Contextual Bandit Problem
This paper presents a new contextual bandit algorithm, NeuralBandit, which
does not need hypothesis on stationarity of contexts and rewards. Several
neural networks are trained to modelize the value of rewards knowing the
context. Two variants, based on multi-experts approach, are proposed to choose
online the parameters of multi-layer perceptrons. The proposed algorithms are
successfully tested on a large dataset with and without stationarity of
rewards.Comment: 21st International Conference on Neural Information Processin
Online and Distribution-Free Robustness: Regression and Contextual Bandits with Huber Contamination
In this work we revisit two classic high-dimensional online learning
problems, namely linear regression and contextual bandits, from the perspective
of adversarial robustness. Existing works in algorithmic robust statistics make
strong distributional assumptions that ensure that the input data is evenly
spread out or comes from a nice generative model. Is it possible to achieve
strong robustness guarantees even without distributional assumptions
altogether, where the sequence of tasks we are asked to solve is adaptively and
adversarially chosen?
We answer this question in the affirmative for both linear regression and
contextual bandits. In fact our algorithms succeed where conventional methods
fail. In particular we show strong lower bounds against Huber regression and
more generally any convex M-estimator. Our approach is based on a novel
alternating minimization scheme that interleaves ordinary least-squares with a
simple convex program that finds the optimal reweighting of the distribution
under a spectral constraint. Our results obtain essentially optimal dependence
on the contamination level , reach the optimal breakdown point, and
naturally apply to infinite dimensional settings where the feature vectors are
represented implicitly via a kernel map.Comment: 66 pages, 1 figure, v3: refined exposition and improved rate
Bias-Robust Bayesian Optimization via Dueling Bandits
We consider Bayesian optimization in settings where observations can be
adversarially biased, for example by an uncontrolled hidden confounder. Our
first contribution is a reduction of the confounded setting to the dueling
bandit model. Then we propose a novel approach for dueling bandits based on
information-directed sampling (IDS). Thereby, we obtain the first efficient
kernelized algorithm for dueling bandits that comes with cumulative regret
guarantees. Our analysis further generalizes a previously proposed
semi-parametric linear bandit model to non-linear reward functions, and
uncovers interesting links to doubly-robust estimation
- …