16 research outputs found
Combining Adversarial Guarantees and Stochastic Fast Rates in Online Learning
We consider online learning algorithms that guarantee worst-case regret rates
in adversarial environments (so they can be deployed safely and will perform
robustly), yet adapt optimally to favorable stochastic environments (so they
will perform well in a variety of settings of practical importance). We
quantify the friendliness of stochastic environments by means of the well-known
Bernstein (a.k.a. generalized Tsybakov margin) condition. For two recent
algorithms (Squint for the Hedge setting and MetaGrad for online convex
optimization) we show that the particular form of their data-dependent
individual-sequence regret guarantees implies that they adapt automatically to
the Bernstein parameters of the stochastic environment. We prove that these
algorithms attain fast rates in their respective settings both in expectation
and with high probability
Lipschitz Adaptivity with Multiple Learning Rates in Online Learning
We aim to design adaptive online learning algorithms that take advantage of
any special structure that might be present in the learning task at hand, with
as little manual tuning by the user as possible. A fundamental obstacle that
comes up in the design of such adaptive algorithms is to calibrate a so-called
step-size or learning rate hyperparameter depending on variance, gradient
norms, etc. A recent technique promises to overcome this difficulty by
maintaining multiple learning rates in parallel. This technique has been
applied in the MetaGrad algorithm for online convex optimization and the Squint
algorithm for prediction with expert advice. However, in both cases the user
still has to provide in advance a Lipschitz hyperparameter that bounds the norm
of the gradients. Although this hyperparameter is typically not available in
advance, tuning it correctly is crucial: if it is set too small, the methods
may fail completely; but if it is taken too large, performance deteriorates
significantly. In the present work we remove this Lipschitz hyperparameter by
designing new versions of MetaGrad and Squint that adapt to its optimal value
automatically. We achieve this by dynamically updating the set of active
learning rates. For MetaGrad, we further improve the computational efficiency
of handling constraints on the domain of prediction, and we remove the need to
specify the number of rounds in advance.Comment: 22 pages. To appear in COLT 201
Beating Stochastic and Adversarial Semi-bandits Optimally and Simultaneously
We develop the first general semi-bandit algorithm that simultaneously
achieves regret for stochastic environments and
regret for adversarial environments without knowledge
of the regime or the number of rounds . The leading problem-dependent
constants of our bounds are not only optimal in some worst-case sense studied
previously, but also optimal for two concrete instances of semi-bandit
problems. Our algorithm and analysis extend the recent work of (Zimmert &
Seldin, 2019) for the special case of multi-armed bandit, but importantly
requires a novel hybrid regularizer designed specifically for semi-bandit.
Experimental results on synthetic data show that our algorithm indeed performs
well uniformly over different environments. We finally provide a preliminary
extension of our results to the full bandit feedback
Lipschitz Adaptivity with Multiple Learning Rates in Online Learning
We aim to design adaptive online learning algorithms that take advantage of any special structure
that might be present in the learning task at hand, with as little manual tuning by the user as possible.
A fundamental obstacle that comes up in the design of such adaptive algorithms is to calibrate
a so-called step-size or learning rate hyperparameter depending on variance, gradient norms, etc.
A recent technique promises to overcome this difficulty by maintaining multiple learning rates in
parallel. This technique has been applied in the MetaGrad algorithm for online convex optimization
and the Squint algorithm for prediction with expert advice. However, in both cases the user still has
to provide in advance a Lipschitz hyperparameter that bounds the norm of the gradients. Although
this hyperparameter is typically not available in advance, tuning it correctly is crucial: if it is set
too small, the methods may fail completely; but if it is taken too large, performance deteriorates
significantly. In the present work we remove this Lipschitz hyperparameter by designing new
versions of MetaGrad and Squint that adapt to its optimal value automatically. We achieve this
by dynamically updating the set of active learning rates. For MetaGrad, we further improve the
computational efficiency of handling constraints on the domain of prediction, and we remove the
need to specify the number of rounds in advance
MetaGrad: Multiple Learning Rates in Online Learning
Analysis and Stochastic