432 research outputs found
Parameter-Free Online Convex Optimization with Sub-Exponential Noise
We consider the problem of unconstrained online convex optimization (OCO)
with sub-exponential noise, a strictly more general problem than the standard
OCO. In this setting, the learner receives a subgradient of the loss functions
corrupted by sub-exponential noise and strives to achieve optimal regret
guarantee, without knowledge of the competitor norm, i.e., in a parameter-free
way. Recently, Cutkosky and Boahen (COLT 2017) proved that, given unbounded
subgradients, it is impossible to guarantee a sublinear regret due to an
exponential penalty. This paper shows that it is possible to go around the
lower bound by allowing the observed subgradients to be unbounded via
stochastic noise. However, the presence of unbounded noise in unconstrained OCO
is challenging; existing algorithms do not provide near-optimal regret bounds
or fail to have a guarantee. So, we design a novel parameter-free OCO algorithm
for Banach space, which we call BANCO, via a reduction to betting on noisy
coins. We show that BANCO achieves the optimal regret rate in our problem.
Finally, we show the application of our results to obtain a parameter-free
locally private stochastic subgradient descent algorithm, and the connection to
the law of iterated logarithms.Comment: v1: Accepted to COLT'19, v2: adjusted Theorem 3, w_t closed form
solution, and typo
Lipschitz Adaptivity with Multiple Learning Rates in Online Learning
We aim to design adaptive online learning algorithms that take advantage of
any special structure that might be present in the learning task at hand, with
as little manual tuning by the user as possible. A fundamental obstacle that
comes up in the design of such adaptive algorithms is to calibrate a so-called
step-size or learning rate hyperparameter depending on variance, gradient
norms, etc. A recent technique promises to overcome this difficulty by
maintaining multiple learning rates in parallel. This technique has been
applied in the MetaGrad algorithm for online convex optimization and the Squint
algorithm for prediction with expert advice. However, in both cases the user
still has to provide in advance a Lipschitz hyperparameter that bounds the norm
of the gradients. Although this hyperparameter is typically not available in
advance, tuning it correctly is crucial: if it is set too small, the methods
may fail completely; but if it is taken too large, performance deteriorates
significantly. In the present work we remove this Lipschitz hyperparameter by
designing new versions of MetaGrad and Squint that adapt to its optimal value
automatically. We achieve this by dynamically updating the set of active
learning rates. For MetaGrad, we further improve the computational efficiency
of handling constraints on the domain of prediction, and we remove the need to
specify the number of rounds in advance.Comment: 22 pages. To appear in COLT 201
Online learning with imperfect hints
We consider a variant of the classical online linear
optimization problem in which at every step,
the online player receives a “hint” vector before
choosing the action for that round. Rather surprisingly,
it was shown that if the hint vector is
guaranteed to have a positive correlation with the
cost vector, then the online player can achieve
a regret of O(log T), thus significantly improving
over the O(pT) regret in the general setting.
However, the result and analysis require the correlation
property at all time steps, thus raising the
natural question: can we design online learning
algorithms that are resilient to bad hints?
In this paper we develop algorithms and nearly
matching lower bounds for online learning with
imperfect directional hints. Our algorithms are
oblivious to the quality of the hints, and the regret
bounds interpolate between the always-correlated
hints case and the no-hints case. Our results also
generalize, simplify, and improve upon previous
results on optimistic regret bounds, which can be
viewed as an additive version of hints.http://proceedings.mlr.press/v119/bhaskara20a.htmlPublished versio
- …