Recently, much work has been done on extending the scope of online learning
and incremental stochastic optimization algorithms. In this paper we contribute
to this effort in two ways: First, based on a new regret decomposition and a
generalization of Bregman divergences, we provide a self-contained, modular
analysis of the two workhorses of online learning: (general) adaptive versions
of Mirror Descent (MD) and the Follow-the-Regularized-Leader (FTRL) algorithms.
The analysis is done with extra care so as not to introduce assumptions not
needed in the proofs and allows to combine, in a straightforward way, different
algorithmic ideas (e.g., adaptivity, optimism, implicit updates) and learning
settings (e.g., strongly convex or composite objectives). This way we are able
to reprove, extend and refine a large body of the literature, while keeping the
proofs concise. The second contribution is a byproduct of this careful
analysis: We present algorithms with improved variational bounds for smooth,
composite objectives, including a new family of optimistic MD algorithms with
only one projection step per round. Furthermore, we provide a simple extension
of adaptive regret bounds to practically relevant non-convex problem settings
with essentially no extra effort.Comment: Accepted to The 28th International Conference on Algorithmic Learning
Theory (ALT 2017). 40 page