Improving Adaptive Online Learning Using Refined Discretization

We study unconstrained Online Linear Optimization with Lipschitz losses. Motivated by the pursuit of instance optimality, we propose a new algorithm that simultaneously achieves ($i$) the AdaGrad-style second order gradient adaptivity; and ($ii$) the comparator norm adaptivity also known as "parameter freeness" in the literature. In particular, - our algorithm does not employ the impractical doubling trick, and does not require an a priori estimate of the time-uniform Lipschitz constant; - the associated regret bound has the optimal $O(\sqrt{V_T})$ dependence on the gradient variance $V_T$, without the typical logarithmic multiplicative factor; - the leading constant in the regret bound is "almost" optimal. Central to these results is a continuous time approach to online learning. We first show that the aimed simultaneous adaptivity can be achieved fairly easily in a continuous time analogue of the problem, where the environment is modeled by an arbitrary continuous semimartingale. Then, our key innovation is a new discretization argument that preserves such adaptivity in the discrete time adversarial setting. This refines a non-gradient-adaptive discretization argument from (Harvey et al., 2023), both algorithmically and analytically, which could be of independent interest.Comment: ALT 202

Temporal aspects of adaptive online learning: continuity and representation

Adaptive online learning, in a very broad sense, is the study of sequential decision making beyond the worst case. Compared to their classical minimax counterparts, adaptive algorithms typically require less manual tuning, while provably performing better in benign environments, or with prior knowledge. This dissertation presents new techniques for designing these algorithms. The central theme is the emphasis on the temporal nature of the problem, which has not received enough attention in the literature. The first part of the dissertation focuses on temporal continuity. While modern online learning almost exclusively studies a discrete time repeated game, it is shown that designing algorithms can be simplified, and in certain cases optimized, by scaling the game towards a continuous time limit and solving the obtained differential equation. Concretely, we develop comparator adaptive algorithms for Online Convex Optimization, achieving optimal static regret bounds in the vanilla setting and its variant with switching costs. The benefits are extended to another classical online learning problem called Learning with Expert Advice. The second part of the dissertation focuses on temporal representation. Different from the first part, here we consider the general objective of dynamic regret minimization, which forms the foundation of time series forecasting. It is shown that by introducing temporal features, the task can be transformed to static regret minimization on a user-specified representation space with growing dimension. Drawing novel connections to wavelet features, we develop a simple algorithm improving the state-of-the-art dynamic regret bound achieved by more sophisticated approaches. An application is the online fine-tuning of a black-box time series forecaster