45,768 research outputs found

    Online Learning With Predictable Sequences

    Get PDF
    We present methods for online linear optimization that take advantage of benign (as opposed to worst-case) sequences. Specifically if the sequence encountered by the learner is described well by a known “predictable process”, the algorithms presented enjoy tighter bounds as compared to the typical worst case bounds. Additionally, the methods achieve the usual worst-case regret bounds if the sequence is not benign. Our approach can be seen as a way of adding prior knowledge about the sequence within the paradigm of online learning. The setting is shown to encompass partial and side information. Variance and path-length bounds Hazan and Kale (2010); Chiang et al. (2012) can be seen as particular examples of online learning with simple predictable sequences. We further extend our methods to include competing with a set of possible predictable processes (models), that is “learning” the predictable process itself concurrently with using it to obtain better regret guarantees. We show that such model selection is possible under various assumptions on the available feedback

    Predictable Sequences and Competing with Strategies

    Get PDF
    First, we study online learning with an extended notion of regret, which is defined with respect to a set of strategies. We develop tools for analyzing the minimax rates and deriving efficient learning algorithms in this scenario. While the standard methods for minimizing the usual notion of regret fail, through our analysis we demonstrate the existence of regret-minimization methods that compete with such sets of strategies as: autoregressive algorithms, strategies based on statistical models, regularized least squares, and follow-the-regularized-leader strategies. In several cases, we also derive efficient learning algorithms. Then we study how online linear optimization competes with strategies while benefiting from the predictable sequence. We analyze the minimax value of the online linear optimization problem and develop algorithms that take advantage of the predictable sequence and that guarantee performance compared to fixed actions. Later, we extend the story to a model selection problem on multiple predictable sequences. At the end, we re-analyze the problem from the perspective of dynamic regret. Last, we study the relationship between Approximate Entropy and Shannon Entropy, and propose the adaptive Shannon Entropy approximation methods (e.g., Lempel-Ziv sliding window method) as an alternative approach to quantify the regularity of data. The new approach has the advantage of adaptively choosing the order of regularity

    Optimization, Learning, and Games with Predictable Sequences

    Get PDF
    We provide several applications of Optimistic Mirror Descent, an online learning algorithm based on the idea of predictable sequences. First, we recover the Mirror Prox algorithm for offline optimization, prove an extension to Holder-smooth functions, and apply the results to saddle-point type problems. Next, we prove that a version of Optimistic Mirror Descent (which has a close relation to the Exponential Weights algorithm) can be used by two strongly-uncoupled players in a finite zero-sum matrix game to converge to the minimax equilibrium at the rate of O((log T)/T). This addresses a question of Daskalakis et al 2011. Further, we consider a partial information version of the problem. We then apply the results to convex programming and exhibit a simple algorithm for the approximate Max Flow problem
    • …
    corecore