The Computational Power of Optimization in Online Learning

We consider the fundamental problem of prediction with expert advice where the experts are "optimizable": there is a black-box optimization oracle that can be used to compute, in constant time, the leading expert in retrospect at any point in time. In this setting, we give a novel online algorithm that attains vanishing regret with respect to $N$ experts in total $\widetilde{O}(\sqrt{N})$ computation time. We also give a lower bound showing that this running time cannot be improved (up to log factors) in the oracle model, thereby exhibiting a quadratic speedup as compared to the standard, oracle-free setting where the required time for vanishing regret is $\widetilde{\Theta}(N)$. These results demonstrate an exponential gap between the power of optimization in online learning and its power in statistical learning: in the latter, an optimization oracle---i.e., an efficient empirical risk minimizer---allows to learn a finite hypothesis class of size $N$ in time $O(\log{N})$. We also study the implications of our results to learning in repeated zero-sum games, in a setting where the players have access to oracles that compute, in constant time, their best-response to any mixed strategy of their opponent. We show that the runtime required for approximating the minimax value of the game in this setting is $\widetilde{\Theta}(\sqrt{N})$, yielding again a quadratic improvement upon the oracle-free setting, where $\widetilde{\Theta}(N)$ is known to be tight

Improved Learning-Augmented Algorithms for the Multi-Option Ski Rental Problem via Best-Possible Competitive Analysis

In this paper, we present improved learning-augmented algorithms for the multi-option ski rental problem. Learning-augmented algorithms take ML predictions as an added part of the input and incorporates these predictions in solving the given problem. Due to their unique strength that combines the power of ML predictions with rigorous performance guarantees, they have been extensively studied in the context of online optimization problems. Even though ski rental problems are one of the canonical problems in the field of online optimization, only deterministic algorithms were previously known for multi-option ski rental, with or without learning augmentation. We present the first randomized learning-augmented algorithm for this problem, surpassing previous performance guarantees given by deterministic algorithms. Our learning-augmented algorithm is based on a new, provably best-possible randomized competitive algorithm for the problem. Our results are further complemented by lower bounds for deterministic and randomized algorithms, and computational experiments evaluating our algorithms' performance improvements.Comment: 23 pages, 1 figur

Online Trajectory Planning Through Combined Trajectory Optimization and Function Approximation: Application to the Exoskeleton Atalante

Autonomous robots require online trajectory planning capability to operate in the real world. Efficient offline trajectory planning methods already exist, but are computationally demanding, preventing their use online. In this paper, we present a novel algorithm called Guided Trajectory Learning that learns a function approximation of solutions computed through trajectory optimization while ensuring accurate and reliable predictions. This function approximation is then used online to generate trajectories. This algorithm is designed to be easy to implement, and practical since it does not require massive computing power. It is readily applicable to any robotics systems and effortless to set up on real hardware since robust control strategies are usually already available. We demonstrate the computational performance of our algorithm on flat-foot walking with the self-balanced exoskeleton Atalante

Online security assessment with load and renewable generation uncertainty: The iTesla project approach

The secure integration of renewable generation into modern power systems requires an appropriate assessment of the security of the system in real-time. The uncertainty associated with renewable power makes it impossible to tackle this problem via a brute-force approach, i.e. it is not possible to run detailed online static or dynamic simulations for all possible security problems and realizations of load and renewable power. Intelligent approaches for online security assessment with forecast uncertainty modeling are being sought to better handle contingency events. This paper reports the platform developed within the iTesla project for online static and dynamic security assessment. This innovative and open-source computational platform is composed of several modules such as detailed static and dynamic simulation, machine learning, forecast uncertainty representation and optimization tools to not only filter contingencies but also to provide the best control actions to avoid possible unsecure situations. Based on High Performance Computing (HPC), the iTesla platform was tested in the French network for a specific security problem: overload of transmission circuits. The results obtained show that forecast uncertainty representation is of the utmost importance, since from apparently secure forecast network states, it is possible to obtain unsecure situations that need to be tackled in advance by the system operator

Learning-based Optimization for Signal and Image Processing

Incorporating machine learning techniques into optimization problems and solvers attracts increasing attention. Given a particular type of optimization problem that needs to be solved repeatedly, machine learning techniques can find some features for this category of optimization and develop algorithms with excellent performance. This thesis deals with algorithms and convergence analysis in learning-based optimization in three aspects: learning dictionaries, learning optimization solvers and learning regularizers.Learning dictionaries for sparse coding is significant for signal processing. Convolutional sparse coding is a form of sparse coding with a structured, translation invariant dictionary. Most convolutional dictionary learning algorithms to date operate in the batch mode, requiring simultaneous access to all training images during the learning process, which results in very high memory usage, and severely limits the training data size that can be used. I proposed two online convolutional dictionary learning algorithms that offered far better scaling of memory and computational cost than batch methods and provided a rigorous theoretical analysis of these methods.Learning fast solvers for optimization is a rising research topic. In recent years, unfolding iterative algorithms as neural networks has become an empirical success in solving sparse recovery problems. However, its theoretical understanding is still immature, which prevents us from fully utilizing the power of neural networks. I studied unfolded ISTA (Iterative Shrinkage Thresholding Algorithm) for sparse signal recovery and established its convergence. Based on the properties of parameters required by convergence, the model can be significantly simplified and, consequently, has much less training cost and better recovery performance.Learning regularizers or priors improves the performance of optimization solvers, especially for signal and image processing tasks. Plug-and-play (PnP) is a non-convex framework that integrates modern priors, such as BM3D or deep learning-based denoisers, into ADMM or other proximal algorithms. Although PnP has been recently studied extensively with great empirical success, theoretical analysis addressing even the most basic question of convergence has been insufficient. In this thesis, the theoretical convergence of PnP-FBS and PnP-ADMM was established, without using diminishing stepsizes, under a certain Lipschitz condition on the denoisers. Furthermore, real spectral normalization was proposed for training deep learning-based denoisers to satisfy the proposed Lipschitz condition

Real-time power system dispatch scheme using grid expert strategy-based imitation learning

With large-scale grid integration of renewable energy sources (RES), power grid operations gradually exhibit the new characteristics of high-order uncertainty, leading to significant challenges for system operational security. Traditional model-driven generation dispatch methods require large computational resources, whereas the widely concerned Reinforcement Learning (RL)-based methods lead to issues such as slow training speed due to the high complexity and dimension of processed grid state information. For this reason, this paper proposes a novel Grid Expert Strategy Imitation Learning (GESIL)-based real-time (5 min intervals in this paper) dispatch method. Firstly, a grid model is established based on the graph theory. Secondly, a pure rule-based grid expert strategy (GES) considering detailed power grid operations is proposed. Then, the GES is combined with the established model to obtain a GESIL agent using imitation learning by offline–online training, which can produce specific grid dispatch decisions for real-time. By designing a graph theory-based grid model, a model-driven purely rule-based GES, and embedding a penalty factor-based loss function into IL offline–online training, GESIL ultimately achieves high training speed, high solution speed, and strong generalization capability. A modified IEEE 118-node system is employed to compare the proposed GESIL to traditional dispatch method and RL method. Results show that GESIL has significantly improved computational efficiency by approximately 17 times and training speed by 14.5 times. GESIL can more stably and efficiently compute real-time dispatch decisions of grid operations, enhancing the optimization effect in terms of transmission overloading mitigation, transmission loading optimization, and power balancing control