MAGMA: Multi-level accelerated gradient mirror descent algorithm for large-scale convex composite minimization

Composite convex optimization models arise in several applications, and are especially prevalent in inverse problems with a sparsity inducing norm and in general convex optimization with simple constraints. The most widely used algorithms for convex composite models are accelerated first order methods, however they can take a large number of iterations to compute an acceptable solution for large-scale problems. In this paper we propose to speed up first order methods by taking advantage of the structure present in many applications and in image processing in particular. Our method is based on multi-level optimization methods and exploits the fact that many applications that give rise to large scale models can be modelled using varying degrees of fidelity. We use Nesterov's acceleration techniques together with the multi-level approach to achieve $\mathcal{O}(1/\sqrt{\epsilon})$ convergence rate, where $\epsilon$ denotes the desired accuracy. The proposed method has a better convergence rate than any other existing multi-level method for convex problems, and in addition has the same rate as accelerated methods, which is known to be optimal for first-order methods. Moreover, as our numerical experiments show, on large-scale face recognition problems our algorithm is several times faster than the state of the art

Approximation of System Components for Pump Scheduling Optimisation

© 2015 The Authors. Published by Elsevier Ltd.The operation of pump systems in water distribution systems (WDS) is commonly the most expensive task for utilities with up to 70% of the operating cost of a pump system attributed to electricity consumption. Optimisation of pump scheduling could save 10-20% by improving efficiency or shifting consumption to periods with low tariffs. Due to the complexity of the optimal control problem, heuristic methods which cannot guarantee optimality are often applied. To facilitate the use of mathematical optimisation this paper investigates formulations of WDS components. We show that linear approximations outperform non-linear approximations, while maintaining comparable levels of accuracy

Bounding Option Prices Using SDP With Change Of Numeraire

Recently, given the first few moments, tight upper and lower bounds of the no arbitrage prices can be obtained by solving semidefinite programming (SDP) or linear programming (LP) problems. In this paper, we compare SDP and LP formulations of the European-style options pricing problem and prefer SDP formulations due to the simplicity of moments constraints. We propose to employ the technique of change of numeraire when using SDP to bound the European type of options. In fact, this problem can then be cast as a truncated Hausdorff moment problem which has necessary and sufficient moment conditions expressed by positive semidefinite moment and localizing matrices. With four moments information we show stable numerical results for bounding European call options and exchange options. Moreover, A hedging strategy is also identified by the dual formulation.moments of measures, semidefinite programming, linear programming, options pricing, change of numeraire

Mean Variance Optimization of Non-Linear Systems and Worst-case Analysis

In this paper, we consider expected value, variance and worst-case optimization of nonlinear models. We present algorithms for computing optimal expected values, and variance, based on iterative Taylor expansions. We establish convergence and consider the relative merits of policies beaded on expected value optimization and worst-case robustness. The latter is a minimax strategy and ensures optimal cover in view of the worst-case scenario(s) while the former is optimal expected performance in a stochastic setting. Both approaches are used with a macroeconomic policy model to illustrate relative performances, robustness and trade-offs between the strategies.

Mean variance optimization of non-linear systems and worst-case analysis

In this paper, we consider expected value, variance and worst-case optimization of nonlinear models. We present algorithms for computing optimal expected values, and variance, based on iterative Taylor expansions. We establish convergence and consider the relative merits of policies beaded on expected value optimization and worst-case robustness. The latter is a minimax strategy and ensures optimal cover in view of the worst-case scenario(s) while the former is optimal expected performance in a stochastic setting. Both approaches are used with a macroeconomic policy model to illustrate relative performances, robustness and trade-offs between the strategies. Klassifikation: C61, E4

A Multilevel Method for Self-Concordant Minimization

The analysis of second-order optimization methods based either on sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates is not scale-invariant, and even if it is, restrictive assumptions are required on the problem structure. The second shortfall is that the fast convergence rates of second-order methods have only been established by making assumptions regarding the input data. In this paper, we close the theoretical gap between the theoretical analysis of the conventional Newton method and randomization-based second-order methods. We propose a Self-concordant Iterative-minimization - Galerkin-based Multilevel Algorithm (SIGMA) and establish its super-linear convergence rate using the well-established theory of self-concordant functions. Our analysis is global and entirely independent of unknown constants such as Lipschitz constants and strong convexity parameters. Our analysis is based on the connections between multigrid optimization methods, and the role of coarse-grained or reduced-order models in the computation of search directions. We take advantage of the insights from the analysis to significantly improve the performance of second-order methods in machine learning applications. We report encouraging initial experiments that suggest SIGMA significantly outperforms the state-of-the-art sub-sampled/sketched Newton methods for both medium and large-scale problems

Demonstrating demand response from water distribution system through pump scheduling

Significant changes in the power generation mix are posing new challenges for the balancing systems of the grid. Many of these challenges are in the secondary electricity grid regulation services and could be met through demand response (DR) services. We explore the opportunities for a water distribution system (WDS) to provide balancing services with demand response through pump scheduling and evaluate the associated benefits. Using a benchmark network and demand response mechanisms available in the UK, these benefits are assessed in terms of reduced green house gas (GHG) emissions from the grid due to the displacement of more polluting power sources and additional revenues for water utilities. The optimal pump scheduling problem is formulated as a mixed-integer optimisation problem and solved using a branch and bound algorithm. This new formulation finds the optimal level of power capacity to commit to the provision of demand response for a range of reserve energy provision and frequency response schemes offered in the UK. For the first time we show that DR from WDS can offer financial benefits to WDS operators while providing response energy to the grid with less greenhouse gas emissions than competing reserve energy technologies. Using a Monte Carlo simulation based on data from 2014, we demonstrate that the cost of providing the storage energy is less than the financial compensation available for the equivalent energy supply. The GHG emissions from the demand response provision from a WDS are also shown to be smaller than those of contemporary competing technologies such as open cycle gas turbines. The demand response services considered vary in their response time and duration as well as commitment requirements. The financial viability of a demand response service committed continuously is shown to be strongly dependent on the utilisation of the pumps and the electricity tariffs used by water utilities. Through the analysis of range of water demand scenarios and financial incentives using real market data, we demonstrate how a WDS can participate in a demand response scheme and generate financial gains and environmental benefits

A stochastic minimum principle and an adaptive pathwise algorithm for stochastic optimal control

We present a numerical method for finite-horizon stochastic optimal control models. We derive a stochastic minimum principle (SMP) and then develop a numerical method based on the direct solution of the SMP. The method combines Monte Carlo pathwise simulation and non-parametric interpolation methods. We present results from a standard linear quadratic control model, and a realistic case study that captures the stochastic dynamics of intermittent power generation in the context of optimal economic dispatch models.National Science Foundation (U.S.) (Grant 1128147)United States. Dept. of Energy. Office of Science (Biological and Environmental Research Program Grant DE-SC0005171)United States. Dept. of Energy. Office of Science (Biological and Environmental Research Program Grant DE-SC0003906

Decomposition-Based Method for Sparse Semidefinite Relaxations of Polynomial Optimization Problems

We consider polynomial optimization problems pervaded by a sparsity pattern. It has been shown in [1, 2] that the optimal solution of a polynomial programming problem with structured sparsity can be computed by solving a series of semidefinite relaxations that possess the same kind of sparsity. We aim at solving the former relaxations with a decompositionbased method, which partitions the relaxations according to their sparsity pattern. The decomposition-based method that we propose is an extension to semidefinite programming of the Benders decomposition for linear programs [3] .Polynomial optimization, Semidefinite programming, Sparse SDP relaxations, Benders decomposition