Conic Optimization Theory: Convexification Techniques and Numerical Algorithms

Optimization is at the core of control theory and appears in several areas of this field, such as optimal control, distributed control, system identification, robust control, state estimation, model predictive control and dynamic programming. The recent advances in various topics of modern optimization have also been revamping the area of machine learning. Motivated by the crucial role of optimization theory in the design, analysis, control and operation of real-world systems, this tutorial paper offers a detailed overview of some major advances in this area, namely conic optimization and its emerging applications. First, we discuss the importance of conic optimization in different areas. Then, we explain seminal results on the design of hierarchies of convex relaxations for a wide range of nonconvex problems. Finally, we study different numerical algorithms for large-scale conic optimization problems.Comment: 18 page

AC OPF in Radial Distribution Networks - Parts I,II

The optimal power-flow problem (OPF) has played a key role in the planning and operation of power systems. Due to the non-linear nature of the AC power-flow equations, the OPF problem is known to be non-convex, therefore hard to solve. Most proposed methods for solving the OPF rely on approximations that render the problem convex, but that may yield inexact solutions. Recently, Farivar and Low proposed a method that is claimed to be exact for radial distribution systems, despite no apparent approximations. In our work, we show that it is, in fact, not exact. On one hand, there is a misinterpretation of the physical network model related to the ampacity constraint of the lines' current flows. On the other hand, the proof of the exactness of the proposed relaxation requires unrealistic assumptions related to the unboundedness of specific control variables. We also show that the extension of this approach to account for exact line models might provide physically infeasible solutions. Recently, several contributions have proposed OPF algorithms that rely on the use of the alternating-direction method of multipliers (ADMM). However, as we show in this work, there are cases for which the ADMM-based solution of the non-relaxed OPF problem fails to converge. To overcome the aforementioned limitations, we propose an algorithm for the solution of a non-approximated, non-convex OPF problem in radial distribution systems that is based on the method of multipliers, and on a primal decomposition of the OPF. This work is divided in two parts. In Part I, we specifically discuss the limitations of BFM and ADMM to solve the OPF problem. In Part II, we provide a centralized version and a distributed asynchronous version of the proposed OPF algorithm and we evaluate its performances using both small-scale electrical networks, as well as a modified IEEE 13-node test feeder

Adaptive Normalized Risk-Averting Training For Deep Neural Networks

This paper proposes a set of new error criteria and learning approaches, Adaptive Normalized Risk-Averting Training (ANRAT), to attack the non-convex optimization problem in training deep neural networks (DNNs). Theoretically, we demonstrate its effectiveness on global and local convexity lower-bounded by the standard $L_p$-norm error. By analyzing the gradient on the convexity index $\lambda$, we explain the reason why to learn $\lambda$ adaptively using gradient descent works. In practice, we show how this method improves training of deep neural networks to solve visual recognition tasks on the MNIST and CIFAR-10 datasets. Without using pretraining or other tricks, we obtain results comparable or superior to those reported in recent literature on the same tasks using standard ConvNets + MSE/cross entropy. Performance on deep/shallow multilayer perceptrons and Denoised Auto-encoders is also explored. ANRAT can be combined with other quasi-Newton training methods, innovative network variants, regularization techniques and other specific tricks in DNNs. Other than unsupervised pretraining, it provides a new perspective to address the non-convex optimization problem in DNNs.Comment: AAAI 2016, 0.39%~0.4% ER on MNIST with single 32-32-256-10 ConvNets, code available at https://github.com/cauchyturing/ANRA

Optimal Reactive Power Dispatch Formulated as Quadratic OPF and Solved via CS-SLP

Increased penetration of inverter interfaced renewable energy resources creates challenges and opportunities for reactive power management in the modern electricity grid. Because of the multiplicity of new resources, new computational tools and optimization models are needed in formulating and solving the Optimal Reactive Power Dispatch Problem (ORPD). In this paper, we propose (1) an object-oriented ORPD formulation based on high-fidelity modeling of each device in the network, especially those with VAR/V control capability and (2) a two-step Convex Solution-Sequential Linear Programming algorithm. The proposed method introduces two innovations: (a) high fidelity quadratized models of each component of the power system with emphasis on those components that have VAR/V control capability; and (b) an object oriented convexification of the resulting quadratic OPF problem; the solution is obtained by first solving the convex problem using public solvers for convex problems and them removing the relaxation and solving the original OPF using SLP, starting from the solution of the relaxed (convex) problem

Distributed Online Modified Greedy Algorithm for Networked Storage Operation under Uncertainty

The integration of intermittent and stochastic renewable energy resources requires increased flexibility in the operation of the electric grid. Storage, broadly speaking, provides the flexibility of shifting energy over time; network, on the other hand, provides the flexibility of shifting energy over geographical locations. The optimal control of storage networks in stochastic environments is an important open problem. The key challenge is that, even in small networks, the corresponding constrained stochastic control problems on continuous spaces suffer from curses of dimensionality, and are intractable in general settings. For large networks, no efficient algorithm is known to give optimal or provably near-optimal performance for this problem. This paper provides an efficient algorithm to solve this problem with performance guarantees. We study the operation of storage networks, i.e., a storage system interconnected via a power network. An online algorithm, termed Online Modified Greedy algorithm, is developed for the corresponding constrained stochastic control problem. A sub-optimality bound for the algorithm is derived, and a semidefinite program is constructed to minimize the bound. In many cases, the bound approaches zero so that the algorithm is near-optimal. A task-based distributed implementation of the online algorithm relying only on local information and neighbor communication is then developed based on the alternating direction method of multipliers. Numerical examples verify the established theoretical performance bounds, and demonstrate the scalability of the algorithm.Comment: arXiv admin note: text overlap with arXiv:1405.778

Matrix Minor Reformulation and SOCP-based Spatial Branch-and-Cut Method for the AC Optimal Power Flow Problem

Alternating current optimal power flow (AC OPF) is one of the most fundamental optimization problems in electrical power systems. It can be formulated as a semidefinite program (SDP) with rank constraints. Solving AC OPF, that is, obtaining near optimal primal solutions as well as high quality dual bounds for this non-convex program, presents a major computational challenge to today's power industry for the real-time operation of large-scale power grids. In this paper, we propose a new technique for reformulation of the rank constraints using both principal and non-principal 2-by-2 minors of the involved Hermitian matrix variable and characterize all such minors into three types. We show the equivalence of these minor constraints to the physical constraints of voltage angle differences summing to zero over three- and four-cycles in the power network. We study second-order conic programming (SOCP) relaxations of this minor reformulation and propose strong cutting planes, convex envelopes, and bound tightening techniques to strengthen the resulting SOCP relaxations. We then propose an SOCP-based spatial branch-and-cut method to obtain the global optimum of AC OPF. Extensive computational experiments show that the proposed algorithm significantly outperforms the state-of-the-art SDP-based OPF solver and on a simple personal computer is able to obtain on average a 0.71% optimality gap in no more than 720 seconds for the most challenging power system instances in the literature

Generalized Linear Programming Solves the Dual

The generalized linear programming algorithm allows an arbitrary mathematical programming minimization problem to be analyzed as a sequence of linear programming approximations. Under fairly general assumptions, it is demonstrated that any limit point of the sequence of optimal linear programming dual prices produced by the algorithm is optimal in a concave maximization problem that is dual to the arbitrary primal problem. This result holds even if the generalized linear programming problem does not solve the primal problem. The result is a consequence of the equivalence that exists between the operations of convexification and dualization of a primal problem. The exact mathematical nature of this equivalence is given.Supported in prt by the U.S. Army Research Office (Durham) under contract DAHC04-73-C-0032