On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergence

We introduce a framework for quasi-Newton forward--backward splitting algorithms (proximal quasi-Newton methods) with a metric induced by diagonal $\pm$ rank-$r$ symmetric positive definite matrices. This special type of metric allows for a highly efficient evaluation of the proximal mapping. The key to this efficiency is a general proximal calculus in the new metric. By using duality, formulas are derived that relate the proximal mapping in a rank-$r$ modified metric to the original metric. We also describe efficient implementations of the proximity calculation for a large class of functions; the implementations exploit the piece-wise linear nature of the dual problem. Then, we apply these results to acceleration of composite convex minimization problems, which leads to elegant quasi-Newton methods for which we prove convergence. The algorithm is tested on several numerical examples and compared to a comprehensive list of alternatives in the literature. Our quasi-Newton splitting algorithm with the prescribed metric compares favorably against state-of-the-art. The algorithm has extensive applications including signal processing, sparse recovery, machine learning and classification to name a few.Comment: arXiv admin note: text overlap with arXiv:1206.115

A differential analysis of the power flow equations

The AC power flow equations are fundamental in all aspects of power systems planning and operations. They are routinely solved using Newton-Raphson like methods. However, there is little theoretical understanding of when these algorithms are guaranteed to find a solution of the power flow equations or how long they may take to converge. Further, it is known that in general these equations have multiple solutions and can exhibit chaotic behavior. In this paper, we show that the power flow equations can be solved efficiently provided that the solution lies in a certain set. We introduce a family of convex domains, characterized by Linear Matrix Inequalities, in the space of voltages such that there is at most one power flow solution in each of these domains. Further, if a solution exists in one of these domains, it can be found efficiently, and if one does not exist, a certificate of non-existence can also be obtained efficiently. The approach is based on the theory of monotone operators and related algorithms for solving variational inequalities involving monotone operators. We validate our approach on IEEE test networks and show that practical power flow solutions lie within an appropriately chosen convex domain.Comment: arXiv admin note: text overlap with arXiv:1506.0847

Conic Optimization Theory: Convexification Techniques and Numerical Algorithms

Optimization is at the core of control theory and appears in several areas of this field, such as optimal control, distributed control, system identification, robust control, state estimation, model predictive control and dynamic programming. The recent advances in various topics of modern optimization have also been revamping the area of machine learning. Motivated by the crucial role of optimization theory in the design, analysis, control and operation of real-world systems, this tutorial paper offers a detailed overview of some major advances in this area, namely conic optimization and its emerging applications. First, we discuss the importance of conic optimization in different areas. Then, we explain seminal results on the design of hierarchies of convex relaxations for a wide range of nonconvex problems. Finally, we study different numerical algorithms for large-scale conic optimization problems.Comment: 18 page