A robust primal-dual interior-point algorithm for nonlinear programs

10.1137/S1052623402400641SIAM Journal on Optimization1441163-118

OSQP: An Operator Splitting Solver for Quadratic Programs

We present a general-purpose solver for convex quadratic programs based on the alternating direction method of multipliers, employing a novel operator splitting technique that requires the solution of a quasi-definite linear system with the same coefficient matrix at almost every iteration. Our algorithm is very robust, placing no requirements on the problem data such as positive definiteness of the objective function or linear independence of the constraint functions. It can be configured to be division-free once an initial matrix factorization is carried out, making it suitable for real-time applications in embedded systems. In addition, our technique is the first operator splitting method for quadratic programs able to reliably detect primal and dual infeasible problems from the algorithm iterates. The method also supports factorization caching and warm starting, making it particularly efficient when solving parametrized problems arising in finance, control, and machine learning. Our open-source C implementation OSQP has a small footprint, is library-free, and has been extensively tested on many problem instances from a wide variety of application areas. It is typically ten times faster than competing interior-point methods, and sometimes much more when factorization caching or warm start is used. OSQP has already shown a large impact with tens of thousands of users both in academia and in large corporations

Conic Optimization Theory: Convexification Techniques and Numerical Algorithms

Optimization is at the core of control theory and appears in several areas of this field, such as optimal control, distributed control, system identification, robust control, state estimation, model predictive control and dynamic programming. The recent advances in various topics of modern optimization have also been revamping the area of machine learning. Motivated by the crucial role of optimization theory in the design, analysis, control and operation of real-world systems, this tutorial paper offers a detailed overview of some major advances in this area, namely conic optimization and its emerging applications. First, we discuss the importance of conic optimization in different areas. Then, we explain seminal results on the design of hierarchies of convex relaxations for a wide range of nonconvex problems. Finally, we study different numerical algorithms for large-scale conic optimization problems.Comment: 18 page

A Primal-Dual Augmented Lagrangian Penalty-Interior-Point Algorithm for Nonlinear Programming

This thesis treats a new numerical solution method for large-scale nonlinear optimization problems. Nonlinear programs occur in a wide range of engineering and academic applications like discretized optimal control processes and parameter identification of physical systems. The most efficient and robust solution approaches for this problem class have been shown to be sequential quadratic programming and primal-dual interior-point methods. The proposed algorithm combines a variant of the latter with a special penalty function to increase its robustness due to an automatic regularization of the nonlinear constraints caused by the penalty term. In detail, a modified barrier function and a primal-dual augmented Lagrangian approach with an exact l2-penalty is used. Both share the property that for certain Lagrangian multiplier estimates the barrier and penalty parameter do not have to converge to zero or diverge, respectively. This improves the conditioning of the internal linear equation systems near the optimal solution, handles rank-deficiency of the constraint derivatives for all non-feasible iterates and helps with identifying infeasible problem formulations. Although the resulting merit function is non-smooth, a certain step direction is a guaranteed descent. The algorithm includes an adaptive update strategy for the barrier and penalty parameters as well as the Lagrangian multiplier estimates based on a sensitivity analysis. Global convergence is proven to yield a first-order optimal solution, a certificate of infeasibility or a Fritz-John point and is maintained by combining the merit function with a filter or piecewise linear penalty function. Unlike the majority of filter methods, no separate feasibility restoration phase is required. For a fixed barrier parameter the method has a quadratic order of convergence. Furthermore, a sensitivity based iterative refinement strategy is developed to approximate the optimal solution of a parameter dependent nonlinear program under parameter changes. It exploits special sensitivity derivative approximations and converges locally with a linear convergence order to a feasible point that further satisfies the perturbed complementarity condition of the modified barrier method. Thereby, active-set changes from active to inactive can be handled. Due to a certain update of the Lagrangian multiplier estimate, the refinement is suitable in the context of warmstarting the penalty-interior-point approach. A special focus of the thesis is the development of an algorithm with excellent performance in practice. Details on an implementation of the proposed primal-dual penalty-interior-point algorithm in the nonlinear programming solver WORHP and a numerical study based on the CUTEst test collection is provided. The efficiency and robustness of the algorithm is further compared to state-of-the-art nonlinear programming solvers, in particular the interior-point solvers IPOPT and KNITRO as well as the sequential quadratic programming solvers SNOPT and WORHP

A sequential semidefinite programming method and an application in passive reduced-order modeling

We consider the solution of nonlinear programs with nonlinear semidefiniteness constraints. The need for an efficient exploitation of the cone of positive semidefinite matrices makes the solution of such nonlinear semidefinite programs more complicated than the solution of standard nonlinear programs. In particular, a suitable symmetrization procedure needs to be chosen for the linearization of the complementarity condition. The choice of the symmetrization procedure can be shifted in a very natural way to certain linear semidefinite subproblems, and can thus be reduced to a well-studied problem. The resulting sequential semidefinite programming (SSP) method is a generalization of the well-known SQP method for standard nonlinear programs. We present a sensitivity result for nonlinear semidefinite programs, and then based on this result, we give a self-contained proof of local quadratic convergence of the SSP method. We also describe a class of nonlinear semidefinite programs that arise in passive reduced-order modeling, and we report results of some numerical experiments with the SSP method applied to problems in that class