On implementation of a self-dual embedding method for convex programming.

by Cheng Tak Wai, Johnny.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 59-62).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 2 --- Background --- p.7Chapter 2.1 --- Self-dual embedding --- p.7Chapter 2.2 --- Conic optimization --- p.8Chapter 2.3 --- Self-dual embedded conic optimization --- p.9Chapter 2.4 --- Connection with convex programming --- p.11Chapter 2.5 --- Chapter summary --- p.15Chapter 3 --- Implementation of the algorithm --- p.17Chapter 3.1 --- The new search direction --- p.17Chapter 3.2 --- Select the step-length --- p.23Chapter 3.3 --- The multi-constraint case --- p.25Chapter 3.4 --- Chapter summary --- p.32Chapter 4 --- Numerical results on randomly generated problem --- p.34Chapter 4.1 --- Single-constraint problems --- p.35Chapter 4.2 --- Multi-constraint problems --- p.36Chapter 4.3 --- Running time and the size of the problem --- p.39Chapter 4.4 --- Chapter summary --- p.42Chapter 5 --- Geometric optimization --- p.45Chapter 5.1 --- Geometric programming --- p.45Chapter 5.1.1 --- Monomials and posynomials --- p.45Chapter 5.1.2 --- Geometric programming --- p.46Chapter 5.1.3 --- Geometric program in convex form --- p.47Chapter 5.2 --- Conic transformation --- p.48Chapter 5.3 --- Computational results of geometric optimization problem --- p.50Chapter 5.4 --- Chapter summary --- p.55Chapter 6 --- Conclusion --- p.5

Conic Descent Redux for Memory-Efficient Optimization

Conic programming has well-documented merits in a gamut of signal processing and machine learning tasks. This contribution revisits a recently developed first-order conic descent (CD) solver, and advances it in three aspects: intuition, theory, and algorithmic implementation. It is found that CD can afford an intuitive geometric derivation that originates from the dual problem. This opens the door to novel algorithmic designs, with a momentum variant of CD, momentum conic descent (MOCO) exemplified. Diving deeper into the dual behavior CD and MOCO reveals: i) an analytically justified stopping criterion; and, ii) the potential to design preconditioners to speed up dual convergence. Lastly, to scale semidefinite programming (SDP) especially for low-rank solutions, a memory efficient MOCO variant is developed and numerically validated

Extended Formulations in Mixed-integer Convex Programming

We present a unifying framework for generating extended formulations for the polyhedral outer approximations used in algorithms for mixed-integer convex programming (MICP). Extended formulations lead to fewer iterations of outer approximation algorithms and generally faster solution times. First, we observe that all MICP instances from the MINLPLIB2 benchmark library are conic representable with standard symmetric and nonsymmetric cones. Conic reformulations are shown to be effective extended formulations themselves because they encode separability structure. For mixed-integer conic-representable problems, we provide the first outer approximation algorithm with finite-time convergence guarantees, opening a path for the use of conic solvers for continuous relaxations. We then connect the popular modeling framework of disciplined convex programming (DCP) to the existence of extended formulations independent of conic representability. We present evidence that our approach can yield significant gains in practice, with the solution of a number of open instances from the MINLPLIB2 benchmark library.Comment: To be presented at IPCO 201

A Tensor Analogy of Yuan's Theorem of the Alternative and Polynomial Optimization with Sign structure

Yuan's theorem of the alternative is an important theoretical tool in optimization, which provides a checkable certificate for the infeasibility of a strict inequality system involving two homogeneous quadratic functions. In this paper, we provide a tractable extension of Yuan's theorem of the alternative to the symmetric tensor setting. As an application, we establish that the optimal value of a class of nonconvex polynomial optimization problems with suitable sign structure (or more explicitly, with essentially non-positive coefficients) can be computed by a related convex conic programming problem, and the optimal solution of these nonconvex polynomial optimization problems can be recovered from the corresponding solution of the convex conic programming problem. Moreover, we obtain that this class of nonconvex polynomial optimization problems enjoy exact sum-of-squares relaxation, and so, can be solved via a single semidefinite programming problem.Comment: acceted by Journal of Optimization Theory and its application, UNSW preprint, 22 page

Strong duality in conic linear programming: facial reduction and extended duals

The facial reduction algorithm of Borwein and Wolkowicz and the extended dual of Ramana provide a strong dual for the conic linear program $$(P) \sup {<c, x> | Ax \leq_K b}$$ in the absence of any constraint qualification. The facial reduction algorithm solves a sequence of auxiliary optimization problems to obtain such a dual. Ramana's dual is applicable when (P) is a semidefinite program (SDP) and is an explicit SDP itself. Ramana, Tuncel, and Wolkowicz showed that these approaches are closely related; in particular, they proved the correctness of Ramana's dual using certificates from a facial reduction algorithm. Here we give a clear and self-contained exposition of facial reduction, of extended duals, and generalize Ramana's dual: -- we state a simple facial reduction algorithm and prove its correctness; and -- building on this algorithm we construct a family of extended duals when $K$ is a {\em nice} cone. This class of cones includes the semidefinite cone and other important cones.Comment: A previous version of this paper appeared as "A simple derivation of a facial reduction algorithm and extended dual systems", technical report, Columbia University, 2000, available from http://www.unc.edu/~pataki/papers/fr.pdf Jonfest, a conference in honor of Jonathan Borwein's 60th birthday, 201

Computational Complexity versus Statistical Performance on Sparse Recovery Problems

We show that several classical quantities controlling compressed sensing performance directly match classical parameters controlling algorithmic complexity. We first describe linearly convergent restart schemes on first-order methods solving a broad range of compressed sensing problems, where sharpness at the optimum controls convergence speed. We show that for sparse recovery problems, this sharpness can be written as a condition number, given by the ratio between true signal sparsity and the largest signal size that can be recovered by the observation matrix. In a similar vein, Renegar's condition number is a data-driven complexity measure for convex programs, generalizing classical condition numbers for linear systems. We show that for a broad class of compressed sensing problems, the worst case value of this algorithmic complexity measure taken over all signals matches the restricted singular value of the observation matrix which controls robust recovery performance. Overall, this means in both cases that, in compressed sensing problems, a single parameter directly controls both computational complexity and recovery performance. Numerical experiments illustrate these points using several classical algorithms.Comment: Final version, to appear in information and Inferenc