Convex optimization over intersection of simple sets: improved convergence rate guarantees via an exact penalty approach

We consider the problem of minimizing a convex function over the intersection of finitely many simple sets which are easy to project onto. This is an important problem arising in various domains such as machine learning. The main difficulty lies in finding the projection of a point in the intersection of many sets. Existing approaches yield an infeasible point with an iteration-complexity of $O(1/\varepsilon^2)$ for nonsmooth problems with no guarantees on the in-feasibility. By reformulating the problem through exact penalty functions, we derive first-order algorithms which not only guarantees that the distance to the intersection is small but also improve the complexity to $O(1/\varepsilon)$ and $O(1/\sqrt{\varepsilon})$ for smooth functions. For composite and smooth problems, this is achieved through a saddle-point reformulation where the proximal operators required by the primal-dual algorithms can be computed in closed form. We illustrate the benefits of our approach on a graph transduction problem and on graph matching

Constraint Qualifications and Optimality Conditions for Nonconvex Semi-Infinite and Infinite Programs

The paper concerns the study of new classes of nonlinear and nonconvex optimization problems of the so-called infinite programming that are generally defined on infinite-dimensional spaces of decision variables and contain infinitely many of equality and inequality constraints with arbitrary (may not be compact) index sets. These problems reduce to semi-infinite programs in the case of finite-dimensional spaces of decision variables. We extend the classical Mangasarian-Fromovitz and Farkas-Minkowski constraint qualifications to such infinite and semi-infinite programs. The new qualification conditions are used for efficient computing the appropriate normal cones to sets of feasible solutions for these programs by employing advanced tools of variational analysis and generalized differentiation. In the further development we derive first-order necessary optimality conditions for infinite and semi-infinite programs, which are new in both finite-dimensional and infinite-dimensional settings.Comment: 28 page

Strong duality in conic linear programming: facial reduction and extended duals

The facial reduction algorithm of Borwein and Wolkowicz and the extended dual of Ramana provide a strong dual for the conic linear program $$(P) \sup {<c, x> | Ax \leq_K b}$$ in the absence of any constraint qualification. The facial reduction algorithm solves a sequence of auxiliary optimization problems to obtain such a dual. Ramana's dual is applicable when (P) is a semidefinite program (SDP) and is an explicit SDP itself. Ramana, Tuncel, and Wolkowicz showed that these approaches are closely related; in particular, they proved the correctness of Ramana's dual using certificates from a facial reduction algorithm. Here we give a clear and self-contained exposition of facial reduction, of extended duals, and generalize Ramana's dual: -- we state a simple facial reduction algorithm and prove its correctness; and -- building on this algorithm we construct a family of extended duals when $K$ is a {\em nice} cone. This class of cones includes the semidefinite cone and other important cones.Comment: A previous version of this paper appeared as "A simple derivation of a facial reduction algorithm and extended dual systems", technical report, Columbia University, 2000, available from http://www.unc.edu/~pataki/papers/fr.pdf Jonfest, a conference in honor of Jonathan Borwein's 60th birthday, 201

Tangential Extremal Principles for Finite and Infinite Systems of Sets, II: Applications to Semi-infinite and Multiobjective Optimization

This paper contains selected applications of the new tangential extremal principles and related results developed in Part I to calculus rules for infinite intersections of sets and optimality conditions for problems of semi-infinite programming and multiobjective optimization with countable constraint