Convex optimization over intersection of simple sets: improved convergence rate guarantees via an exact penalty approach

We consider the problem of minimizing a convex function over the intersection of finitely many simple sets which are easy to project onto. This is an important problem arising in various domains such as machine learning. The main difficulty lies in finding the projection of a point in the intersection of many sets. Existing approaches yield an infeasible point with an iteration-complexity of $O(1/\varepsilon^2)$ for nonsmooth problems with no guarantees on the in-feasibility. By reformulating the problem through exact penalty functions, we derive first-order algorithms which not only guarantees that the distance to the intersection is small but also improve the complexity to $O(1/\varepsilon)$ and $O(1/\sqrt{\varepsilon})$ for smooth functions. For composite and smooth problems, this is achieved through a saddle-point reformulation where the proximal operators required by the primal-dual algorithms can be computed in closed form. We illustrate the benefits of our approach on a graph transduction problem and on graph matching

Self-Adaptive and Relaxed Self-Adaptive Projection Methods for Solving the Multiple-Set Split Feasibility Problem

Given nonempty closed convex subsets , and nonempty closed convex subsets , , in the - and -dimensional Euclidean spaces, respectively. The multiple-set split feasibility problem (MSSFP) proposed by Censor is to find a vector such that , where is a given real matrix. It serves as a model for many inverse problems where constraints are imposed on the solutions in the domain of a linear operator as well as in the operator’s range. MSSFP has a variety of specific applications in real world, such as medical care, image reconstruction, and signal processing. In this paper, for the MSSFP, we first propose a new self-adaptive projection method by adopting Armijo-like searches, which dose not require estimating the Lipschitz constant and calculating the largest eigenvalue of the matrix ; besides, it makes a sufficient decrease of the objective function at each iteration. Then we introduce a relaxed self-adaptive projection method by using projections onto half-spaces instead of those onto convex sets. Obviously, the latter are easy to implement. Global convergence for both methods is proved under a suitable condition