Semi-proximal Mirror-Prox for Nonsmooth Composite Minimization

We propose a new first-order optimisation algorithm to solve high-dimensional non-smooth composite minimisation problems. Typical examples of such problems have an objective that decomposes into a non-smooth empirical risk part and a non-smooth regularisation penalty. The proposed algorithm, called Semi-Proximal Mirror-Prox, leverages the Fenchel-type representation of one part of the objective while handling the other part of the objective via linear minimization over the domain. The algorithm stands in contrast with more classical proximal gradient algorithms with smoothing, which require the computation of proximal operators at each iteration and can therefore be impractical for high-dimensional problems. We establish the theoretical convergence rate of Semi-Proximal Mirror-Prox, which exhibits the optimal complexity bounds, i.e. $O(1/\epsilon^2)$, for the number of calls to linear minimization oracle. We present promising experimental results showing the interest of the approach in comparison to competing methods

Variational Inclusions with General Over-relaxed Proximal Point and Variational-like Inequalities with Densely Pseudomonotonicity

This dissertation focuses on the existence and uniqueness of the solutions of variational inclusion and variational inequality problems and then attempts to develop efficient algorithms to estimate numerical solutions for the problems. The dissertation consists a total of five chapters. Chapter 1 is an introduction to variational inequality problems, variational inclusion problems, monotone operators, and some basic definitions and preliminaries from convex analysis. Chapter 2 is a study of a general class of nonlinear implicit inclusion problems. The objective of this study is to explore how to omit the Lipschitz continuity condition by using an alternating approach to the proximal point algorithm to estimate the numerical solution of the implicit inclusion problems. In chapter 3 we introduce generalized densely relaxed ƞ - α pseudomonotone operators and generalized relaxed ƞ - α proper quasimonotone operators as well as relaxed ƞ - α quasimonotone operators. Using these generalized monotonicity notions, we establish the existence results for the generalized variational-like inequality in the general setting of Banach spaces. In chapter 4, we use the auxiliary principle technique to introduce a general algorithm for solutions of the densely relaxed pseudomonotone variational-like inequalities. Chapter 5 is the chapter concluding remarks and scope for future work

Inexact Model: A Framework for Optimization and Variational Inequalities

In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many known methods as a special case, the list including accelerated gradient method, composite optimization methods, level-set methods, proximal methods. The idea of the framework is based on constructing an inexact model of the main problem component, i.e. objective function in optimization or operator in variational inequalities. Besides reproducing known results, our framework allows to construct new methods, which we illustrate by constructing a universal method for variational inequalities with composite structure. This method works for smooth and non-smooth problems with optimal complexity without a priori knowledge of the problem smoothness. We also generalize our framework for strongly convex objectives and strongly monotone variational inequalities.Comment: 41 page