On robust solutions to uncertain linear complementarity problems and their variants

A popular approach for addressing uncertainty in variational inequality problems is by solving the expected residual minimization (ERM) problem. This avenue necessitates distributional information associated with the uncertainty and requires minimizing nonconvex expectation-valued functions. We consider a distinctly different approach in the context of uncertain linear complementarity problems with a view towards obtaining robust solutions. Specifically, we define a robust solution to a complementarity problem as one that minimizes the worst-case of the gap function. In what we believe is amongst the first efforts to comprehensively address such problems in a distribution-free environment, we show that under specified assumptions on the uncertainty sets, the robust solutions to uncertain monotone linear complementarity problem can be tractably obtained through the solution of a single convex program. We also define uncertainty sets that ensure that robust solutions to non-monotone generalizations can also be obtained by solving convex programs. More generally, robust counterparts of uncertain non-monotone LCPs are proven to be low-dimensional nonconvex quadratically constrained quadratic programs. We show that these problems may be globally resolved by customizing an existing branching scheme. We further extend the tractability results to include uncertain affine variational inequality problems defined over uncertain polyhedral sets as well as to hierarchical regimes captured by mathematical programs with uncertain complementarity constraints. Preliminary numerics on uncertain linear complementarity and traffic equilibrium problems suggest that the presented avenues hold promise.Comment: 37 pages, 3 figures, 8 table

Canonical Duality-Triality Theory: Bridge Between Nonconvex Analysis/Mechanics and Global Optimization in Complex Systems

Canonical duality-triality is a breakthrough methodological theory, which can be used not only for modeling complex systems within a unified framework, but also for solving a wide class of challenging problems from real-world applications. This paper presents a brief review on this theory, its philosophical origin, physics foundation, and mathematical statements in both finite and infinite dimensional spaces, with emphasizing on its role for bridging the gap between nonconvex analysis/mechanics and global optimization. Special attentions are paid on unified understanding the fundamental difficulties in large deformation mechanics, bifurcation/chaos in nonlinear science, and the NP-hard problems in global optimization, as well as the theorems, methods, and algorithms for solving these challenging problems. Misunderstandings and confusions on some basic concepts, such as objectivity, nonlinearity, Lagrangian, and generalized convexities are discussed and classified. Breakthrough from recent challenges and conceptual mistakes by M. Voisei, C. Zalinescu and his co-worker are addressed. Some open problems and future works in global optimization and nonconvex mechanics are proposed.Comment: 43 pages, 4 figures. appears in Mathematics and Mechanics of Solids, 201

Alternating minimization and alternating descent over nonconvex sets

We analyze the performance of alternating minimization for loss functions optimized over two variables, where each variable may be restricted to lie in some potentially nonconvex constraint set. This type of setting arises naturally in high-dimensional statistics and signal processing, where the variables often reflect different structures or components within the signals being considered. Our analysis relies on the notion of local concavity coefficients, which has been proposed in Barber and Ha to measure and quantify the concavity of a general nonconvex set. Our results further reveal important distinctions between alternating and non-alternating methods. Since computing the alternating minimization steps may not be tractable for some problems, we also consider an inexact version of the algorithm and provide a set of sufficient conditions to ensure fast convergence of the inexact algorithms. We demonstrate our framework on several examples, including low rank + sparse decomposition and multitask regression, and provide numerical experiments to validate our theoretical results

On quadratic optimization problems and canonical duality theory

DY Gao solely or together with some of his collaborators applied his Canonical duality theory (CDT) for solving some quadratic optimization problems with quadratic constraints. Unfortunately, in almost all papers we read on CDT there are unclear definitions, non convincing arguments in the proofs, and even false results. The aim of this paper is to treat rigorously quadratic optimization problems by the method suggested by CDT and to compare what we get with the results obtained by DY Gao and his collaborators on this topic in several papers.Comment: 20 pages; presented at 4th International Conference on Nonlinear Analysis and Optimization, Zanjan (Iran

Canonical duality for solving general nonconvex constrained problems

This paper presents a canonical duality theory for solving a general nonconvex constrained optimization problem within a unified framework to cover Lagrange multiplier method and KKT theory. It is proved that if both target function and constraints possess certain patterns necessary for modeling real systems, a perfect dual problem (without duality gap)can be obtained in a unified form with global optimality conditions provided. While the popular augmented Lagrangian method may produce more difficult nonconvex problems due to the nonlinearity of constraints.Comment: 14 pages, 3 figure

Penalty Dual Decomposition Method For Nonsmooth Nonconvex Optimization

Many contemporary signal processing, machine learning and wireless communication applications can be formulated as nonconvex nonsmooth optimization problems. Often there is a lack of efficient algorithms for these problems, especially when the optimization variables are nonlinearly coupled in some nonconvex constraints. In this work, we propose an algorithm named penalty dual decomposition (PDD) for these difficult problems and discuss its various applications. The PDD is a double-loop iterative algorithm. Its inner iterations is used to inexactly solve a nonconvex nonsmooth augmented Lagrangian problem via block-coordinate-descenttype methods, while its outer iteration updates the dual variables and/or a penalty parameter. In Part I of this work, we describe the PDD algorithm and rigorously establish its convergence to KKT solutions. In Part II we evaluate the performance of PDD by customizing it to three applications arising from signal processing and wireless communications.Comment: Two part paper, 27 figure

A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data

This article presents a powerful algorithmic framework for big data optimization, called the Block Successive Upper bound Minimization (BSUM). The BSUM includes as special cases many well-known methods for analyzing massive data sets, such as the Block Coordinate Descent (BCD), the Convex-Concave Procedure (CCCP), the Block Coordinate Proximal Gradient (BCPG) method, the Nonnegative Matrix Factorization (NMF), the Expectation Maximization (EM) method and so on. In this article, various features and properties of the BSUM are discussed from the viewpoint of design flexibility, computational efficiency, parallel/distributed implementation and the required communication overhead. Illustrative examples from networking, signal processing and machine learning are presented to demonstrate the practical performance of the BSUM framewor

Computing B-Stationary Points of Nonsmooth DC Programs

Motivated by a class of applied problems arising from physical layer based security in a digital communication system, in particular, by a secrecy sum-rate maximization problem, this paper studies a nonsmooth, difference-of-convex (dc) minimization problem. The contributions of this paper are: (i) clarify several kinds of stationary solutions and their relations; (ii) develop and establish the convergence of a novel algorithm for computing a d-stationary solution of a problem with a convex feasible set that is arguably the sharpest kind among the various stationary solutions; (iii) extend the algorithm in several directions including: a randomized choice of the subproblems that could help the practical convergence of the algorithm, a distributed penalty approach for problems whose objective functions are sums of dc functions, and problems with a specially structured (nonconvex) dc constraint. For the latter class of problems, a pointwise Slater constraint qualification is introduced that facilitates the verification and computation of a B(ouligand)-stationary point

A Second-Order Cone Based Approach for Solving the Trust Region Subproblem and Its Variants

We study the trust-region subproblem (TRS) of minimizing a nonconvex quadratic function over the unit ball with additional conic constraints. Despite having a nonconvex objective, it is known that the classical TRS and a number of its variants are polynomial-time solvable. In this paper, we follow a second-order cone (SOC) based approach to derive an exact convex reformulation of the TRS under a structural condition on the conic constraint. Our structural condition is immediately satisfied when there is no additional conic constraints, and it generalizes several such conditions studied in the literature. As a result, our study highlights an explicit connection between the classical nonconvex TRS and smooth convex quadratic minimization, which allows for the application of cheap iterative methods such as Nesterov's accelerated gradient descent, to the TRS. Furthermore, under slightly stronger conditions, we give a low-complexity characterization of the convex hull of the epigraph of the nonconvex quadratic function intersected with the constraints defining the domain without any additional variables. We also explore the inclusion of additional hollow constraints to the domain of the TRS, and convexification of the associated epigraph

Nonconvex and Nonsmooth Sparse Optimization via Adaptively Iterative Reweighted Methods

We present a general formulation of nonconvex and nonsmooth sparse optimization problems with a convexset constraint, which takes into account most existing types of nonconvex sparsity-inducing terms. It thus brings strong applicability to a wide range of applications. We further design a general algorithmic framework of adaptively iterative reweighted algorithms for solving the nonconvex and nonsmooth sparse optimization problems. This is achieved by solving a sequence of weighted convex penalty subproblems with adaptively updated weights. The first-order optimality condition is then derived and the global convergence results are provided under loose assumptions. This makes our theoretical results a practical tool for analyzing a family of various iteratively reweighted algorithms. In particular, for the iteratively reweighed $\ell_1$-algorithm, global convergence analysis is provided for cases with diminishing relaxation parameter. For the iteratively reweighed $\ell_2$-algorithm, adaptively decreasing relaxation parameter is applicable and the existence of the cluster point to the algorithm is established. The effectiveness and efficiency of our proposed formulation and the algorithms are demonstrated in numerical experiments in various sparse optimization problems