80 research outputs found
Accelerated Nonconvex ADMM with Self-Adaptive Penalty for Rank-Constrained Model Identification
The alternating direction method of multipliers (ADMM) has been widely
adopted in low-rank approximation and low-order model identification tasks;
however, the performance of nonconvex ADMM is highly reliant on the choice of
penalty parameter. To accelerate ADMM for solving rankconstrained
identification problems, this paper proposes a new self-adaptive strategy for
automatic penalty update. Guided by first-order analysis of the increment of
the augmented Lagrangian, the self-adaptive penalty updating enables effective
and balanced minimization of both primal and dual residuals and thus ensures a
stable convergence. Moreover, improved efficiency can be obtained within the
Anderson acceleration scheme. Numerical examples show that the proposed
strategy significantly accelerates the convergence of nonconvex ADMM while
alleviating the critical reliance on tedious tuning of penalty parameters.Comment: 7 pages, 4 figures. Submitted to 62nd IEEE Conference on Decision and
Control (CDC 2023
An efficient approach for nonconvex semidefinite optimization via customized alternating direction method of multipliers
We investigate a class of general combinatorial graph problems, including
MAX-CUT and community detection, reformulated as quadratic objectives over
nonconvex constraints and solved via the alternating direction method of
multipliers (ADMM).
We propose two reformulations: one using vector variables and a binary
constraint, and the other further reformulating the Burer-Monteiro form for
simpler subproblems.
Despite the nonconvex constraint, we prove the ADMM iterates converge to a
stationary point in both formulations, under mild assumptions.
Additionally, recent work suggests that in this latter form, when the matrix
factors are wide enough, local optimum with high probability is also the global
optimum.
To demonstrate the scalability of our algorithm, we include results for
MAX-CUT, community detection, and image segmentation benchmark and simulated
examples.Comment: arXiv admin note: text overlap with arXiv:1805.1067
A Primal-Dual Algorithmic Framework for Constrained Convex Minimization
We present a primal-dual algorithmic framework to obtain approximate
solutions to a prototypical constrained convex optimization problem, and
rigorously characterize how common structural assumptions affect the numerical
efficiency. Our main analysis technique provides a fresh perspective on
Nesterov's excessive gap technique in a structured fashion and unifies it with
smoothing and primal-dual methods. For instance, through the choices of a dual
smoothing strategy and a center point, our framework subsumes decomposition
algorithms, augmented Lagrangian as well as the alternating direction
method-of-multipliers methods as its special cases, and provides optimal
convergence rates on the primal objective residual as well as the primal
feasibility gap of the iterates for all.Comment: This paper consists of 54 pages with 7 tables and 12 figure
A Unified Bregman Alternating Minimization Algorithm for Generalized DC Programming with Application to Imaging Data
In this paper, we consider a class of nonconvex (not necessarily
differentiable) optimization problems called generalized DC
(Difference-of-Convex functions) programming, which is minimizing the sum of
two separable DC parts and one two-block-variable coupling function. To
circumvent the nonconvexity and nonseparability of the problem under
consideration, we accordingly introduce a Unified Bregman Alternating
Minimization Algorithm (UBAMA) by maximally exploiting the favorable DC
structure of the objective. Specifically, we first follow the spirit of
alternating minimization to update each block variable in a sequential order,
which can efficiently tackle the nonseparablitity caused by the coupling
function. Then, we employ the Fenchel-Young inequality to approximate the
second DC components (i.e., concave parts) so that each subproblem reduces to a
convex optimization problem, thereby alleviating the computational burden of
the nonconvex DC parts. Moreover, each subproblem absorbs a Bregman proximal
regularization term, which is usually beneficial for inducing closed-form
solutions of subproblems for many cases via choosing appropriate Bregman kernel
functions. It is remarkable that our algorithm not only provides an algorithmic
framework to understand the iterative schemes of some novel existing
algorithms, but also enjoys implementable schemes with easier subproblems than
some state-of-the-art first-order algorithms developed for generic nonconvex
and nonsmooth optimization problems. Theoretically, we prove that the sequence
generated by our algorithm globally converges to a critical point under the
Kurdyka-{\L}ojasiewicz (K{\L}) condition. Besides, we estimate the local
convergence rates of our algorithm when we further know the prior information
of the K{\L} exponent.Comment: 44 pages, 7figures, 5 tables. Any comments are welcom
- …