Global Optimization with Polynomials

The class of POP (Polynomial Optimization Problems) covers a wide rang of optimization problems such as 0 - 1 integer linear and quadratic programs, nonconvex quadratic programs and bilinear matrix inequalities. In this paper, we review some methods on solving the unconstraint case: minimize a real-valued polynomial p(x) : Rn â R, as well the constraint case: minimize p(x) on a semialgebraic set K, i.e., a set defined by polynomial equalities and inequalities. We also summarize some questions that we are currently considering.Singapore-MIT Alliance (SMA

A semi-proximal-based strictly contractive Peaceman-Rachford splitting method

The Peaceman-Rachford splitting method is very efficient for minimizing sum of two functions each depends on its variable, and the constraint is a linear equality. However, its convergence was not guaranteed without extra requirements. Very recently, He et al. (SIAM J. Optim. 24: 1011 - 1040, 2014) proved the convergence of a strictly contractive Peaceman-Rachford splitting method by employing a suitable underdetermined relaxation factor. In this paper, we further extend the so-called strictly contractive Peaceman-Rachford splitting method by using two different relaxation factors, and to make the method more flexible, we introduce semi-proximal terms to the subproblems. We characterize the relation of these two factors, and show that one factor is always underdetermined while the other one is allowed to be larger than 1. Such a flexible conditions makes it possible to cover the Glowinski's ADMM whith larger stepsize. We show that the proposed modified strictly contractive Peaceman-Rachford splitting method is convergent and also prove $O(1/t)$ convergence rate in ergodic and nonergodic sense, respectively. The numerical tests on an extensive collection of problems demonstrate the efficiency of the proposed method

Optimal parameter for the SOR-like iteration method for solving the system of absolute value equations

The SOR-like iteration method for solving the absolute value equations~(AVE) of finding a vector $x$ such that $Ax - |x| - b = 0$ with $\nu = \|A^{-1}\|_2 < 1$ is investigated. The convergence conditions of the SOR-like iteration method proposed by Ke and Ma ([{\em Appl. Math. Comput.}, 311:195--202, 2017]) are revisited and a new proof is given, which exhibits some insights in determining the convergent region and the optimal iteration parameter. Along this line, the optimal parameter which minimizes $\|T_\nu(\omega)\|_2$ with $$T_\nu(\omega) = \left(\begin{array}{cc} |1-\omega| & \omega^2\nu \\ |1-\omega| & |1-\omega| +\omega^2\nu \end{array}\right)$$ and the approximate optimal parameter which minimizes $\eta_{\nu}(\omega) =\max\{|1-\omega|,\nu\omega^2\}$ are explored. The optimal and approximate optimal parameters are iteration-independent and the bigger value of $\nu$ is, the smaller convergent region of the iteration parameter $\omega$ is. Numerical results are presented to demonstrate that the SOR-like iteration method with the optimal parameter is superior to that with the approximate optimal parameter proposed by Guo, Wu and Li ([{\em Appl. Math. Lett.}, 97:107--113, 2019]). In some situation, the SOR-like itration method with the optimal parameter performs better, in terms of CPU time, than the generalized Newton method (Mangasarian, [{\em Optim. Lett.}, 3:101--108, 2009]) for solving the AVE.Comment: 23 pages, 7 figures, 7 table

On the convergence analysis of the greedy randomized Kaczmarz method

In this paper, we analyze the greedy randomized Kaczmarz (GRK) method proposed in Bai and Wu (SIAM J. Sci. Comput., 40(1):A592--A606, 2018) for solving linear systems. We develop more precise greedy probability criteria to effectively select the working row from the coefficient matrix. Notably, we prove that the linear convergence of the GRK method is deterministic and demonstrate that using a tighter threshold parameter can lead to a faster convergence rate. Our result revises existing convergence analyses, which are solely based on the expected error by realizing that the iterates of the GRK method are random variables. Consequently, we obtain an improved iteration complexity for the GRK method. Moreover, the Polyak's heavy ball momentum technique is incorporated to improve the performance of the GRK method. We propose a refined convergence analysis, compared with the technique used in Loizou and Richt\'{a}rik (Comput. Optim. Appl., 77(3):653--710, 2020), of momentum variants of randomized iterative methods, which shows that the proposed GRK method with momentum (mGRK) also enjoys a deterministic linear convergence. Numerical experiments show that the mGRK method is more efficient than the GRK method

On adaptive stochastic heavy ball momentum for solving linear systems

The stochastic heavy ball momentum (SHBM) method has gained considerable popularity as a scalable approach for solving large-scale optimization problems. However, one limitation of this method is its reliance on prior knowledge of certain problem parameters, such as singular values of a matrix. In this paper, we propose an adaptive variant of the SHBM method for solving stochastic problems that are reformulated from linear systems using user-defined distributions. Our adaptive SHBM (ASHBM) method utilizes iterative information to update the parameters, addressing an open problem in the literature regarding the adaptive learning of momentum parameters. We prove that our method converges linearly in expectation, with a better convergence rate compared to the basic method. Notably, we demonstrate that the deterministic version of our ASHBM algorithm can be reformulated as a variant of the conjugate gradient (CG) method, inheriting many of its appealing properties, such as finite-time convergence. Consequently, the ASHBM method can be further generalized to develop a brand-new framework of the stochastic CG (SCG) method for solving linear systems. Our theoretical results are supported by numerical experiments

Fast stochastic dual coordinate descent algorithms for linearly constrained convex optimization

The problem of finding a solution to the linear system $Ax = b$ with certain minimization properties arises in numerous scientific and engineering areas. In the era of big data, the stochastic optimization algorithms become increasingly significant due to their scalability for problems of unprecedented size. This paper focuses on the problem of minimizing a strongly convex function subject to linear constraints. We consider the dual formulation of this problem and adopt the stochastic coordinate descent to solve it. The proposed algorithmic framework, called fast stochastic dual coordinate descent, utilizes sampling matrices sampled from user-defined distributions to extract gradient information. Moreover, it employs Polyak's heavy ball momentum acceleration with adaptive parameters learned through iterations, overcoming the limitation of the heavy ball momentum method that it requires prior knowledge of certain parameters, such as the singular values of a matrix. With these extensions, the framework is able to recover many well-known methods in the context, including the randomized sparse Kaczmarz method, the randomized regularized Kaczmarz method, the linearized Bregman iteration, and a variant of the conjugate gradient (CG) method. We prove that, with strongly admissible objective function, the proposed method converges linearly in expectation. Numerical experiments are provided to confirm our results.Comment: arXiv admin note: text overlap with arXiv:2305.0548