On complexity and convergence of high-order coordinate descent algorithms

Coordinate descent methods with high-order regularized models for box-constrained minimization are introduced. High-order stationarity asymptotic convergence and first-order stationarity worst-case evaluation complexity bounds are established. The computer work that is necessary for obtaining first-order $\varepsilon$-stationarity with respect to the variables of each coordinate-descent block is $O(\varepsilon^{-(p+1)/p})$ whereas the computer work for getting first-order $\varepsilon$-stationarity with respect to all the variables simultaneously is $O(\varepsilon^{-(p+1)})$. Numerical examples involving multidimensional scaling problems are presented. The numerical performance of the methods is enhanced by means of coordinate-descent strategies for choosing initial points

Large-scale unconstrained optimization using separable cubic modeling and matrix-free subspace minimization

We present a new algorithm for solving large-scale unconstrained optimization problems that uses cubic models, matrix-free subspace minimization, and secant-type parameters for defining the cubic terms. We also propose and analyze a specialized trust-region strategy to minimize the cubic model on a properly chosen low-dimensional subspace, which is built at each iteration using the Lanczos process. For the convergence analysis we present, as a general framework, a model trust-region subspace algorithm with variable metric and we establish asymptotic as well as complexity convergence results. Preliminary numerical results, on some test functions and also on the well-known disk packing problem, are presented to illustrate the performance of the proposed scheme when solving large-scale problems169FAPESP – Fundação de Amparo à Pesquisa Do Estado De São Paulo2011/51305-02; 2013/05475-7; 2013/07375-0E-26/111.449/2010-APQ1FAPERJ - Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de JaneiroPRONEX-CNPq/FAPERJCarlos Chagas Filho Foundation for Research Support of the State of Rio de Janeiro (FAPERJ) [E-26/111.449/2010-APQ1]; CEPID-Industrial Mathematics/FAPESP [2011/51305-02]; FAPESPFundacao de Amparo a Pesquisa do Estado de Sao Paulo (FAPESP) [2013/05475-7, 2013/07375-0]; Fundacao para a Ciencia e a Tecnologia (Portuguese Foundation for Science and Technology)Portuguese Foundation for Science and Technology [UID/MAT/00297/2019