Asymptotic convergence rates for coordinate descent in polyhedral sets

We consider a family of parallel methods for constrained optimization based on projected gradient descents along individual coordinate directions. In the case of polyhedral feasible sets, local convergence towards a regular solution occurs unconstrained in a reduced space, allowing for the computation of tight asymptotic convergence rates by sensitivity analysis, this even when global convergence rates are unavailable or too conservative. We derive linear asymptotic rates of convergence in polyhedra for variants of the coordinate descent approach, including cyclic, synchronous, and random modes of implementation. Our results find application in stochastic optimization, and with recently proposed optimization algorithms based on Taylor approximations of the Newton step.Comment: 20 pages. A version of this paper will be submitted for publicatio

Local and Global Convergence of a General Inertial Proximal Splitting Scheme

This paper is concerned with convex composite minimization problems in a Hilbert space. In these problems, the objective is the sum of two closed, proper, and convex functions where one is smooth and the other admits a computationally inexpensive proximal operator. We analyze a general family of inertial proximal splitting algorithms (GIPSA) for solving such problems. We establish finiteness of the sum of squared increments of the iterates and optimality of the accumulation points. Weak convergence of the entire sequence then follows if the minimum is attained. Our analysis unifies and extends several previous results. We then focus on $\ell_1$-regularized optimization, which is the ubiquitous special case where the nonsmooth term is the $\ell_1$-norm. For certain parameter choices, GIPSA is amenable to a local analysis for this problem. For these choices we show that GIPSA achieves finite "active manifold identification", i.e. convergence in a finite number of iterations to the optimal support and sign, after which GIPSA reduces to minimizing a local smooth function. Local linear convergence then holds under certain conditions. We determine the rate in terms of the inertia, stepsize, and local curvature. Our local analysis is applicable to certain recent variants of the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA), for which we establish active manifold identification and local linear convergence. Our analysis motivates the use of a momentum restart scheme in these FISTA variants to obtain the optimal local linear convergence rate.Comment: 33 pages 1 figur

Gradient Dynamic Approach to the Tensor Complementarity Problem

Nonlinear gradient dynamic approach for solving the tensor complementarity problem (TCP) is presented. Theoretical analysis shows that each of the defined dynamical system models ensures the convergence performance. The computer simulation results further substantiate that the considered dynamical system can solve the tensor complementarity problem (TCP).Comment: 18pages. arXiv admin note: text overlap with arXiv:1804.00406 by other author

On the R-superlinear convergence of the KKT residues generated by the augmented Lagrangian method for convex composite conic programming

Due to the possible lack of primal-dual-type error bounds, the superlinear convergence for the Karush-Kuhn-Tucker (KKT) residues of the sequence generated by augmented Lagrangian method (ALM) for solving convex composite conic programming (CCCP) has long been an outstanding open question. In this paper, we aim to resolve this issue by first conducting convergence rate analysis for the ALM with Rockafellar's stopping criteria under only a mild quadratic growth condition on the dual of CCCP. More importantly, by further assuming that the Robinson constraint qualification holds, we establish the R-superlinear convergence of the KKT residues of the iterative sequence under easy-to-implement stopping criteria {for} the augmented Lagrangian subproblems. Equipped with this discovery, we gain insightful interpretations on the impressive numerical performance of several recently developed semismooth Newton-CG based ALM solvers for solving linear and convex quadratic semidefinite programming

Fundamentals of cone regression

Cone regression is a particular case of quadratic programming that minimizes a weighted sum of squared residuals under a set of linear inequality constraints. Several important statistical problems such as isotonic, concave regression or ANOVA under partial orderings, just to name a few, can be considered as particular instances of the cone regression problem. Given its relevance in Statistics, this paper aims to address the fundamentals of cone regression from a theoretical and practical point of view. Several formulations of the cone regression problem are considered and, focusing on the particular case of concave regression as example, several algorithms are analyzed and compared both qualitatively and quantitatively through numerical simulations. Several improvements to enhance numerical stability and bound the computational cost are proposed. For each analyzed algorithm, the pseudo-code and its corresponding code in Scilab are provided. The results from this study demonstrate that the choice of the optimization approach strongly impacts the numerical performances. It is also shown that methods are not currently available to solve efficiently cone regression problems with large dimension (more than many thousands of points). We suggest further research to fill this gap by exploiting and adapting classical multi-scale strategy to compute an approximate solution

Black Hole Evolution

Black hole formation and evaporation is studied in the semiclassical approximation in simple 1+1-dimensional models, with emphasis on issues related to Hawking's information paradox. Exact semiclassical solutions are described and questions of boundary conditions and vacuum stability are discussed. The validity of the semiclassical approximation has been called into question in the context of the information puzzle. A different approach, where black hole evolution is assumed to be unitary, is described. It requires unusual causal properties and kinematic behavior of matter that may be realized in string theory. Based on lectures given at the 1994 Trieste Spring SchoolComment: 30 pages, 8 figures, late

Uniform Convergence and Rate Adaptive Estimation of a Convex Function

This paper addresses the problem of estimating a convex regression function under both the sup-norm risk and the pointwise risk using B-splines. The presence of the convex constraint complicates various issues in asymptotic analysis, particularly uniform convergence analysis. To overcome this difficulty, we establish the uniform Lipschitz property of optimal spline coefficients in the $\ell_\infty$-norm by exploiting piecewise linear and polyhedral theory. Based upon this property, it is shown that this estimator attains optimal rates of convergence on the entire interval of interest over the H\"older class under both the risks. In addition, adaptive estimates are constructed under both the sup-norm risk and the pointwise risk when the exponent of the H\"older class is between one and two. These estimates achieve a maximal risk within a constant factor of the minimax risk over the H\"older class

Rapidly computable viscous friction and no-slip rigid contact models

This article presents computationally efficient algorithms for modeling two special cases of rigid contact---contact with only viscous friction and contact without slip---that have particularly useful applications in robotic locomotion and grasping. Modeling rigid contact with Coulomb friction generally exhibits $O(n^3)$ expected time complexity in the number of contact points and $2^{O(n)}$ worst-case complexity. The special cases we consider exhibit $O(m^3 + m^2n)$ time complexity ($m$ is the number of independent coordinates in the multi rigid body system) in the expected case and polynomial complexity in the worst case; thus, asymptotic complexity is no longer driven by number of contact points (which is conceivably limitless) but instead is more dependent on the number of bodies in the system (which is often fixed). These special cases also require considerably fewer constrained nonlinear optimization variables thus yielding substantial improvements in running time. Finally, these special cases also afford one other advantage: the nonlinear optimization problems are numerically easier to solve

New Constraint Qualifications for Optimization Problems in Banach Spaces based on Asymptotic KKT Conditions

Optimization theory in Banach spaces suffers from the lack of available constraint qualifications. Despite the fact that there exist only a very few constraint qualifications, they are, in addition, often violated even in simple applications. This is very much in contrast to finite-dimensional nonlinear programs, where a large number of constraint qualifications is known. Since these constraint qualifications are usually defined using the set of active inequality constraints, it is difficult to extend them to the infinite-dimensional setting. One exception is a recently introduced sequential constraint qualification based on asymptotic KKT conditions. This paper shows that this so-called asymptotic KKT regularity allows suitable extensions to the Banach space setting in order to obtain new constraint qualifications. The relation of these new constraint qualifications to existing ones is discussed in detail. Their usefulness is also shown by several examples as well as an algorithmic application to the class of augmented Lagrangian methods

M-theory observables for cosmological space-times

We discuss the construction of the analog of an S-matrix for space-times that begin with a Big-Bang and asymptote to an FRW universe with nonnegative cosmological constant. When the cosmological constant is positive there are many such S-matrices, related mathematically by gauge transformations and physically by an analog of the principle of black hole complementarity. In the limit of vanishing $\Lambda$ these become (approximate) Poincare transforms of each other. Considerations of the initial state require a quantum treatment of space-time, and some preliminary steps towards constructing such a theory are proposed. In this context we propose a model for the earliest semiclassical state of the universe, which suggests a solution for the horizon problem different from that provided by inflation.Comment: JHEP LaTeX, 29 page