A hybrid algorithm for the two-trust-region subproblem

Two-trust-region subproblem (TTRS), which is the minimization of a general quadratic function over the intersection of two full-dimensional ellipsoids, has been the subject of several recent research. In this paper, to solve TTRS, a hybrid of efficient algorithms for finding global and local-nonglobal minimizers of trust-region subproblem and the alternating direction method of multipliers (ADMM) is proposed. The convergence of the ADMM steps to the first order stationary condition is proved under certain conditions. On several classes of test problems, we compare the new algorithm with the recent algorithm of Sakaue et. al's \cite{SakaueNakat:16} and Snopt software

Local Nonglobal Minima for Solving Large Scale Extended Trust Region Subproblems

We study large scale extended trust region subproblems (eTRS) i.e., the minimization of a general quadratic function subject to a norm constraint, known as the trust region subproblem (TRS) but with an additional linear inequality constraint. It is well known that strong duality holds for the TRS and that there are efficient algorithms for solving large scale TRS problems. It is also known that there can exist at most one local non-global minimizer (LNGM) for TRS. We combine this with known characterizations for strong duality for eTRS and, in particular, connect this with the so-called hard case for TRS. We begin with a recent characterization of the minimum for the TRS via a generalized eigenvalue problem and extend this result to the LNGM. We then use this to derive an efficient algorithm that finds the global minimum for eTRS by solving at most three generalized eigenvalue problems.Comment: 25 pages including table of contents and index; 8 table

Optimization of triangular networks with spatial constraints

A common representation of a three dimensional object in computer applications, such as graphics and design, is in the form of a triangular mesh. In many instances, individual or groups of triangles in such representation need to satisfy spatial constraints that are imposed either by observation from the real world, or by concrete design specifications of the object. As these problems tend to be of large scale, choosing a mathematical optimization approach can be particularly challenging. In this paper, we model various geometric constraints as convex sets in Euclidean spaces, and find the corresponding projections in closed forms. We also present an interesting idea to successfully maneuver around some important nonconvex constraints while still preserving the intrinsic nature of the original design problem. We then use these constructions in modern first-order splitting methods to find optimal solutions

Inertia laws and localization of real eigenvalues for generalized indefinite eigenvalue problems

Sylvester's law of inertia states that the number of positive, negative and zero eigenvalues of Hermitian matrices is preserved under congruence transformations. The same is true of generalized Hermitian definite eigenvalue problems, in which the two matrices are allowed to undergo different congruence transformations, but not for the indefinite case. In this paper we investigate the possible change in inertia under congruence for generalized Hermitian indefinite eigenproblems, and derive sharp bounds that show the inertia of the two individual matrices often still provides useful information about the eigenvalues of the pencil, especially when one of the matrices is almost definite. A prominent application of the original Sylvester's law is in finding the number of eigenvalues in an interval. Our results can be used for estimating the number of real eigenvalues in an interval for generalized indefinite and nonlinear eigenvalue problems.Comment: 19 page

Explicit minimisation of a convex quadratic under a general quadratic constraint: a global, analytic approach

A novel approach is introduced to a very widely occurring problem, providing a complete, explicit resolution of it: minimisation of a convex quadratic under a general quadratic, equality or inequality, constraint. Completeness comes via identification of a set of mutually exclusive and exhaustive special cases. Explicitness, via algebraic expressions for each solution set. Throughout, underlying geometry illuminates and informs algebraic development. In particular, centrally to this new approach, affine equivalence is exploited to re-express the same problem in simpler coordinate systems. Overall, the analysis presented provides insight into the diverse forms taken both by the problem itself and its solution set, showing how each may be intrinsically unstable. Comparisons of this global, analytic approach with the, intrinsically complementary, local, computational approach of (generalised) trust region methods point to potential synergies between them. Points of contact with simultaneous diagonalisation results are noted

Novel reformulations and efficient algorithms for the generalized trust region subproblem

We present a new solution framework to solve the generalized trust region subproblem (GTRS) of minimizing a quadratic objective over a quadratic constraint. More specifically, we derive a convex quadratic reformulation (CQR) via minimizing a linear objective over two convex quadratic constraints for the GTRS. We show that an optimal solution of the GTRS can be recovered from an optimal solution of the CQR. We further prove that this CQR is equivalent to minimizing the maximum of the two convex quadratic functions derived from the CQR for the case under our investigation. Although the latter minimax problem is nonsmooth, it is well-structured and convex. We thus develop two steepest descent algorithms corresponding to two different line search rules. We prove for both algorithms their global sublinear convergence rates. We also obtain a local linear convergence rate of the first algorithm by estimating the Kurdyka- Lojasiewicz exponent at any optimal solution under mild conditions. We finally demonstrate the efficiency of our algorithms in our numerical experiments

A Second-Order Cone Based Approach for Solving the Trust Region Subproblem and Its Variants

We study the trust-region subproblem (TRS) of minimizing a nonconvex quadratic function over the unit ball with additional conic constraints. Despite having a nonconvex objective, it is known that the classical TRS and a number of its variants are polynomial-time solvable. In this paper, we follow a second-order cone (SOC) based approach to derive an exact convex reformulation of the TRS under a structural condition on the conic constraint. Our structural condition is immediately satisfied when there is no additional conic constraints, and it generalizes several such conditions studied in the literature. As a result, our study highlights an explicit connection between the classical nonconvex TRS and smooth convex quadratic minimization, which allows for the application of cheap iterative methods such as Nesterov's accelerated gradient descent, to the TRS. Furthermore, under slightly stronger conditions, we give a low-complexity characterization of the convex hull of the epigraph of the nonconvex quadratic function intersected with the constraints defining the domain without any additional variables. We also explore the inclusion of additional hollow constraints to the domain of the TRS, and convexification of the associated epigraph

A conjugate gradient-based algorithm for large-scale quadratic programming problem with one quadratic constraint

In this paper, we consider the nonconvex quadratically constrained quadratic programming (QCQP) with one quadratic constraint. By employing the conjugate gradient method, an efficient algorithm is proposed to solve QCQP that exploits the sparsity of the involved matrices and solves the problem via solving a sequence of positive definite system of linear equations after identifying suitable generalized eigenvalues. Some numerical experiments are given to show the effectiveness of the proposed method and to compare it with some recent algorithms in the literature

Potential-based analyses of first-order methods for constrained and composite optimization

We propose potential-based analyses for first-order algorithms applied to constrained and composite minimization problems. We first propose ``idealized'' frameworks for algorithms in the strongly and non-strongly convex cases and argue based on a potential that methods following the framework achieve the best possible rate. Then we show that the geometric descent (GD) algorithm by Bubeck et al.\ as extended to the constrained and composite setting by Chen et al.\ achieves this rate using the potential-based analysis for the strongly convex case. Next, we extend the GD algorithm to the case of non-strongly convex problems. We show using a related potential-based argument that our extension achieves the best possible rate in this case as well. The new GD algorithm achieves the best possible rate in the nonconvex case also. We also analyze accelerated gradient using the new potentials. We then turn to the special case of a quadratic function with a single ball constraint, the famous trust-region subproblem. For this case, the first-order trust-region Lanczos method by Gould et al.\ finds the optimal point in an increasing sequence of Krylov spaces. Our results for the general case immediately imply convergence rates for their method in both the strongly convex and non-strongly convex cases. We also establish the same convergence rates for their method using arguments based on Chebyshev polynomial approximation. To the best of our knowledge, no convergence rate has previously been established for the trust-region Lanczos method

The convergence of the Generalized Lanczos Trust-Region Method for the Trust-Region Subproblem

Solving the trust-region subproblem (TRS) plays a key role in numerical optimization and many other applications. The generalized Lanczos trust-region (GLTR) method is a well-known Lanczos type approach for solving a large-scale TRS. The method projects the original large-scale TRS onto a $k$ dimensional Krylov subspace, whose orthonormal basis is generated by the symmetric Lanczos process, and computes an approximate solution from the underlying subspace. There have been some a-priori error bounds for the optimal solution and the optimal objective value in the literature, but no a-priori result exists on the convergence of Lagrangian multipliers involved in projected TRS's and the residual norm of approximate solution. In this paper, a general convergence theory of the GLTR method is established, and a-priori bounds are derived for the errors of the optimal Lagrangian multiplier, the optimal solution, the optimal objective value and the residual norm of approximate solution. Numerical experiments demonstrate that our bounds are realistic and predict the convergence rates of the three errors and residual norms accurately.Comment: 28 pages, 5 figure