Scaling Sparse Matrices for Optimization Algorithms

To iteratively solve large scale optimization problems in various contexts like planning, operations, design etc., we need to generate descent directions that are based on linear system solutions. Irrespective of the optimization algorithm or the solution method employed for the linear systems, ill conditioning introduced by problem characteristics or the algorithm or both need to be addressed. In [GL01] we used an intuitive heuristic approach in scaling linear systems that improved performance of a large scale interior point algorithm significantly. We saw a factor of 10*3* improvements in condition number estimates. In this paper, given our experience with optimization problems from a variety of application backgrounds like economics, finance, engineering, planning etc., we examine the theoretical basis for scaling while solving the linear systems. Our goal is to develop reasonably "good" scaling schemes with sound theoretical basis. We introduce concepts and define "good" scaling schemes in section (1), as well as explain related work in this area. Scaling has been studied extensively and though there is a broad agreement on its importance, the same cannot be said about what constitutes good scaling. A theoretical framework to scale an m x n real matrix is established in section (2). We use the first order conditions associated with the Euclidean metric to develop iterative schemes in section (2.3) that approximate solution in O(mn) time for real matrice. We discuss symmetry preserving scale factors for an n x n symmetric matrix in (3). The importance of symmetry preservation is discussed in section (3.1). An algorithm to directly compute symmetry preserving scale factors in O(n2) time based on Euclidean metric is presented in section (3.4) We also suggest scaling schemes based on rectilinear norm in section (2.4). Though all p-norms are theoretically equivalent, the importance of outliers increases as p increases. For barrier methods, due to large diagnal corrections, we believe that the taxicab metric (p = 1) may be more appropriate. We develop a linear programming model for it and look at a "reduced" dual that can be formulated as a minimum cost flow problem on networks. We are investigating algorithms to solve it in O(mn) time that we require for an efficient scaling procedure. We hope that in future special structure of the "reduced" dual could be exploited to solve it quickly. The dual information can then be used to compute the required scale factors. We discuss Manhattan metric for symmetric matrices in section (3.5) and as in the case of real matrices, we are unable to propose an efficient computational scheme for this metric. We look at a linearized ideal penalty function that only uses deviations out of the desired range in section (2.5). If we could use such a metric to generate an efficient solution, then we would like to see impact of changing the range on the numerical behavior.

Scaling Sparse Constrained Nonlinear Problems for Iterative Solvers

We look at scaling a nonlinear optimization problem for iterative solvers that use at least first derivatives. These derivatives are either computed analytically or by differncing. We ignore iterative methods that are based on function evaluations only and that do not use any derivative information. We also exclude methods where the full problem structure is unknown like variants of delayed column generation. We look at related work in section (1). Despite its importance as evidenced in widely used implementations of nonlinear programming algorithms, scaling has not received enough attention from a theoretical point of view. What do we mean by scaling a nonlinear problem itself is not very clear. In this paper we attempt a scaling framework definition. We start with a description of a nonlinear problem in section (2). Various authors prefer different forms, but all forms can be converted to the form we show. We then describe our scaling framework in section (3). We show the equivalence between the original problem and the scaled problem. The correctness results of section (3.3) play an important role in the dynamic scaling scheme suggested. In section (4), we develop a prototypical algorithm that can be used to represent a variety of iterative solution methods. Using this we examine the impact of scaling in section (5). In the last section (6), we look at what the goal should be for an ideal scaling scheme and make some implementation suggestions for nonlinear solvers.

Power System State Estimation and Contingency Constrained Optimal Power Flow - A Numerically Robust Implementation

The research conducted in this dissertation is divided into two main parts. The first part provides further improvements in power system state estimation and the second part implements Contingency Constrained Optimal Power Flow (CCOPF) in a stochastic multiple contingency framework. As a real-time application in modern power systems, the existing Newton-QR state estimation algorithms are too slow and too fragile numerically. This dissertation presents a new and more robust method that is based on trust region techniques. A faster method was found among the class of Krylov subspace iterative methods, a robust implementation of the conjugate gradient method, called the LSQR method. Both algorithms have been tested against the widely used Newton-QR state estimator on the standard IEEE test networks. The trust region method-based state estimator was found to be very reliable under severe conditions (bad data, topological and parameter errors). This enhanced reliability justifies the additional time and computational effort required for its execution. The numerical simulations indicate that the iterative Newton-LSQR method is competitive in robustness with classical direct Newton-QR. The gain in computational efficiency has not come at the cost of solution reliability. The second part of the dissertation combines Sequential Quadratic Programming (SQP)-based CCOPF with Monte Carlo importance sampling to estimate the operating cost of multiple contingencies. We also developed an LP-based formulation for the CCOPF that can efficiently calculate Locational Marginal Prices (LMPs) under multiple contingencies. Based on Monte Carlo importance sampling idea, the proposed algorithm can stochastically assess the impact of multiple contingencies on LMP-congestion prices

Finding a point in the relative interior of a polyhedron

A new initialization or `Phase I' strategy for feasible interior point methods for linear programming is proposed that computes a point on the primal-dual central path associated with the linear program. Provided there exist primal-dual strictly feasible points - an all-pervasive assumption in interior point method theory that implies the existence of the central path - our initial method (Algorithm 1) is globally Q-linearly and asymptotically Q-quadratically convergent, with a provable worst-case iteration complexity bound. When this assumption is not met, the numerical behaviour of Algorithm 1 is highly disappointing, even when the problem is primal-dual feasible. This is due to the presence of implicit equalities, inequality constraints that hold as equalities at all the feasible points. Controlled perturbations of the inequality constraints of the primal-dual problems are introduced - geometrically equivalent to enlarging the primal-dual feasible region and then systematically contracting it back to its initial shape - in order for the perturbed problems to satisfy the assumption. Thus Algorithm 1 can successfully be employed to solve each of the perturbed problems.\ud We show that, when there exist primal-dual strictly feasible points of the original problems, the resulting method, Algorithm 2, finds such a point in a finite number of changes to the perturbation parameters. When implicit equalities are present, but the original problem and its dual are feasible, Algorithm 2 asymptotically detects all the primal-dual implicit equalities and generates a point in the relative interior of the primal-dual feasible set. Algorithm 2 can also asymptotically detect primal-dual infeasibility. Successful numerical experience with Algorithm 2 on linear programs from NETLIB and CUTEr, both with and without any significant preprocessing of the problems, indicates that Algorithm 2 may be used as an algorithmic preprocessor for removing implicit equalities, with theoretical guarantees of convergence

Implementation of the nonlinear programming algorithm when optimizing the final turning mode

For computer-aided multi-variant design of machining technologies, it is important to optimize the cutting parameters at the final pass in each technological operation. When carrying out designing procedures, there emerge problems relating to the algorithm of choosing the decision-making method, the objective function, and the regions of feasibility at final machining steps. Linear programming is time-consuming for multi-variant and multi-pass machining, if the algorithm is to be clear. It is known that when simulating the optimal metal-cutting process, the optimization criterion and the system of constraints are non-linear. Therefore, a computational algorithm can be made significantly more efficient if it is a non-linear algorithm based on Lagrange multipliers Such approach to design helps simplify automating the computational algorithm for multi-pass single-tool machining with a precision cutting tool (a reamer) This is the method discussed herein. © 2020 IOP Publishing Ltd. All rights reserved

Interior-point methods for PDE-constrained optimization

In applied sciences PDEs model an extensive variety of phenomena. Typically the final goal of simulations is a system which is optimal in a certain sense. For instance optimal control problems identify a control to steer a system towards a desired state. Inverse problems seek PDE parameters which are most consistent with measurements. In these optimization problems PDEs appear as equality constraints. PDE-constrained optimization problems are large-scale and often nonconvex. Their numerical solution leads to large ill-conditioned linear systems. In many practical problems inequality constraints implement technical limitations or prior knowledge. In this thesis interior-point (IP) methods are considered to solve nonconvex large-scale PDE-constrained optimization problems with inequality constraints. To cope with enormous fill-in of direct linear solvers, inexact search directions are allowed in an inexact interior-point (IIP) method. This thesis builds upon the IIP method proposed in [Curtis, Schenk, Wächter, SIAM Journal on Scientific Computing, 2010]. SMART tests cope with the lack of inertia information to control Hessian modification and also specify termination tests for the iterative linear solver. The original IIP method needs to solve two sparse large-scale linear systems in each optimization step. This is improved to only a single linear system solution in most optimization steps. Within this improved IIP framework, two iterative linear solvers are evaluated: A general purpose algebraic multilevel incomplete L D L^T preconditioned SQMR method is applied to PDE-constrained optimization problems for optimal server room cooling in three space dimensions and to compute an ambient temperature for optimal cooling. The results show robustness and efficiency of the IIP method when compared with the exact IP method. These advantages are even more evident for a reduced-space preconditioned (RSP) GMRES solver which takes advantage of the linear system's structure. This RSP-IIP method is studied on the basis of distributed and boundary control problems originating from superconductivity and from two-dimensional and three-dimensional parameter estimation problems in groundwater modeling. The numerical results exhibit the improved efficiency especially for multiple PDE constraints. An inverse medium problem for the Helmholtz equation with pointwise box constraints is solved by IP methods. The ill-posedness of the problem is explored numerically and different regularization strategies are compared. The impact of box constraints and the importance of Hessian modification on the optimization algorithm is demonstrated. A real world seismic imaging problem is solved successfully by the RSP-IIP method

Algorithms for Trust-Region Subproblems with Linear Inequality Constraints

In the trust-region framework for optimizing a general nonlinear function subject to nonlinear inequality constraints, sequential quadratic programming (SQP) techniques generate subproblems in which a quadratic function must be minimized over a spherical region subject to linear inequality constraints. An interior-point algorithm proposed by Kearsley approximately solves these subproblems when the objective functions are large-scale and convex. Kearsley's algorithm handles the inequality constraints with a classical log-barrier function, minimizing quadratic models of the log-barrier function for fixed values of the barrier parameter subject to the trust-region constraint. Kearsley recommends the LSTRS algorithm of Rojas et al. for minimizing these models. For the convex case, we prove convergence of Kearsley's algorithm and suggest alternatives to the LSTRS algorithm. These alternatives include the new annulus algorithm of Griffin et al., which blends the conjugate gradient and sequential subspace minimization methods to yield promising numerical results. For the nonconvex case, we propose and test a new interior-point algorithm that incorporates the annulus algorithm into an SQP framework with trust regions.Doctor of Philosoph

Recent Experiences in Multidisciplinary Analysis and Optimization, part 2

The papers presented at the NASA Symposium on Recent Experiences in Multidisciplinary Analysis and Optimization held at NASA Langley Research Center, Hampton, Virginia, April 24 to 26, 1984 are given. The purposes of the symposium were to exchange information about the status of the application of optimization and the associated analyses in industry or research laboratories to real life problems and to examine the directions of future developments