Extended Trust-Tech Methodology For Nonlinear Optimization: Analyses, Methods And Applications

Many theoretical and practical problems can be formulated as a global optimization problem. Traditional local optimization methods can only attain a local optimal solution and be entrapped in the local optimal solution; while existing global optimization algorithms usually sparsely approximates the global optimal solution in a stochastic manner. In contrast, the transformation under stability-retaining equilibrium characterization (TRUST-TECH) methodology prevails over existing algorithms due to its capability of locating multiple, if not all, local optimal solutions to the optimization problem deterministically and systematically in a tier-by-tier manner. The TRUST-TECH methodology was developed to solve unconstrained and constrained nonlinear optimization problems. This work extends the TRUST-TECH methodology by incorporating new analytical results, developing new solution methods and solving new problems in practical applications. This work first provides analytical results regarding the invariance of partial stability region in quasi-gradient systems. Our motivation is to resolve numerical difficulties arising in implementations of trajectory based methods, including TRUST-TECH. Improved algorithms were developed to resolve these issues by altering the original problem to speed-up movement of the trajectory. However, such operations can lead the trajectory converge to a different solution, which could be undesired under specific situations. This work attempts to answer the question regarding invariant convergence for a special class of numerical operations whose dynamical behaviours can be characterized by a quasi-gradient dynamical system. To this end, we study relationship between a gradient dynamical system and its associated quasi-gradient system and reveal the invariance of partial stability region in the quasi-gradient system. These analytical results lead to methods for checking invariant convergence of the trajectory starting from a given point in the quasi-gradient system and the algorithm to maintain invariant convergence. This work also develops new solution methods to enhance TRUST-TECH's capability of solving constrained nonlinear optimization problems and applies them to solve practical problems arising in different applications. Specifically, TRUST-TECH based methods are first developed for feasibility computation and restoration and are applied to power system applications, including power flow computation and feasibility restoration for infeasible optimal power flow problems. Indeed, a unified framework based on TRUST-TECH is introduced for analysing feasibility and infeasibility for nonlinear problems. Secondly, the TRUST-TECH based interior point method (TT-IPM) and the reduced projected gradient method are developed to better tackle constrained nonlinear optimization problems. As application, the TT-IPM method is used to solve mixed-integer nonlinear programs (MINLPs). Finally, this work develops the ensemble of optimal, input-pruned neural networks using TRUST-TECH (ELITE) method for constructing high-quality neural network ensembles and applies ELITE to build a short-term load forecaster named ELITE-STLF with promising performance. Possible extensions of the TRUST-TECH methodology to a much broader range of optimization models, including multi-objective optimization and variational optimization, are suggested for future research efforts

On the relationship between bilevel decomposition algorithms and direct interior-point methods

Engineers have been using bilevel decomposition algorithms to solve certain nonconvex large-scale optimization problems arising in engineering design projects. These algorithms transform the large-scale problem into a bilevel program with one upperlevel problem (the master problem) and several lower-level problems (the subproblems). Unfortunately, there is analytical and numerical evidence that some of these commonly used bilevel decomposition algorithms may fail to converge even when the starting point is very close to the minimizer. In this paper, we establish a relationship between a particular bilevel decomposition algorithm, which only performs one iteration of an interior-point method when solving the subproblems, and a direct interior-point method, which solves the problem in its original (integrated) form. Using this relationship, we formally prove that the bilevel decomposition algorithm converges locally at a superlinear rate. The relevance of our analysis is that it bridges the gap between the incipient local convergence theory of bilevel decomposition algorithms and the mature theory of direct interior-point methods

On affine scaling inexact dogleg methods for bound-constrained nonlinear systems

Within the framework of affine scaling trust-region methods for bound constrained problems, we discuss the use of a inexact dogleg method as a tool for simultaneously handling the trust-region and the bound constraints while seeking for an approximate minimizer of the model. Focusing on bound-constrained systems of nonlinear equations, an inexact affine scaling method for large scale problems, employing the inexact dogleg procedure, is described. Global convergence results are established without any Lipschitz assumption on the Jacobian matrix, and locally fast convergence is shown under standard assumptions. Convergence analysis is performed without specifying the scaling matrix used to handle the bounds, and a rather general class of scaling matrices is allowed in actual algorithms. Numerical results showing the performance of the method are also given

A distributed primal-dual interior-point method for loosely coupled problems using ADMM

In this paper we propose an efficient distributed algorithm for solving loosely coupled convex optimization problems. The algorithm is based on a primal-dual interior-point method in which we use the alternating direction method of multipliers (ADMM) to compute the primal-dual directions at each iteration of the method. This enables us to join the exceptional convergence properties of primal-dual interior-point methods with the remarkable parallelizability of ADMM. The resulting algorithm has superior computational properties with respect to ADMM directly applied to our problem. The amount of computations that needs to be conducted by each computing agent is far less. In particular, the updates for all variables can be expressed in closed form, irrespective of the type of optimization problem. The most expensive computational burden of the algorithm occur in the updates of the primal variables and can be precomputed in each iteration of the interior-point method. We verify and compare our method to ADMM in numerical experiments.Comment: extended version, 50 pages, 9 figure

An interior-point method for mpecs based on strictly feasible relaxations.

An interior-point method for solving mathematical programs with equilibrium constraints (MPECs) is proposed. At each iteration of the algorithm, a single primaldual step is computed from each subproblem of a sequence. Each subproblem is defined as a relaxation of the MPEC with a nonempty strictly feasible region. In contrast to previous approaches, the proposed relaxation scheme preserves the nonempty strict feasibility of each subproblem even in the limit. Local and superlinear convergence of the algorithm is proved even with a less restrictive strict complementarity condition than the standard one. Moreover, mechanisms for inducing global convergence in practice are proposed. Numerical results on the MacMPEC test problem set demonstrate the fast-local convergence properties of the algorithm