Global optimization method for design problems

In structural design optimization method, numerical techniques are increasingly used. In typical structural optimization problems there may be many locally minimum configurations. For that reason, the application of a global method, which may escape from the locally minimum points, remains essential. In this paper, a new hybrid simulated annealing algorithm for global optimization with constraints is proposed. We have developed a new algorithm called Adaptive Simulated Annealing Penalty Simultaneous Perturbation Stochastic Approximation algorithm (ASAPSPSA) that uses Adaptive Simulated Annealing algorithm (ASA); ASA is a series of modifications done to the traditional simulated annealing algorithm that gives the global solution of an objective function. In addition, the stochastic method Simultaneous Perturbation Stochastic Approximation (SPSA) for solving unconstrained optimization problems is used to refine the solution. We also propose Penalty SPSA (PSPSA) for solving constrained optimization problems. The constraints are handled using exterior point penalty functions. The hybridization of both techniques ASA and PSPSA provides a powerful hybrid heuristic optimization method; the proposed method is applicable to any problem where the topology of the structure is not fixed; it is simple and capable of handling problems subject to any number of nonlinear constraints. Extensive tests on the ASAPSPSA as a global optimization method are presented; its performance as a viable optimization method is demonstrated by applying it first to a series of benchmark functions with 2 - 50 dimensions and then it is used in structural design to demonstrate its applicability and efficiency

An inertial ADMM for a class of nonconvex composite optimization with nonlinear coupling constraints

In this paper, we propose an inertial alternating direction method of multipliers for solving a class of non-convex multi-block optimization problems with \emph{nonlinear coupling constraints}. Distinctive features of our proposed method, when compared with other alternating direction methods of multipliers for solving non-convex problems with nonlinear coupling constraints, include: (i) we apply the inertial technique to the update of primal variables and (ii) we apply a non-standard update rule for the multiplier by scaling the multiplier by a factor before moving along the ascent direction where a relaxation parameter is allowed. Subsequential convergence and global convergence are presented for the proposed algorithm

A smoothing SQP method for nonlinear programs with stability constraints arising from power systems

This paper investigates a new class of optimization problems arising from power systems, known as nonlinear programs with stability constraints (NPSC), which is an extension of ordinary nonlinear programs. Since the stability constraint is described generally by eigenvalues or norm of Jacobian matrices of systems, this results in the semismooth property of NPSC problems. The optimal conditions of both NPSC and its smoothing problem are studied. A smoothing SQP algorithm is proposed for solving such optimization problem. The global convergence of algorithm is established. A numerical example from optimal power flow (OPF) is done. The computational results show efficiency of the new model and algorithm. © The Author(s) 2010.published_or_final_versionSpringer Open Choice, 21 Feb 201

Partially distributed outer approximation

This paper presents a novel partially distributed outer approximation algorithm, named PaDOA, for solving a class of structured mixed integer convex programming problems to global optimality. The proposed scheme uses an iterative outer approximation method for coupled mixed integer optimization problems with separable convex objective functions, affine coupling constraints, and compact domain. PaDOA proceeds by alternating between solving large-scale structured mixed-integer linear programming problems and partially decoupled mixed-integer nonlinear programming subproblems that comprise much fewer integer variables. We establish conditions under which PaDOA converges to global minimizers after a finite number of iterations and verify these properties with an application to thermostatically controlled loads and to mixed-integer regression

An improved ant system algorithm for maximizing system reliability in the compatible module

This paper presents an improved Ant System (AS) algorithm called AS-2Swap for solving one of the reliability optimization problems. The objective is to selection a compatible module in order to maximize the system reliability and subject to budget constraints. This problem is NP-hard and formulated as a binary integer-programming problem with a nonlinear objective function. The proposed algorithm is based on the original AS algorithm and the improvement, focused on choosing the feasible solutions, neighborhood search with Swap technique for each loop of finding the solution. The implementation was tested by the five groups of data sets from the existing meta-heuristic found in the literature. The computational results show that the proposed algorithm can find the global optimal solution and is more accurate for larger problems

Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems

We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic equality constraints. We consider a fully stochastic setting, where at each step a single sample is generated to estimate the objective gradient. The algorithm adaptively selects the trust-region radius and, compared to the existing line-search StoSQP schemes, allows us to utilize indefinite Hessian matrices (i.e., Hessians without modification) in SQP subproblems. As a trust-region method for constrained optimization, our algorithm must address an infeasibility issue -- the linearized equality constraints and trust-region constraints may lead to infeasible SQP subproblems. In this regard, we propose an adaptive relaxation technique to compute the trial step, consisting of a normal step and a tangential step. To control the lengths of these two steps while ensuring a scale-invariant property, we adaptively decompose the trust-region radius into two segments, based on the proportions of the rescaled feasibility and optimality residuals to the rescaled full KKT residual. The normal step has a closed form, while the tangential step is obtained by solving a trust-region subproblem, to which a solution ensuring the Cauchy reduction is sufficient for our study. We establish a global almost sure convergence guarantee for TR-StoSQP, and illustrate its empirical performance on both a subset of problems in the CUTEst test set and constrained logistic regression problems using data from the LIBSVM collection.Comment: 10 figures, 33 page

Competitive Mirror Descent

Constrained competitive optimization involves multiple agents trying to minimize conflicting objectives, subject to constraints. This is a highly expressive modeling language that subsumes most of modern machine learning. In this work we propose competitive mirror descent (CMD): a general method for solving such problems based on first order information that can be obtained by automatic differentiation. First, by adding Lagrange multipliers, we obtain a simplified constraint set with an associated Bregman potential. At each iteration, we then solve for the Nash equilibrium of a regularized bilinear approximation of the full problem to obtain a direction of movement of the agents. Finally, we obtain the next iterate by following this direction according to the dual geometry induced by the Bregman potential. By using the dual geometry we obtain feasible iterates despite only solving a linear system at each iteration, eliminating the need for projection steps while still accounting for the global nonlinear structure of the constraint set. As a special case we obtain a novel competitive multiplicative weights algorithm for problems on the positive cone