Newton-MR: Inexact Newton Method With Minimum Residual Sub-problem Solver

We consider a variant of inexact Newton Method, called Newton-MR, in which the least-squares sub-problems are solved approximately using Minimum Residual method. By construction, Newton-MR can be readily applied for unconstrained optimization of a class of non-convex problems known as invex, which subsumes convexity as a sub-class. For invex optimization, instead of the classical Lipschitz continuity assumptions on gradient and Hessian, Newton-MR's global convergence can be guaranteed under a weaker notion of joint regularity of Hessian and gradient. We also obtain Newton-MR's problem-independent local convergence to the set of minima. We show that fast local/global convergence can be guaranteed under a novel inexactness condition, which, to our knowledge, is much weaker than the prior related works. Numerical results demonstrate the performance of Newton-MR as compared with several other Newton-type alternatives on a few machine learning problems.Comment: 35 page

A new logarithmic penalty function approach for nonlinear constrained optimization problem

This paper presents a new penalty function called logarithmic penalty function (LPF) and examines the convergence of the proposed LPF method. Furthermore, the LaGrange multiplier for equality constrained optimization is derived based on the first-order necessary condition. The proposed LPF belongs to both categories: a classical penalty function and an exact penalty function, depending on the choice of penalty parameter. Moreover, the proposed LPF is capable of dealing with some of the problems with irregular features from Hock-Schittkowski collections of test problems

New Exact Penalty Functions for Nonlinear Constrained Optimization Problems

For two kinds of nonlinear constrained optimization problems, we propose two simple penalty functions, respectively, by augmenting the dimension of the primal problem with a variable that controls the weight of the penalty terms. Both of the penalty functions enjoy improved smoothness. Under mild conditions, it can be proved that our penalty functions are both exact in the sense that local minimizers of the associated penalty problem are precisely the local minimizers of the original constrained problem

Symbolic approaches and artificial intelligence algorithms for solving multi-objective optimisation problems

Problems that have more than one objective function are of great importance in engineering sciences and many other disciplines. This class of problems are known as multi-objective optimisation problems (or multicriteria). The difficulty here lies in the conflict between the various objective functions. Due to this conflict, one cannot find a single ideal solution which simultaneously satisfies all the objectives. But instead one can find the set of Pareto-optimal solutions (Pareto-optimal set) and consequently the Pareto-optimal front is established. Finding these solutions plays an important role in multi-objective optimisation problems and mathematically the problem is considered to be solved when the Pareto-optimal set, i.e. the set of all compromise solutions is found. The Pareto-optimal set may contain information that can help the designer make a decision and thus arrive at better trade-off solutions. The aim of this research is to develop new multi-objective optimisation symbolic algorithms capable of detecting relationship(s) among decision variables that can be used for constructing the analytical formula of Pareto-optimal front based on the extension of the current optimality conditions. A literature survey of theoretical and evolutionary computation techniques for handling multiple objectives, constraints and variable interaction highlights a lack of techniques to handle variable interaction. This research, therefore, focuses on the development of techniques for detecting the relationships between the decision variables (variable interaction) in the presence of multiple objectives and constraints. It attempts to fill the gap in this research by formally extending the theoretical results (optimality conditions). The research then proposes first-order multi-objective symbolic algorithm or MOSA-I and second-order multi-objective symbolic algorithm or MOSA-II that are capable of detecting the variable interaction. The performance of these algorithms is analysed and compared to a current state-of-the-art optimisation algorithm using popular test problems. The performance of the MOSA-II algorithm is finally validated using three appropriately chosen problems from literature. In this way, this research proposes a fully tested and validated methodology for dealing with multi-objective optimisation problems. In conclusion, this research proposes two new symbolic algorithms that are used for identifying the variable interaction responsible for constructing Pareto-optimal front among objectives in multi-objective optimisation problems. This is completed based on a development and relaxation of the first and second-order optimality conditions of Karush-Kuhn-Tucker.EThOS - Electronic Theses Online ServiceGBUnited Kingdo