The numerical solution of nonlinear equations representing chemical processes

Imperial Users onl

Historical development of the BFGS secant method and its characterization properties

The BFGS secant method is the preferred secant method for finite-dimensional unconstrained optimization. The first part of this research consists of recounting the historical development of secant methods in general and the BFGS secant method in particular. Many people believe that the secant method arose from Newton's method using finite difference approximations to the derivative. We compile historical evidence revealing that a special case of the secant method predated Newton's method by more than 3000 years. We trace the evolution of secant methods from 18th-century B.C. Babylonian clay tablets and the Egyptian Rhind Papyrus. Modifications to Newton's method yielding secant methods are discussed and methods we believe influenced and led to the construction of the BFGS secant method are explored. In the second part of our research, we examine the construction of several rank-two secant update classes that had not received much recognition in the literature. Our study of the underlying mathematical principles and characterizations inherent in the updates classes led to theorems and their proofs concerning secant updates. One class of symmetric rank-two updates that we investigate is the Dennis class. We demonstrate how it can be derived from the general rank-one update formula in a purely algebraic manner not utilizing Powell's method of iterated projections as Dennis did it. The literature abounds with update classes; we show how some are related and show containment when possible. We derive the general formula that could be used to represent all symmetric rank-two secant updates. From this, particular parameter choices yielding well-known updates and update classes are presented. We include two derivations of the Davidon class and prove that it is a maximal class. We detail known characterization properties of the BFGS secant method and describe new characterizations of several secant update classes known to contain the BFGS update. Included is a formal proof of the conjecture made by Schnabel in his 1977 Ph.D. thesis that the BFGS update is in some asymptotic sense the average of the DFP update and the Greenstadt update

A survey on numerical methods for unconstrained optimization problems.

by Chung Shun Shing.Thesis (M.Phil.)--Chinese University of Hong Kong, 2002.Includes bibliographical references (leaves 158-170).Abstracts in English and Chinese.List of Figures --- p.xChapter 1 --- Introduction --- p.1Chapter 1.1 --- Background and Historical Development --- p.1Chapter 1.2 --- Practical Problems --- p.3Chapter 1.2.1 --- Statistics --- p.3Chapter 1.2.2 --- Aerodynamics --- p.4Chapter 1.2.3 --- Factory Allocation Problem --- p.5Chapter 1.2.4 --- Parameter Problem --- p.5Chapter 1.2.5 --- Chemical Engineering --- p.5Chapter 1.2.6 --- Operational Research --- p.6Chapter 1.2.7 --- Economics --- p.6Chapter 1.3 --- Mathematical Models for Optimization Problems --- p.6Chapter 1.4 --- Unconstrained Optimization Techniques --- p.8Chapter 1.4.1 --- Direct Method - Differential Calculus --- p.8Chapter 1.4.2 --- Iterative Methods --- p.10Chapter 1.5 --- Main Objectives of the Thesis --- p.11Chapter 2 --- Basic Concepts in Optimizations of Smooth Func- tions --- p.14Chapter 2.1 --- Notation --- p.14Chapter 2.2 --- Different Types of Minimizer --- p.16Chapter 2.3 --- Necessary and Sufficient Conditions for Optimality --- p.18Chapter 2.4 --- Quadratic Functions --- p.22Chapter 2.5 --- Convex Functions --- p.24Chapter 2.6 --- "Existence, Uniqueness and Stability of a Minimum" --- p.29Chapter 2.6.1 --- Existence of a Minimum --- p.29Chapter 2.6.2 --- Uniqueness of a Minimum --- p.30Chapter 2.6.3 --- Stability of a Minimum --- p.31Chapter 2.7 --- Types of Convergence --- p.34Chapter 2.8 --- Minimization of Functionals --- p.35Chapter 3 --- Steepest Descent Method --- p.37Chapter 3.1 --- Background --- p.37Chapter 3.2 --- Line Search Method and the Armijo Rule --- p.39Chapter 3.3 --- Steplength Control with Polynomial Models --- p.43Chapter 3.3.1 --- Quadratic Polynomial Model --- p.43Chapter 3.3.2 --- Safeguarding --- p.45Chapter 3.3.3 --- Cubic Polynomial Model --- p.46Chapter 3.3.4 --- General Line Search Strategy --- p.49Chapter 3.3.5 --- Algorithm of Steepest Descent Method --- p.51Chapter 3.4 --- Advantages of the Armijo Rule --- p.54Chapter 3.5 --- Convergence Analysis --- p.56Chapter 4 --- Iterative Methods Using Second Derivatives --- p.63Chapter 4.1 --- Background --- p.63Chapter 4.2 --- Newton's Method --- p.64Chapter 4.2.1 --- Basic Concepts --- p.64Chapter 4.2.2 --- Convergence Analysis of Newton's Method --- p.65Chapter 4.2.3 --- Newton's Method with Steplength --- p.69Chapter 4.2.4 --- Convergence Analysis of Newton's Method with Step-length --- p.70Chapter 4.3 --- Greenstadt's Method --- p.72Chapter 4.4 --- Marquardt-Levenberg Method --- p.74Chapter 4.5 --- Fiacco and McComick Method --- p.76Chapter 4.6 --- Matthews and Davies Method --- p.79Chapter 4.7 --- Numerically Stable Modified Newton's Method --- p.80Chapter 4.8 --- The Role of the Second Derivative Methods --- p.89Chapter 5 --- Multi-step Methods --- p.92Chapter 5.1 --- Background --- p.93Chapter 5.2 --- Heavy Ball Method --- p.94Chapter 5.3 --- Conjugate Gradient Method --- p.99Chapter 5.3.1 --- Some Types of Conjugate Gradient Method --- p.99Chapter 5.3.2 --- Convergence Analysis of Conjugate Gradient Method --- p.108Chapter 5.4 --- Methods of Variable Metric and Methods of Conju- gate Directions --- p.111Chapter 5.5 --- Other Approaches for Constructing the First-order Methods --- p.116Chapter 6 --- Quasi-Newton Methods --- p.121Chapter 6.1 --- Disadvantages of Newton's Method --- p.122Chapter 6.2 --- General Idea of Quasi-Newton Method --- p.124Chapter 6.2.1 --- Quasi-Newton Methods --- p.124Chapter 6.2.2 --- Convergence of Quasi-Newton Methods --- p.129Chapter 6.3 --- Properties of Quasi-Newton Methods --- p.131Chapter 6.4 --- Some Particular Algorithms for Quasi-Newton Methods --- p.137Chapter 6.4.1 --- Single-Rank Algorithms --- p.137Chapter 6.4.2 --- Double-Rank Algorithms --- p.144Chapter 6.4.3 --- Other Applications --- p.149Chapter 6.5 --- Conclusion --- p.152Chapter 7 --- Choice of Methods in Optimization Problems --- p.154Chapter 7.1 --- Choice of Methods --- p.154Chapter 7.2 --- Conclusion --- p.157Bibliography --- p.15

Combination Of The Sequential Secant Method And Broyden's Method With Projected Updates

We introduce a new algorithm for solving nonlinear simultaneous equations, which is a combination of the sequential secant method with Broyden's Quasi-Newton method with projected updates as introduced by Gay and Schnabel. The new algorithm has the order of convergence of the sequential secant method and the choice of the first increments is justified by the minimum variation principles of Quasi-Newton methods. Two versions of the method are compared numerically with some well-known test problems. © 1980 Springer-Verlag.25437938

Newton-type methods under generalized self-concordance and inexact oracles

Many modern applications in machine learning, image/signal processing, and statistics require to solve large-scale convex optimization problems. These problems share some common challenges such as high-dimensionality, nonsmoothness, and complex objectives and constraints. Due to these challenges, the theoretical assumptions for existing numerical methods are not satisfied. In numerical methods, it is also impractical to do exact computations in many cases (e.g. noisy computation, storage or time limitation). Therefore, new approaches as well as inexact computations to design new algorithms should be considered. In this thesis, we develop fundamental theories and numerical methods, especially second-order methods, to solve some classes of convex optimization problems, where first-order methods are inefficient or do not have a theoretical guarantee. We aim at exploiting the underlying smoothness structures of the problem to design novel Newton-type methods. More specifically, we generalize a powerful concept called \mbox{self-concordance} introduced by Nesterov and Nemirovski to a broader class of convex functions. We develop several basic properties of this concept and prove key estimates for function values and its derivatives. Then, we apply our theory to design different Newton-type methods such as damped-step Newton methods, full-step Newton methods, and proximal Newton methods. Our new theory allows us to establish both global and local convergence guarantees of these methods without imposing unverifiable conditions as in classical Newton-type methods. Numerical experiments show that our approach has several advantages compared to existing works. In the second part of this thesis, we introduce new global and local inexact oracle settings, and apply them to develop inexact proximal Newton-type schemes for optimizing general composite convex problems equipped with such inexact oracles. These schemes allow us to measure errors theoretically and systematically and still lead to desired convergence results. Moreover, they can be applied to solve a wider class of applications arising in statistics and machine learning.Doctor of Philosoph

Recent Experiences in Multidisciplinary Analysis and Optimization, part 1

Papers presented at the NASA Symposium on Recent Experiences in Multidisciplinary Analysis and Optimization held at NASA Langley Research Center, Hampton, Virginia April 24 to 26, 1984 are given. The purposes of the symposium were to exchange information about the status of the application of optimization and associated analyses in industry or research laboratories to real life problems and to examine the directions of future developments. Information exchange has encompassed the following: (1) examples of successful applications; (2) attempt and failure examples; (3) identification of potential applications and benefits; (4) synergistic effects of optimized interaction and trade-offs occurring among two or more engineering disciplines and/or subsystems in a system; and (5) traditional organization of a design process as a vehicle for or an impediment to the progress in the design methodology

Robust simulation and optimization methods for natural gas liquefaction processes

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Chemical Engineering, 2018.Cataloged from PDF version of thesis.Includes bibliographical references (pages 313-324).Natural gas is one of the world's leading sources of fuel in terms of both global production and consumption. The abundance of reserves that may be developed at relatively low cost, paired with escalating societal and regulatory pressures to harness low carbon fuels, situates natural gas in a position of growing importance to the global energy landscape. However, the nonuniform distribution of readily-developable natural gas sources around the world necessitates the existence of an international gas market that can serve those regions without reasonable access to reserves. International transmission of natural gas via pipeline is generally cost-prohibitive beyond around two thousand miles, and so suppliers instead turn to the production of liquefied natural gas (LNG) to yield a tradable commodity. While the production of LNG is by no means a new technology, it has not occupied a dominant role in the gas trade to date. However, significant growth in LNG exports has been observed within the last few years, and this trend is expected to continue as major new liquefaction operations have and continue to become operational worldwide. Liquefaction of natural gas is an energy-intensive process requiring specialized cryogenic equipment, and is therefore expensive both in terms of operating and capital costs. However, optimization of liquefaction processes is greatly complicated by the inherently complex thermodynamic behavior of process streams that simultaneously change phase and exchange heat at closely-matched cryogenic temperatures. The determination of optimal conditions for a given process will also generally be nontransferable information between LNG plants, as both the specifics of design (e.g. heat exchanger size and configuration) and the operation (e.g. source gas composition) may have significantly variability between sites. Rigorous evaluation of process concepts for new production facilities is also challenging to perform, as economic objectives must be optimized in the presence of constraints involving equipment size and safety precautions even in the initial design phase. The absence of reliable and versatile software to perform such tasks was the impetus for this thesis project. To address these challenging problems, the aim of this thesis was to develop new models, methods and algorithms for robust liquefaction process simulation and optimization, and to synthesize these advances into reliable and versatile software. Recent advances in the sensitivity analysis of nondifferentiable functions provided an advantageous foundation for the development of physically-informed yet compact process models that could be embedded in established simulation and optimization algorithms with strong convergence properties. Within this framework, a nonsmooth model for the core unit operation in all industrially-relevant liquefaction processes, the multi-stream heat exchanger, was first formulated. The initial multistream heat exchanger model was then augmented to detect and handle internal phase transitions, and an extension of a classic vapor-liquid equilibrium model was proposed to account for the potential existence of solutions in single-phase regimes, all through the use of additional nonsmooth equations. While these initial advances enabled the simulation of liquefaction processes under the conditions of simple, idealized thermodynamic models, it became apparent that these methods would be unable to handle calculations involving nonideal thermophysical property models reliably. To this end, robust nonsmooth extensions of the celebrated inside-out algorithms were developed. These algorithms allow for challenging phase equilibrium calculations to be performed successfully even in the absence of knowledge about the phase regime of the solution, as is the case when model parameters are chosen by a simulation or optimization algorithm. However, this still was not enough to equip realistic liquefaction process models with a completely reliable thermodynamics package, and so new nonsmooth algorithms were designed for the reasonable extrapolation of density from an equation of state under conditions where a given phase does not exist. This procedure greatly enhanced the ability of the nonsmooth inside-out algorithms to converge to physical solutions for mixtures at very high temperature and pressure. These models and submodels were then integrated into a flowsheeting framework to perform realistic simulations of natural gas liquefaction processes robustly, efficiently and with extremely high accuracy. A reliable optimization strategy using an interior-point method and the nonsmooth process models was then developed for complex problem formulations that rigorously minimize thermodynamic irreversibilities. This approach significantly outperforms other strategies proposed in the literature or implemented in commercial software in terms of the ease of initialization, convergence rate and quality of solutions found. The performance observed and results obtained suggest that modeling and optimizing such processes using nondifferentiable models and appropriate sensitivity analysis techniques is a promising new approach to these challenging problems. Indeed, while liquefaction processes motivated this thesis, the majority of the methods described herein are applicable in general to processes with complex thermodynamic or heat transfer considerations embedded. It is conceivable that these models and algorithms could therefore inform a new, robust generation of process simulation and optimization software.by Harry Alexander James Watson.Ph. D