Projection based algorithms for variational inequalities

This dissertation is about the theory and iterative algorithms for solving variational inequalities. Chapter 1 introduces the problem, various situations in which variational inequalities arise naturally, reformulations of the problem, several characteristics of the problem based on those reformulations, as well as the basic existence and uniqueness results. Following that, chapter 2 describes the general approaches to solving variational inequalities, focusing on projection based methods towards the end, with some convergence results. That chapter also discusses the merits and demerits of those approaches. In chapter 3, we describe a relaxed projection method, and a descent method for solving variational inequalities with some examples. An application of the descent framework to a game theory problem leads to an algorithm for solving box constrained variational inequalities. Relaxed projection methods require a sequence of parameters that approach zero, which leads to slow convergence as the iterates approach a solution. Chapter 4 describes a local convergence result that can be used as a guideline for finding a bound on the parameter as a relaxed projection algorithm reaches a solution

Forward and inverse American option pricing via a complementarity approach

This dissertation considers three topics. The first part discusses the pricing of American options under a local volatility model and two jump diffusion models: Kou's jump diffusion model and the Dupire system. In Chapter 2, we establish partial differential complementarity systems for pricing American options under the aforementioned three models. We also introduce two different discretization schemes, a finite difference method and a finite element method, for the discretization of the complementarity systems into a collection of linear complementarity problems (LCPs). In Chapter 3, we discuss four popular existing numerical algorithms---a PSOR method, a two phase active-set method, a semi-smooth Newton method and a pivoting method---for solving LCPs that arise under Kou's jump diffusion model and the Dupire system. The numerical results presented in the thesis summarize the effectiveness of each approach for solving the corresponding LCPs. %In particular, we are interested in the numerical evaluation of four algorithms pricing these options: a PSOR method, a two-phase active-set method, a semi-smooth Newton method, and a parametric pivoting method. In the second part, we consider the calibration problems of computing an implied volatility parameter for American options under the Dupire system and the local volatility model. In Chapter 4, we formulate the calibration problem as an inverse problem of the forward pricing problem, which is modeled as a discretized partial differential linear complementarity system in Chapter 2. The resulting inverse problem then becomes an instance of a mathematical program with complementarity constraints (MPCC). Two methods for solving MPCCs, an implicit programming algorithm (IMPA) and a new hybrid algorithm, are studied in this dissertation. We test both algorithms and report their numerical performance for solving MPCCs derived under the Dupire system and the local volatility model with synthetic and market data. In the third part of this thesis, we investigate a new class of MPCCs, a doubly uni-parametric MPCC, for which the calibration of American options under the Black-Scholes-Merton (BSM) model is a special case. In particular, we consider one new algorithm for solving this problem when the problem matrices are positive definite, and a second algorithm for the more general case when the matrices are merely positive semi-definite. We study the convergence of both algorithms based on the local stability of the solutions as well as the numerical performance of both algorithms for solving doubly uni-parametric MPCCs with tridiagonal matrices, which are applicable for the calibration problems under the BSM model

Complementarity and related problems

In this thesis, we present results related to complementarity problems. We study the linear complementarity problems on extended second order cones. We convert a linear complementarity problem on an extended second order cone into a mixed complementarity problem on the non-negative orthant. We present algorithms for this problem, and exemplify it by a numerical example. Following this result, we explore the stochastic version of this linear complementarity problem. Finally, we apply complementarity problems on extended second order cones in a portfolio optimisation problem. In this application, we exploit our theoretical results to find an analytical solution to a new portfolio optimisation model. We also study the spherical quasi-convexity of quadratic functions on spherically self-dual convex sets. We start this study by exploring the characterisations and conditions for the spherical positive orthant. We present several conditions characterising the spherical quasi-convexity of quadratic functions. Then we generalise the conditions to the spherical quasi-convexity on spherically self-dual convex sets. In particular, we highlight the case of spherical second order cones

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more

Convex Optimization and Extensions, with a View Toward Large-Scale Problems

Machine learning is a major source of interesting optimization problems of current interest. These problems tend to be challenging because of their enormous scale, which makes it difficult to apply traditional optimization algorithms. We explore three avenues to designing algorithms suited to handling these challenges, with a view toward large-scale ML tasks. The first is to develop better general methods for unconstrained minimization. The second is to tailor methods to the features of modern systems, namely the availability of distributed computing. The third is to use specialized algorithms to exploit specific problem structure. Chapters 2 and 3 focus on improving quasi-Newton methods, a mainstay of unconstrained optimization. In Chapter 2, we analyze an extension of quasi-Newton methods wherein we use block updates, which add curvature information to the Hessian approximation on a higher-dimensional subspace. This defines a family of methods, Block BFGS, that form a spectrum between the classical BFGS method and Newton's method, in terms of the amount of curvature information used. We show that by adding a correction step, the Block BFGS method inherits the convergence guarantees of BFGS for deterministic problems, most notably a Q-superlinear convergence rate for strongly convex problems. To explore the tradeoff between reduced iterations and greater work per iteration of block methods, we present a set of numerical experiments. In Chapter 3, we focus on the problem of step size determination. To obviate the need for line searches, and for pre-computing fixed step sizes, we derive an analytic step size, which we call curvature-adaptive, for self-concordant functions. This adaptive step size allows us to generalize the damped Newton method of Nesterov to other iterative methods, including gradient descent and quasi-Newton methods. We provide simple proofs of convergence, including superlinear convergence for adaptive BFGS, allowing us to obtain superlinear convergence without line searches. In Chapter 4, we move from general algorithms to hardware-influenced algorithms. We consider a form of distributed stochastic gradient descent that we call Leader SGD, which is inspired by the Elastic Averaging SGD method. These methods are intended for distributed settings where communication between machines may be expensive, making it important to set their consensus mechanism. We show that LSGD avoids an issue with spurious stationary points that affects EASGD, and provide a convergence analysis of LSGD. In the stochastic strongly convex setting, LSGD converges at the rate O(1/k) with diminishing step sizes, matching other distributed methods. We also analyze the impact of varying communication delays, stochasticity in the selection of the leader points, and under what conditions LSGD may produce better search directions than the gradient alone. In Chapter 5, we switch again to focus on algorithms to exploit problem structure. Specifically, we consider problems where variables satisfy multiaffine constraints, which motivates us to apply the Alternating Direction Method of Multipliers (ADMM). Problems that can be formulated with such a structure include representation learning (e.g with dictionaries) and deep learning. We show that ADMM can be applied directly to multiaffine problems. By extending the theory of nonconvex ADMM, we prove that ADMM is convergent on multiaffine problems satisfying certain assumptions, and more broadly, analyze the theoretical properties of ADMM for general problems, investigating the effect of different types of structure

A generic interior-point framework for nonsmooth and complementarity constrained nonlinear optimization

[no abstract

On deep learning in physics

Machine learning, and most notably deep neural networks, have seen unprecedented success in recent years due to their ability to learn complex nonlinear mappings by ingesting large amounts of data through the process of training. This learning-by-example approach has slowly made its way into the physical sciences in recent years. In this dissertation I present a collection of contributions at the intersection of the fields of physics and deep learning. These contributions constitute some of the earlier introductions of deep learning to the physical sciences, and comprises a range of machine learning techniques, such as feed forward neural networks, generative models, and reinforcement learning. A focus will be placed on the lessons and techniques learned along the way that would influence future research projects

Numerical Analysis of Algorithms for Infinitesimal Associated and Non-Associated Elasto-Plasticity

The thesis studies nonlinear solution algorithms for problems in infinitesimal elastoplasticity and their numerical realization within a parallel computing framework. New algorithms like Active Set and Augmented Lagrangian methods are proposed and analyzed within a semismooth Newton setting. The analysis is often carried out in function space which results in stable algorithms. Large scale computer experiments demonstrate the efficiency of the new algorithms

On deep learning in physics

Machine learning, and most notably deep neural networks, have seen unprecedented success in recent years due to their ability to learn complex nonlinear mappings by ingesting large amounts of data through the process of training. This learning-by-example approach has slowly made its way into the physical sciences in recent years. In this dissertation I present a collection of contributions at the intersection of the fields of physics and deep learning. These contributions constitute some of the earlier introductions of deep learning to the physical sciences, and comprises a range of machine learning techniques, such as feed forward neural networks, generative models, and reinforcement learning. A focus will be placed on the lessons and techniques learned along the way that would influence future research projects

A note on the Lipschitz continuity of the gradient of the squared norm of the matrix-valued Fischer-Burmeister function

10.1007/s10107-005-0697-xMathematical Programming1073547-55