Derivative-free optimization methods

In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints

On the optimal control of some nonsmooth distributed parameter systems arising in mechanics

Variational inequalities are an important mathematical tool for modelling free boundary problems that arise in different application areas. Due to the intricate nonsmooth structure of the resulting models, their analysis and optimization is a difficult task that has drawn the attention of researchers for several decades. In this paper we focus on a class of variational inequalities, called of the second kind, with a twofold purpose. First, we aim at giving a glance at some of the most prominent applications of these types of variational inequalities in mechanics, and the related analytical and numerical difficulties. Second, we consider optimal control problems constrained by these variational inequalities and provide a thorough discussion on the existence of Lagrange multipliers and the different types of optimality systems that can be derived for the characterization of local minima. The article ends with a discussion of the main challenges and future perspectives of this important problem class

Exact determination of the volume of an inclusion in a body having constant shear modulus

We derive an exact formula for the volume fraction of an inclusion in a body when the inclusion and the body are linearly elastic materials with the same shear modulus. Our formula depends on an appropriate measurement of the displacement and traction around the boundary of the body. In particular, the boundary conditions around the boundary of the body must be such that they mimic the body being placed in an infinite medium with an appropriate displacement applied at infinity

The Askey-scheme of hypergeometric orthogonal polynomials and its q-analogue

We list the so-called Askey-scheme of hypergeometric orthogonal polynomials. In chapter 1 we give the definition, the orthogonality relation, the three term recurrence relation and generating functions of all classes of orthogonal polynomials in this scheme. In chapeter 2 we give all limit relation between different classes of orthogonal polynomials listed in the Askey-scheme. In chapter 3 we list the q-analogues of the polynomials in the Askey-scheme. We give their definition, orthogonality relation, three term recurrence relation and generating functions. In chapter 4 we give the limit relations between those basic hypergeometric orthogonal polynomials. Finally in chapter 5 we point out how the `classical` hypergeometric orthogonal polynomials of the Askey-scheme can be obtained from their q-analogues

A Multiscale Method for Model Order Reduction in PDE Parameter Estimation

Estimating parameters of Partial Differential Equations (PDEs) is of interest in a number of applications such as geophysical and medical imaging. Parameter estimation is commonly phrased as a PDE-constrained optimization problem that can be solved iteratively using gradient-based optimization. A computational bottleneck in such approaches is that the underlying PDEs needs to be solved numerous times before the model is reconstructed with sufficient accuracy. One way to reduce this computational burden is by using Model Order Reduction (MOR) techniques such as the Multiscale Finite Volume Method (MSFV). In this paper, we apply MSFV for solving high-dimensional parameter estimation problems. Given a finite volume discretization of the PDE on a fine mesh, the MSFV method reduces the problem size by computing a parameter-dependent projection onto a nested coarse mesh. A novelty in our work is the integration of MSFV into a PDE-constrained optimization framework, which updates the reduced space in each iteration. We also present a computationally tractable way of differentiating the MOR solution that acknowledges the change of basis. As we demonstrate in our numerical experiments, our method leads to computational savings particularly for large-scale parameter estimation problems and can benefit from parallelization.Comment: 22 pages, 4 figures, 3 table

A stroll in the jungle of error bounds

The aim of this paper is to give a short overview on error bounds and to provide the first bricks of a unified theory. Inspired by the works of [8, 15, 13, 16, 10], we show indeed the centrality of the Lojasiewicz gradient inequality. For this, we review some necessary and sufficient conditions for global/local error bounds, both in the convex and nonconvex case. We also recall some results on quantitative error bounds which play a major role in convergence rate analysis and complexity theory of many optimization methods

An Integral Equation Method for the Cahn-Hilliard Equation in the Wetting Problem

We present an integral equation approach to solving the Cahn-Hilliard equation equipped with boundary conditions that model solid surfaces with prescribed Young's angles. The discretization of the system in time using convex splitting leads to a modified biharmonic equation at each time step. To solve it, we split the solution into a volume potential computed with free space kernels, plus the solution to a second kind integral equation (SKIE). The volume potential is evaluated with the help of a box-based volume-FMM method. For non-box domains, source density is extended by solving a biharmonic Dirichlet problem. The near-singular boundary integrals are computed using quadrature by expansion (QBX) with FMM acceleration. Our method has linear complexity in the number of surface/volume degrees of freedom and can achieve high order convergence with adaptive refinement to manage error from function extension

Identification of a chemotactic sensitivity in a coupled system

Chemotaxis is the process by which cells behave in a way that follows the chemical gradient. Applications to bacteria growth, tissue inflammation, and vascular tumors provide a focus on optimization strategies. Experiments can characterize the form of possible chemotactic sensitivities. This paper addresses the recovery of the chemotactic sensitivity from these experiments while allowing for nonlinear dependence of the parameter on the state variables. The existence of solutions to the forward problem is analyzed. The identification of a chemotactic parameter is determined by inverse problem techniques. Tikhonov regularization is investigated and appropriate convergence results are obtained. Numerical results of concentration dependent chemotactic terms are explored

Recent Advances in Denoising of Manifold-Valued Images

Modern signal and image acquisition systems are able to capture data that is no longer real-valued, but may take values on a manifold. However, whenever measurements are taken, no matter whether manifold-valued or not, there occur tiny inaccuracies, which result in noisy data. In this chapter, we review recent advances in denoising of manifold-valued signals and images, where we restrict our attention to variational models and appropriate minimization algorithms. The algorithms are either classical as the subgradient algorithm or generalizations of the half-quadratic minimization method, the cyclic proximal point algorithm, and the Douglas-Rachford algorithm to manifolds. An important aspect when dealing with real-world data is the practical implementation. Here several groups provide software and toolboxes as the Manifold Optimization (Manopt) package and the manifold-valued image restoration toolbox (MVIRT)

Projection Methods: An Annotated Bibliography of Books and Reviews

Projections onto sets are used in a wide variety of methods in optimization theory but not every method that uses projections really belongs to the class of projection methods as we mean it here. Here projection methods are iterative algorithms that use projections onto sets while relying on the general principle that when a family of (usually closed and convex) sets is present then projections (or approximate projections) onto the given individual sets are easier to perform than projections onto other sets (intersections, image sets under some transformation, etc.) that are derived from the given family of individual sets. Projection methods employ projections (or approximate projections) onto convex sets in various ways. They may use different kinds of projections and, sometimes, even use different projections within the same algorithm. They serve to solve a variety of problems which are either of the feasibility or the optimization types. They have different algorithmic structures, of which some are particularly suitable for parallel computing, and they demonstrate nice convergence properties and/or good initial behavior patterns. This class of algorithms has witnessed great progress in recent years and its member algorithms have been applied with success to many scientific, technological, and mathematical problems. This annotated bibliography includes books and review papers on, or related to, projection methods that we know about, use, and like. If you know of books or review papers that should be added to this list please contact us.Comment: Revised version. Accepted for publication in the journal "Optimization