Derivative-free optimization methods

In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints

Exact determination of the volume of an inclusion in a body having constant shear modulus

We derive an exact formula for the volume fraction of an inclusion in a body when the inclusion and the body are linearly elastic materials with the same shear modulus. Our formula depends on an appropriate measurement of the displacement and traction around the boundary of the body. In particular, the boundary conditions around the boundary of the body must be such that they mimic the body being placed in an infinite medium with an appropriate displacement applied at infinity

Suitable Spaces for Shape Optimization

The differential-geometric structure of certain shape spaces is investigated and applied to the theory of shape optimization problems constrained by partial differential equations and variational inequalities. Furthermore, we define a diffeological structure on a new space of so-called $H^{1/2}$-shapes. This can be seen as a first step towards the formulation of optimization techniques on diffeological spaces. The $H^{1/2}$-shapes are a generalization of smooth shapes and arise naturally in shape optimization problems

Numerical method for the time-fractional porous medium equation

This papers deals with a construction and convergence analysis of a finite difference scheme for solving time-fractional porous medium equation. The governing equation exhibits both nonlocal and nonlinear behaviour making the numerical computations challenging. Our strategy is to reduce the problem into a single one-dimensional Volterra integral equation for the self-similar solution and then to apply the discretization. The main difficulty arises due to the non-Lipschitzian behaviour of the equation's nonlinearity. By the analysis of the recurrence relation for the error we are able to prove that there exists a family of finite difference methods that is convergent for a large subset of the parameter space. We illustrate our results with a concrete example of a method based on the midpoint quadrature

Concave Quadratic Cuts for Mixed-Integer Quadratic Problems

The technique of semidefinite programming (SDP) relaxation can be used to obtain a nontrivial bound on the optimal value of a nonconvex quadratically constrained quadratic program (QCQP). We explore concave quadratic inequalities that hold for any vector in the integer lattice ${\bf Z}^n$, and show that adding these inequalities to a mixed-integer nonconvex QCQP can improve the SDP-based bound on the optimal value. This scheme is tested using several numerical problem instances of the max-cut problem and the integer least squares problem.Comment: 24 pages, 1 figur

A Multiscale Method for Model Order Reduction in PDE Parameter Estimation

Estimating parameters of Partial Differential Equations (PDEs) is of interest in a number of applications such as geophysical and medical imaging. Parameter estimation is commonly phrased as a PDE-constrained optimization problem that can be solved iteratively using gradient-based optimization. A computational bottleneck in such approaches is that the underlying PDEs needs to be solved numerous times before the model is reconstructed with sufficient accuracy. One way to reduce this computational burden is by using Model Order Reduction (MOR) techniques such as the Multiscale Finite Volume Method (MSFV). In this paper, we apply MSFV for solving high-dimensional parameter estimation problems. Given a finite volume discretization of the PDE on a fine mesh, the MSFV method reduces the problem size by computing a parameter-dependent projection onto a nested coarse mesh. A novelty in our work is the integration of MSFV into a PDE-constrained optimization framework, which updates the reduced space in each iteration. We also present a computationally tractable way of differentiating the MOR solution that acknowledges the change of basis. As we demonstrate in our numerical experiments, our method leads to computational savings particularly for large-scale parameter estimation problems and can benefit from parallelization.Comment: 22 pages, 4 figures, 3 table

Projection Methods: An Annotated Bibliography of Books and Reviews

Projections onto sets are used in a wide variety of methods in optimization theory but not every method that uses projections really belongs to the class of projection methods as we mean it here. Here projection methods are iterative algorithms that use projections onto sets while relying on the general principle that when a family of (usually closed and convex) sets is present then projections (or approximate projections) onto the given individual sets are easier to perform than projections onto other sets (intersections, image sets under some transformation, etc.) that are derived from the given family of individual sets. Projection methods employ projections (or approximate projections) onto convex sets in various ways. They may use different kinds of projections and, sometimes, even use different projections within the same algorithm. They serve to solve a variety of problems which are either of the feasibility or the optimization types. They have different algorithmic structures, of which some are particularly suitable for parallel computing, and they demonstrate nice convergence properties and/or good initial behavior patterns. This class of algorithms has witnessed great progress in recent years and its member algorithms have been applied with success to many scientific, technological, and mathematical problems. This annotated bibliography includes books and review papers on, or related to, projection methods that we know about, use, and like. If you know of books or review papers that should be added to this list please contact us.Comment: Revised version. Accepted for publication in the journal "Optimization

An Integral Equation Method for the Cahn-Hilliard Equation in the Wetting Problem

We present an integral equation approach to solving the Cahn-Hilliard equation equipped with boundary conditions that model solid surfaces with prescribed Young's angles. The discretization of the system in time using convex splitting leads to a modified biharmonic equation at each time step. To solve it, we split the solution into a volume potential computed with free space kernels, plus the solution to a second kind integral equation (SKIE). The volume potential is evaluated with the help of a box-based volume-FMM method. For non-box domains, source density is extended by solving a biharmonic Dirichlet problem. The near-singular boundary integrals are computed using quadrature by expansion (QBX) with FMM acceleration. Our method has linear complexity in the number of surface/volume degrees of freedom and can achieve high order convergence with adaptive refinement to manage error from function extension

The Askey-scheme of hypergeometric orthogonal polynomials and its q-analogue

We list the so-called Askey-scheme of hypergeometric orthogonal polynomials. In chapter 1 we give the definition, the orthogonality relation, the three term recurrence relation and generating functions of all classes of orthogonal polynomials in this scheme. In chapeter 2 we give all limit relation between different classes of orthogonal polynomials listed in the Askey-scheme. In chapter 3 we list the q-analogues of the polynomials in the Askey-scheme. We give their definition, orthogonality relation, three term recurrence relation and generating functions. In chapter 4 we give the limit relations between those basic hypergeometric orthogonal polynomials. Finally in chapter 5 we point out how the `classical` hypergeometric orthogonal polynomials of the Askey-scheme can be obtained from their q-analogues

A Conservative Flux Optimization Finite Element Method for Convection-Diffusion Equations

This article presents a new finite element method for convection-diffusion equations by enhancing the continuous finite element space with a flux space for flux approximations that preserve the important mass conservation locally on each element. The numerical scheme is based on a constrained flux optimization approach where the constraint was given by local mass conservation equations and the flux error is minimized in a prescribed topology/metric. This new scheme provides numerical approximations for both the primal and the flux variables. It is shown that the numerical approximations for the primal and the flux variables are convergent with optimal order in some discrete Sobolev norms. Numerical experiments are conducted to confirm the convergence theory. Furthermore, the new scheme was employed in the computational simulation of a simplified two-phase flow problem in highly heterogeneous porous media. The numerical results illustrate an excellent performance of the method in scientific computing