Active-set strategy in Powell's method for optimization without derivatives

In this article we present an algorithm for solving bound constrained optimization problems without derivatives based on Powell's method for derivative-free optimization. First we consider the unconstrained optimization problem. At each iteration a quadratic interpolation model of the objective function is constructed around the current iterate and this model is minimized to obtain a new trial point. The whole process is embedded within a trust-region framework. Our algorithm uses infinity norm instead of the Euclidean norm and we solve a box constrained quadratic subproblem using an active-set strategy to explore faces of the box. Therefore, a bound constrained optimization algorithm is easily extended. We compare our implementation with NEWUOA and BOBYQA, Powell's algorithms for unconstrained and bound constrained derivative free optimization respectively. Numerical experiments show that, in general, our algorithm require less functional evaluations than Powell's algorithms. Mathematical subject classification: Primary: 06B10; Secondary: 06D05.Facultad de Ciencias Exacta

Active-set strategy in Powell's method for optimization without derivatives

In this article we present an algorithm for solving bound constrained optimization problems without derivatives based on Powell's method for derivative-free optimization. First we consider the unconstrained optimization problem. At each iteration a quadratic interpolation model of the objective function is constructed around the current iterate and this model is minimized to obtain a new trial point. The whole process is embedded within a trust-region framework. Our algorithm uses infinity norm instead of the Euclidean norm and we solve a box constrained quadratic subproblem using an active-set strategy to explore faces of the box. Therefore, a bound constrained optimization algorithm is easily extended. We compare our implementation with NEWUOA and BOBYQA, Powell's algorithms for unconstrained and bound constrained derivative free optimization respectively. Numerical experiments show that, in general, our algorithm require less functional evaluations than Powell's algorithms. Mathematical subject classification: Primary: 06B10; Secondary: 06D05.Facultad de Ciencias Exacta

Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging

Many graphics and vision problems can be expressed as non-linear least squares optimizations of objective functions over visual data, such as images and meshes. The mathematical descriptions of these functions are extremely concise, but their implementation in real code is tedious, especially when optimized for real-time performance on modern GPUs in interactive applications. In this work, we propose a new language, Opt (available under http://optlang.org), for writing these objective functions over image- or graph-structured unknowns concisely and at a high level. Our compiler automatically transforms these specifications into state-of-the-art GPU solvers based on Gauss-Newton or Levenberg-Marquardt methods. Opt can generate different variations of the solver, so users can easily explore tradeoffs in numerical precision, matrix-free methods, and solver approaches. In our results, we implement a variety of real-world graphics and vision applications. Their energy functions are expressible in tens of lines of code, and produce highly-optimized GPU solver implementations. These solver have performance competitive with the best published hand-tuned, application-specific GPU solvers, and orders of magnitude beyond a general-purpose auto-generated solver

A Derivative Free Optimization method for reservoir characterization inverse problem

International audienceReservoir characterization inverse problem aims at building reservoir models consistent with available production and seismic data for better forecasting of the production of a field. These observed data (pressures, oil/water/gas rates at the wells and 4D seismic data) are compared with simulated data to determine unknown petrophysical properties of the reservoir. The underlying optimization problem is usually formulated as the minimization of a least-squares objective function composed of two terms : the production data and the seismic data mismatch. In practice, this problem is often solved by nonlinear optimization methods, such as Sequential Quadratic Programming methods with derivatives approximated by finite differences. In applications involving 4D seismic data, the use of the classical Gauss-Newton algorithm is often infeasible because the computation of the Jacobian matrix is CPU time consuming and its storage is impossible for large datasets like seismic-related ones. Consequently, this optimization problem requires dedicated techniques: derivatives are not available, the associated forward problems are CPU time consuming and some constraints may be introduced to handle a priori information. We propose a derivative free optimization method under constraints based on trust region approach coupled with local quadratic interpolating models of the cost function and of non linear constraints. Results obtained with this method on a synthetic reservoir application with the joint inversion of production data and 4D seismic data are presented. Its performance is compared with a classical SQP method (quasi-Newton approach based on classical BFGS approximation of the Hessian of the objective function with derivatives approximated by finite differences) in terms of number of simulations of the forward problem