Convergence of the restricted Nelder-Mead algorithm in two dimensions

The Nelder-Mead algorithm, a longstanding direct search method for unconstrained optimization published in 1965, is designed to minimize a scalar-valued function f of n real variables using only function values, without any derivative information. Each Nelder-Mead iteration is associated with a nondegenerate simplex defined by n+1 vertices and their function values; a typical iteration produces a new simplex by replacing the worst vertex by a new point. Despite the method's widespread use, theoretical results have been limited: for strictly convex objective functions of one variable with bounded level sets, the algorithm always converges to the minimizer; for such functions of two variables, the diameter of the simplex converges to zero, but examples constructed by McKinnon show that the algorithm may converge to a nonminimizing point. This paper considers the restricted Nelder-Mead algorithm, a variant that does not allow expansion steps. In two dimensions we show that, for any nondegenerate starting simplex and any twice-continuously differentiable function with positive definite Hessian and bounded level sets, the algorithm always converges to the minimizer. The proof is based on treating the method as a discrete dynamical system, and relies on several techniques that are non-standard in convergence proofs for unconstrained optimization.Comment: 27 page

Sheet-metal press line parameter tuning using a combined DIRECT and Nelder-Mead algorithm

It is a great challenge to obtain an efficient algorithm for global optimisation of nonlinear, nonconvex and high dimensional objective functions. This paper shows how the combination of DIRECT and Nelder-Mead algorithms can improve the efficiency in the parameter tuning of a sheet-metal press line. A combined optimisation algorithm is proposed that determines and utilises all local optimal points from DIRECT algorithm as Nelder-Mead starting points. To reduce the total optimisation time, all Nelder-Mead optimisations can be executed in parallel. Additionally, a Collision Inspection Method is implemented in the simulation model to reduce the evaluation time. Altogether, this results in an industrially useful parameter tuning method. Improvements of an increased production rate of 7% and 40% smoother robot motions have been achieved

An extension of Nelder-Mead method to nonlinear mixed-integer optimization problems

Este artículo presenta un algoritmo nuevo basado en una extensión del método algorítmico simplex de Nelder Mead para la identificación de al menos un óptimo local cuando se usa en problemas no lineales enteros mixtos irrestrictos. Este método algorítmico, denominado Algoritmo Simplex Entero Mixto (ASEM) por el autor, se basa en una doble estructura de símpleces que está compuesta por una estructura simplex real de dimensión n (simplex real) y otra estructura simplex entera de igual dimensión n (simplex entero). Las operaciones propuestas originalmente por Nelder Mead se ejecutan sobre el simplex real. Por otra parte, un grupo de nuevas operaciones presentadas en este artículo se emplean en el simplex entero. Este conjunto de nuevas operaciones junto con las operaciones originales del método de Nelder Mead garantizan que en cada iteración del ASEM se arroje un nuevo punto de prueba definido en el campo numérico de dimensión 2n enteros mixtos {\textstyle {\mathbb {R} }^{n}\times {\mathbb {Z} }^{n}} , con el objetivo de garantizar la identificación del óptimo local entero mixto, sin convertir las variables enteras en variables reales.This article presents a new algorithm based on the Nelder-Mead simplex algorithmic method for identifying a local optimum, at least on unconstrained nonlinear mixed-integer problems. The algorithmic method, Integer Mixed Simplex Algorithm (IMSA), so called by the author, is based on a double simplex structure, which is composed of a real n -dimensional simplex structure (real simplex) and an integer n -dimensional simplex structure (integer simplex). The original Nelder-Mead operations are applied on the real simplex. Meanwhile, a novel group of operations are applied on the integer simplex. This new set of operations, together with the original Nelder-Mead operations, guarantee a new trail point at each IMSA iteration in the search of the local optimum in the integer real mixed 2n -dimensional numerical field {\textstyle {\mathbb {R} }^{n}\times {\mathbb {Z} }^{n}} without the need of integer to real conversions.Peer Reviewe

Sensitivity analysis of permeability parameters of bovine nucleus pulposus obtained through inverse fitting of the nonlinear biphasic equation : effect of sampling strategy

Permeability controls the fluid flow into and out of soft tissue, and plays an important role in maintaining the health status of such tissue. Accurate determination of the parameters that define permeability is important for the interpretation of models that incorporate such processes. This paper describes the determination of strain-dependent permeability parameters from the nonlinear biphasic equation from experimental data of different sampling frequencies using the Nelder–Mead simplex method. The ability of this method to determine the global optimum was assessed by constructing the whole manifold arising from possible parameter combinations. Many parameter combinations yielded similar fits with the Nelder–Mead algorithm able to identify the global maximum within the resolution of the manifold. Furthermore, the sampling strategy affected the optimum values of the permeability parameters. Therefore, permeability parameter estimations arising from inverse methods should be utilised with the knowledge that they come with large confidence intervals

Adaptive extensions of the Nelder and Mead Simplex Method for optimization of stochastic simulation models

We consider the Nelder and Mead Simplex Method for the optimization of stochastic simulation models. Existing and new adaptive extensions of the Nelder and Mead simplex method designed to improve the accuracy and consistency of the observed best point are studied. We comparethe performance of the extensions on a small microsimulation model, as well as on five test functions. We found that gradually decreasing the noise during an optimization run is the most preferred approach for stochastic objective functions. The amount of computation effort needed for successful optimization is very sensitive to the timing of noise reduction and to the rate of decrease of the noise. Restarting the algorithm during the optimization run, in the sense that the algorithm applies a fresh simplex at certain iterations during an optimization run, has adverse effects in our tests for the microsimulation model and for most test functions.simulation;health care;programming;Nelder and Mead Simplex Method

A Three-Level Parallelisation Scheme and Application to the Nelder-Mead Algorithm

We consider a three-level parallelisation scheme. The second and third levels define a classical two-level parallelisation scheme and some load balancing algorithm is used to distribute tasks among processes. It is well-known that for many applications the efficiency of parallel algorithms of the second and third level starts to drop down after some critical parallelisation degree is reached. This weakness of the two-level template is addressed by introduction of one additional parallelisation level. As an alternative to the basic solver some new or modified algorithms are considered on this level. The idea of the proposed methodology is to increase the parallelisation degree by using less efficient algorithms in comparison with the basic solver. As an example we investigate two modified Nelder-Mead methods. For the selected application, a few partial differential equations are solved numerically on the second level, and on the third level the parallel Wang's algorithm is used to solve systems of linear equations with tridiagonal matrices. A greedy workload balancing heuristic is proposed, which is oriented to the case of a large number of available processors. The complexity estimates of the computational tasks are model-based, i.e. they use empirical computational data