Adaptive extensions of the Nelder and Mead Simplex Method for optimization of stochastic simulation models

We consider the Nelder and Mead Simplex Method for the optimization of stochastic simulation models. Existing and new adaptive extensions of the Nelder and Mead simplex method designed to improve the accuracy and consistency of the observed best point are studied. We compare the performance of the extensions on a small microsimulation model, as well as on five test functions. We found that gradually decreasing the noise during an optimization run is the most preferred approach for stochastic objective functions. The amount of computation effort needed for successful optimization is very sensitive to the timing of noise reduction and to the rate of decrease of the noise. Restarting the algorithm during the optimization run, in the sense that the algorithm applies a fresh simplex at certain iterations during an optimization run, has adverse effects in our tests for the microsimulation model and for most test functions

Pattern Search Ranking and Selection Algorithms for Mixed-Variable Optimization of Stochastic Systems

A new class of algorithms is introduced and analyzed for bound and linearly constrained optimization problems with stochastic objective functions and a mixture of design variable types. The generalized pattern search (GPS) class of algorithms is extended to a new problem setting in which objective function evaluations require sampling from a model of a stochastic system. The approach combines GPS with ranking and selection (R&S) statistical procedures to select new iterates. The derivative-free algorithms require only black-box simulation responses and are applicable over domains with mixed variables (continuous, discrete numeric, and discrete categorical) to include bound and linear constraints on the continuous variables. A convergence analysis for the general class of algorithms establishes almost sure convergence of an iteration subsequence to stationary points appropriately defined in the mixed-variable domain. Additionally, specific algorithm instances are implemented that provide computational enhancements to the basic algorithm. Implementation alternatives include the use modern R&S procedures designed to provide efficient sampling strategies and the use of surrogate functions that augment the search by approximating the unknown objective function with nonparametric response surfaces. In a computational evaluation, six variants of the algorithm are tested along with four competing methods on 26 standardized test problems. The numerical results validate the use of advanced implementations as a means to improve algorithm performance

Tabu search with fully sequential procedure for simulation optimization

Cataloged from PDF version of article.Simulation is a descriptive technique that is used to understand the behaviour of both conceptual and real systems. Most of the real life systems are dynamic and stochastic that it may be very difficult to derive analytical representation. Simulation can be used to model and to analyze these systems. Although simulation provides insightful information about the system behaviour, it cannot be used to optimize the system performance. With the development of the metaheuristics, the concept simulation optimization has became a reality in recent years. A simulation optimization technique uses simulation as an evaluator, and tries to optimize the systems performance by setting appropriate values of simulation input. On the other hand, statistical ranking and selection procedures are used to find the best system design among a set of alternatives with a desired confidence level. In this study, we combine these two methodologies and investigate the performance of the hybrid procedure. Tabu Search (TS) heuristic is combined with the Fully Sequential Procedure (FSP) in simulation optimization context. The performance of the combined procedure is examined in four different systems. The effectiveness of the FSP is assessed considering the computational effort and the convergence to the best (near optimal) solution.Çevik, SavaşM.S

A Revised Simplex Search Procedure for Stochastic Simulation Response Surface Optimization

We develop a variant of the Nelder-Mead (NM) simplex search procedure for stochastic simulation optimization that is designed to avoid many of the weaknesses encumbering similar direct-search methods—in particular, excessive sensitivity to starting values, premature termination at a local optimum, lack of robustness against noisy responses, and computational inefficiency. The Revised Simplex Search (RSS) procedure consists of a three-phase application of the NM method in which: (a) the ending values for one phase become the starting values for the next phase; (b) the step size for the initial simplex (respectively, the shrink coefficient) decreases geometrically (respectively, increases linearly) over successive phases; and (c) the final estimated optimum is the best of the ending values for the three phases. To compare RSS versus NM and procedure RS�S9 due to Barton and Ivey, we summarize a simulatio