Meta-Modeling by Symbolic Regression and Pareto Simulated Annealing

The subject of this paper is a new approach to Symbolic Regression.Other publications on Symbolic Regression use Genetic Programming.This paper describes an alternative method based on Pareto Simulated Annealing.Our method is based on linear regression for the estimation of constants.Interval arithmetic is applied to ensure the consistency of a model.In order to prevent over-fitting, we merit a model not only on predictions in the data points, but also on the complexity of a model.For the complexity we introduce a new measure.We compare our new method with the Kriging meta-model and against a Symbolic Regression meta-model based on Genetic Programming.We conclude that Pareto Simulated Annealing based Symbolic Regression is very competitive compared to the other meta-model approachesapproximation;meta-modeling;pareto simulated annealing;symbolic regression

Efficient Approximation of Black-Box Functions and Pareto Sets.

In the case of time-consuming simulation models or other so-called black-box functions, we determine a metamodel which approximates the relation between the input- and output-variables of the simulation model. To solve multi-objective optimization problems, we approximate the Pareto set, i.e. the set of Pareto optimal solutions for which it is not possible to improve one objective without deteriorating another.

Enhancement of Sandwich Algorithms for Approximating Higher Dimensional Convex Pareto Sets

In many fields, we come across problems where we want to optimize several conflicting objectives simultaneously. To find a good solution for such multi-objective optimization problems, an approximation of the Pareto set is often generated. In this paper, we con- sider the approximation of Pareto sets for problems with three or more convex objectives and with convex constraints. For these problems, sandwich algorithms can be used to de- termine an inner and outer approximation between which the Pareto set is 'sandwiched'. Using these two approximations, we can calculate an upper bound on the approximation error. This upper bound can be used to determine which parts of the approximations must be improved and to provide a quality guarantee to the decision maker. In this paper, we extend higher dimensional sandwich algorithms in three different ways. Firstly, we introduce the new concept of adding dummy points to the inner approx- imation of a Pareto set. By using these dummy points, we can determine accurate inner and outer approximations more e±ciently, i.e., using less time-consuming optimizations. Secondly, we introduce a new method for the calculation of an error measure which is easy to interpret. The combination of easy calculation and easy interpretation makes this measure very suitable for sandwich algorithms. Thirdly, we show how transforming cer- tain objective functions can improve the results of sandwich algorithms and extend their applicability to certain non-convex problems. The calculation of the introduced error measure when using transformations will also be discussed. To show the effect of these enhancements, we make a numerical comparison using four test cases, including a four-dimensional case from the field of intensity-modulated radiation therapy (IMRT). The results of the different cases show that we can indeed achieve an accurate approximation using significantly fewer optimizations by using the enhancements.Convexity;e-efficiency;e-Pareto optimality;Geometric programming;Higher dimensional;Inner and outer approximation;IMRT;Pareto set;Multi-objective optimiza- tion;Sandwich algorithms;Transformations

Bounds for Maximin Latin Hypercube Designs

Latin hypercube designs (LHDs) play an important role when approximating computer simula- tion models. To obtain good space-filling properties, the maximin criterion is frequently used. Unfortunately, constructing maximin LHDs can be quite time-consuming when the number of dimensions and design points increase. In these cases, we can use approximate maximin LHDs. In this paper, we construct bounds for the separation distance of certain classes of maximin LHDs. These bounds are useful for assessing the quality of approximate maximin LHDs. Until now only upper bounds are known for the separation distance of certain classes of unrestricted maximin designs, i.e. for maximin designs without a Latin hypercube struc- ture. The separation distance of maximin LHDs also satisfies these “unrestricted” bounds. By using some of the special properties of LHDs, we are able to find new and tighter bounds for maximin LHDs. Within the different methods used to determine the upper bounds, a vari- ety of combinatorial optimization techniques are employed. Mixed Integer Programming, the Travelling Salesman Problem, and the Graph Covering Problem are among the formulations used to obtain the bounds. Besides these bounds, also a construction method is described for generating LHDs that meet Baer’s bound for the ℓ1 distance measure for certain values of n.Latin hypercube design;maximin;space-filling;mixed integer programming;trav- elling salesman problem;graph covering.

Nested Maximin Latin Hypercube Designs

In the field of design of computer experiments (DoCE), Latin hypercube designs are frequently used for the approximation and optimization of black-boxes. In certain situations, we need a special type of designs consisting of two separate designs, one being a subset of the other. These nested designs can be used to deal with training and test sets, models with different levels of accuracy, linking parameters, and sequential evaluations. In this paper, we construct nested maximin Latin hypercube designs for up to ten dimensions. We show that different types of grids should be considered when constructing nested designs and discuss how to determine which grid to use for a specific application. To determine nested maximin designs for dimensions higher than two, four different variants of the ESE-algorithm of Jin et al. (2005) are introduced and compared. In the appendix, maximin distances for different numbers of points are provided; the corresponding nested maximin designs can be found on the website http://www.spacefillingdesigns.nl.Design of computer experiments;Latin hypercube design;linking parameter;nested designs;sequential simulation;space-filling;training and test set

Space-Filling Latin Hypercube Designs For Computer Experiments (Revision of CentER DP 2006-18)

In the area of computer simulation, Latin hypercube designs play an important role. In this paper the classes of maximin and Audze-Eglais Latin hypercube designs are considered. Up to now only several two-dimensional designs and a few higher dimensional designs for these classes have been published. Using periodic designs and the Enhanced Stochastic Evolutionary algorithm of Jin et al. (2005), we obtain new results which we compare to existing results. We thus construct a database of approximate maximin and Audze-Eglais Latin hypercube designs for up to ten dimensions and for up to 300 design points. All these designs can be downloaded from the website http://www.spacefillingdesigns.nl.Audze-Eglais;computer experiment;Enhanced Stochastic Evolutionary algorithm;Latin hypercube design;maximin;non-collapsing;packing problem;simulated annealing;space-filling

Meta-Modeling by Symbolic Regression and Pareto Simulated Annealing

The subject of this paper is a new approach to Symbolic Regression.Other publications on Symbolic Regression use Genetic Programming.This paper describes an alternative method based on Pareto Simulated Annealing.Our method is based on linear regression for the estimation of constants.Interval arithmetic is applied to ensure the consistency of a model.In order to prevent over-fitting, we merit a model not only on predictions in the data points, but also on the complexity of a model.For the complexity we introduce a new measure.We compare our new method with the Kriging meta-model and against a Symbolic Regression meta-model based on Genetic Programming.We conclude that Pareto Simulated Annealing based Symbolic Regression is very competitive compared to the other meta-model approaches