Metamodel variability in robust simulation-optimization: a bootstrap analysis

Abstract

Metamodels are often used in simulation-optimization for the design and management of complex systems. These metamodels yield insight into the relationship between responses and decision variables, providing fast analysis tools instead of the more expensive computer simulations. Moreover, these metamodels enable the integration of discipline-dependent analysis into the overall decision process. The use of stochastic simulation experiments and metamodels introduces a source of uncertainty in the decision process that we refer to as metamodel variability. To quantify this variability, we use bootstrapping. More speci�cally, we combine cross-validation and bootstrapping to simulate the metamodel construction process in stochastic environments. The resulting methodology is illustrated through the well-known Economic Order Quantity (EOQ) model using Kriging and regression metamodels. The relative validation errors are small, so they suggest that the metamodels give an adequate approximation, and bootstrapping these errors allows us to quantify the metamodels' variability in an acceptable way