Sensitivity analysis of expensive black-box systems using metamodeling

Simulations are becoming ever more common as a tool for designing complex products. Sensitivity analysis techniques can be applied to these simulations to gain insight, or to reduce the complexity of the problem at hand. However, these simulators are often expensive to evaluate and sensitivity analysis typically requires a large amount of evaluations. Metamodeling has been successfully applied in the past to reduce the amount of required evaluations for design tasks such as optimization and design space exploration. In this paper, we propose a novel sensitivity analysis algorithm for variance and derivative based indices using sequential sampling and metamodeling. Several stopping criteria are proposed and investigated to keep the total number of evaluations minimal. The results show that both variance and derivative based techniques can be accurately computed with a minimal amount of evaluations using fast metamodels and FLOLA-Voronoi or density sequential sampling algorithms.Comment: proceedings of winter simulation conference 201

Kriging Metamodeling in Simulation: A Review

This article reviews Kriging (also called spatial correlation modeling). It presents the basic Kriging assumptions and formulas contrasting Kriging and classic linear regression metamodels. Furthermore, it extends Kriging to random simulation, and discusses bootstrapping to estimate the variance of the Kriging predictor. Besides classic one-shot statistical designs such as Latin Hypercube Sampling, it reviews sequentialized and customized designs. It ends with topics for future research.Kriging;Metamodel;Response Surface;Interpolation;Design

Customized Sequential Designs for Random Simulation Experiments: Kriging Metamodelling and Bootstrapping

This paper proposes a novel method to select an experimental design for interpolation in random simulation.(Though the paper focuses on Kriging, this method may also apply to other types of metamodels such as linear regression models.)Assuming that simulation requires much computer time, it is important to select a design with a small number of observations (or simulation runs).The proposed method is therefore sequential.Its novelty is that it accounts for the specific input/output behavior (or response function) of the particular simulation at hand; i.e., the method is customized or application-driven.A tool for this customization is bootstrapping, which enables the estimation of the variances of predictions for inputs not yet simulated.The new method is tested through the classic M/M/1 queueing simulation.For this simulation the novel design indeed gives better results than a Latin Hypercube Sampling (LHS) with a prefixed sample of the same size.simulation;statistical methods;bootstrap

Application-driven Sequential Designs for Simulation Experiments: Kriging Metamodeling

This paper proposes a novel method to select an experimental design for interpolation in simulation.Though the paper focuses on Kriging in deterministic simulation, the method also applies to other types of metamodels (besides Kriging), and to stochastic simulation.The paper focuses on simulations that require much computer time, so it is important to select a design with a small number of observations.The proposed method is therefore sequential.The novelty of the method is that it accounts for the specific input/output function of the particular simulation model at hand; i.e., the method is application-driven or customized.This customization is achieved through cross-validation and jackknifing.The new method is tested through two academic applications, which demonstrate that the method indeed gives better results than a design with a prefixed sample size.experimental design;simulation;interpolation;sampling;sensitivity analysis;metamodels

Design of Experiments: An Overview

Design Of Experiments (DOE) is needed for experiments with real-life systems, and with either deterministic or random simulation models. This contribution discusses the different types of DOE for these three domains, but focusses on random simulation. DOE may have two goals: sensitivity analysis including factor screening and optimization. This contribution starts with classic DOE including 2k-p and Central Composite designs. Next, it discusses factor screening through Sequential Bifurcation. Then it discusses Kriging including Latin Hyper cube Sampling and sequential designs. It ends with optimization through Generalized Response Surface Methodology and Kriging combined with Mathematical Programming, including Taguchian robust optimization.simulation;sensitivity analysis;optimization;factor screening;Kriging;RSM;Taguchi

Design and Analysis of Monte Carlo Experiments

monte carlo experiments;simulation models;mathematical analysis;sensitivity analysis;experimental design

Kriging metamodeling for simulation

Many scientific disciplines use mathematical models to describe complicated real systems. Often, analytical methods are inadequate, so simulation is applied. This thesis focuses on computer intensive simulation experiments in Operations Research/Management Science. For such experiments it is necessary to apply interpolation. In this thesis, Kriging interpolation for random simulation is proposed and a novel type of Kriging - called Detrended Kriging - is developed. Kriging turns out to give better predictions in random simulation than classic low-order polynomial regression. Kriging is not sensitive to variance heterogeneity: i.e. Kriging is a robust method. Moreover, the thesis develops a novel method to select experimental designs for expensive simulation. This method is sequential, and accounts for the specific input/output function implied by the underlying simulation model. For deterministic simulation the designs are constructed through cross-validation and jackknifing, whereas for random simulation the customization is achieved through bootstrapping. The novel method simulates relatively more input combinations in the interesting parts of the input/output function, and gives better predictions than traditional Latin Hypercube Sample designs with prefixed sample sizes.