Ethical Issues in Engineering Models: Personal Reflections

I start this contribution with an overview of my personal involvement—as an Operations Research consultant—in several engineering case-studies that may raise ethical questions; these case studies employ simulation models. Next, I present an overview of the recent literature on ethical issues in modeling, focusing on the validation of the model’s assumptions; the decisive role of these assumptions leads to the quest for robust models. Actually, models are meant to solve practical problems; these problems may have ethical implications for the various stakeholders; namely, modelers, clients, and the public at large. Finally, I briefly discuss whistle blowing.ethics;code of conduct;stakeholders;validity;risk analysis;simulation;operations research

Monotonicity-Preserving Bootstrapped Kriging Metamodels for Expensive Simulations

Kriging (Gaussian process, spatial correlation) metamodels approximate the Input/Output (I/O) functions implied by the underlying simulation models; such metamodels serve sensitivity analysis and optimization, especially for computationally expensive simulations. In practice, simulation analysts often know that the I/O function is monotonic. To obtain a Kriging metamodel that preserves this known shape, this article uses bootstrapping (or resampling). Parametric bootstrapping assuming normality may be used in deterministic simulation, but this article focuses on stochastic simulation (including discrete-event simulation) using distribution-free bootstrapping. In stochastic simulation, the analysts should simulate each input combination several times to obtain a more reliable average output per input combination. Nevertheless, this average still shows sampling variation, so the Kriging metamodel does not need to interpolate the average outputs. Bootstrapping provides a simple method for computing a noninterpolating Kriging model. This method may use standard Kriging software, such as the free Matlab toolbox called DACE. The method is illustrated through the M/M/1 simulation model with as outputs either the estimated mean or the estimated 90% quantile; both outputs are monotonic functions of the traffic rate, and have nonnormal distributions. The empirical results demonstrate that monotonicity-preserving bootstrapped Kriging may give higher probability of covering the true simulation output, without lengthening the confidence interval.Queues

Simulation-Optimization via Kriging and Bootstrapping:A Survey (Revision of CentER DP 2011-064)

Abstract: This article surveys optimization of simulated systems. The simulation may be either deterministic or random. The survey reflects the author’s extensive experience with simulation-optimization through Kriging (or Gaussian process) metamodels. The analysis of these metamodels may use parametric bootstrapping for deterministic simulation or distribution-free bootstrapping (or resampling) for random simulation. The survey covers: (1) Simulation-optimization through "efficient global optimization" (EGO) using "expected improvement" (EI); this EI uses the Kriging predictor variance, which can be estimated through parametric bootstrapping accounting for estimation of the Kriging parameters. (2) Optimization with constraints for multiple random simulation outputs and deterministic inputs through mathematical programming applied to Kriging metamodels validated through distribution-free bootstrapping. (3) Taguchian robust optimization for uncertain environments, using mathematical programming— applied to Kriging metamodels— and distribution- free bootstrapping to estimate the variability of the Kriging metamodels and the resulting robust solution. (4) Bootstrapping for improving convexity or preserving monotonicity of the Kriging metamodel.

Variance Reduction Techniques in Monte Carlo Methods

Monte Carlo methods are simulation algorithms to estimate a numerical quantity in a statistical model of a real system. These algorithms are executed by computer programs. Variance reduction techniques (VRT) are needed, even though computer speed has been increasing dramatically, ever since the introduction of computers. This increased computer power has stimulated simulation analysts to develop ever more realistic models, so that the net result has not been faster execution of simulation experiments; e.g., some modern simulation models need hours or days for a single ’run’ (one replication of one scenario or combination of simulation input values). Moreover there are some simulation models that represent rare events which have extremely small probabilities of occurrence), so even modern computer would take ’for ever’ (centuries) to execute a single run - were it not that special VRT can reduce theses excessively long runtimes to practical magnitudes.common random numbers;antithetic random numbers;importance sampling;control variates;conditioning;stratied sampling;splitting;quasi Monte Carlo

Statistical Testing of Optimality Conditions in Multiresponse Simulation-Based Optimization (Replaced by Discussion Paper 2007-45)

This paper derives a novel procedure for testing the Karush-Kuhn-Tucker (KKT) first-order optimality conditions in models with multiple random responses.Such models arise in simulation-based optimization with multivariate outputs.This paper focuses on expensive simulations, which have small sample sizes.The paper estimates the gradients (in the KKT conditions) through low-order polynomials, fitted locally.These polynomials are estimated using Ordinary Least Squares (OLS), which also enables estimation of the variability of the estimated gradients.Using these OLS results, the paper applies the bootstrap (resampling) method to test the KKT conditions.Furthermore, it applies the classic Student t test to check whether the simulation outputs are feasible, and whether any constraints are binding.The paper applies the new procedure to both a synthetic example and an inventory simulation; the empirical results are encouraging.stopping rule;metaheuristics;RSM;design of experiments

The Cost of Risk-Aversion In Inventory Management:An (s,S) Case Study

To model a risk-averse attitude (instead of a risk-neutral attitude), we may require that the 90% quantile (instead of the expected value) of a specific uncertain (or random) response satisfy a prespecified threshold which corresponds with a chance constraint. We include a case study; namely, an (s, S) inventory model that is specified in the literature. In this study we require that the 90% quantile of the service level exceed a prespecified threshold. So, we need to estimate the optimal values of s and S, which satisfy this service-level constraint while minimizing the expected inventory cost. To solve the resulting constrained optimization problem, we apply a recent variant of “efficient global optimization” (also known as “Bayesian optimization” and related to “active” machine learning). Our numerical results for the case study imply that the mean inventory cost increases by 2.4% if management is risk-averse instead of risk-neutral

Constrained Optimization in Random Simulation:Efficient Global Optimization and Karush-Kuhn-Tucker Conditions

We develop a novel method for solving constrained optimization problems in random (or stochastic) simulation; i.e., our method minimizes the goal output subject to one or more output constraints and input constraints. Our method is indeed novel, as it combines the Karush-Kuhn-Tucker (KKT) conditions with the popular algorithm called "effciient global optimization" (EGO), which is also known as "Bayesian optimization" and is related to “active learning". Originally, EGO solves non-constrained optimization problems in deterministic simulation; EGO is a sequential algorithm that uses Kriging (or Gaussian process) metamodeling of the underlying simulation model, treating the simulation as a black box. Though there are many variants of EGO - for these non-constrained deterministic problems and for variants of these problems - none of these EGO-variants use the KKT conditions - even though these conditions are well-known (first-order necessary) optimality conditions in white-box problems. Because the simulation is random, we apply stochastic Kriging. Furthermore, we allow for variance heterogeneity and apply a popular sample allocation rule to determine the number of replicated simulation outputs for selected combinations of simulation inputs. Moreover, our algorithm can take advantage of parallel computing. We numerically compare the performance of our algorithm and the popular proprietary OptQuest algorithm, in two familiar examples (namely, a mathematical toy example and a practical inventory system with a service-level constraint); we conclude that our algorithm is more efficient (requires fewer expensive simulation runs) and effective (gives better estimates of the true global optimum)

Expected Improvement in Efficient Global Optimization Through Bootstrapped Kriging - Replaces CentER DP 2010-62

This article uses a sequentialized experimental design to select simulation input com- binations for global optimization, based on Kriging (also called Gaussian process or spatial correlation modeling); this Kriging is used to analyze the input/output data of the simulation model (computer code). This design and analysis adapt the clas- sic "expected improvement" (EI) in "efficient global optimization" (EGO) through the introduction of an unbiased estimator of the Kriging predictor variance; this estimator uses parametric bootstrapping. Classic EI and bootstrapped EI are com- pared through various test functions, including the six-hump camel-back and several Hartmann functions. These empirical results demonstrate that in some applications bootstrapped EI finds the global optimum faster than classic EI does; in general, however, the classic EI may be considered to be a robust global optimizer.Simulation;Optimization;Kriging;Bootstrap