Sensitivity analysis and optimization of system dynamics models: Regression analysis and statistical design of experiments

This tutorial discusses what-if analysis and optimization of System Dynamics models. These problems are solved, using the statistical techniques of regression analysis and design of experiments (DOE). These issues are illustrated by applying the statistical techniques to a System Dynamics model for coal transportation, taken from Wolstenholme's book "System Enquiry: a System Dynamics Approach" (1990). The regression analysis uses the least squares algorithm. DOE uses classic designs, namely, fractional factorials and central composite designs. Compared with intuitive approaches, DOE is more efficient: DOE gives more accurate estimators of input effects. Moreover DOE is more effective: interactions are estimable too. The System Dynamics model is also optimized. A heuristic is derived, inspired by Response Surface Methodology (RSM) but accounting for constraints. Some remaining pertinent problems are briefly discussed, namely DOE for cases with many factors, and DOE for random System Dynamics models. Conclusions are presented for the case study, and general principles are derived. Finally 23 references are given for further study.Regression Analysis;Experimental Design;System Dynamics Models;statistics

Training for design of experiments using a catapult

Design of experiments (DOE) is a powerful approach for discovering a set of process (or design) variables which are most important to the process and then determine at what levels these variables must be kept to optimize the response (or quality characteristic) of interest. This paper presents two catapult experiments which can be easily taught to engineers and managers in organizations to train for design of experiments. The results of this experiment have been taken from a real live catapult experiment performed by a group of engineers in a company during the training program on DOE. The first experiment was conducted to separate out the key factors (or variables) from the trivial and the second experiment was carried out using the key factors to understand the nature of interactions among the key factors. The results of the experiment were analysed using simple but powerful graphical tools for rapid and easier understanding of the results to engineers with limited statistical competency

Using design-of-experiments techniques for an efficient finite element study of the influence of changed parameters in design

All designs are marred by uncertainties and tolerances in dimen- sions, load levels etc. Traditionally, one has often over-dimensioned to take these uncertainties into account. The demand for optimized designs with high quality and reliability increases, which means that more sophisticated methods have been developed, see e.g. Lochner and Matar (1990). By describing the fluctuations in design parame- ters in terms of distributions with expectation and variance, the design can be examined with statistical methods, which results in a more op-timized design. This treatment of the design often demands several experiments, and to plan these experiments Design Of Experiments (DOE) techniques, see e.g. Montgomery (1991), are often used. By using DOE methods the design variables are systematically altered, which minimizes the number of experiments needed. The output of the experiments is the results of a specified response function, giving an indication of the influence of design variable fluctuations. A FEM system is a suitable tool when performing repeated, similar analyses. Examples exist where the DOE process has been performed external- ly and then transferred to the FEM system in the form of parameter sets defining the analysis cases that are to be solved, see e.g. Summers et al. (1996) and Billings (1996). This paper describes a statistical DOE module based on Taguchi’s method that works within ANSYS. The module plans the FEM anal-ysis and calculates the standard statistical moments of the FEM result. This module serves as a powerful tool for the engineering designer or analysts when examining the influence of variance and mean value of different design variables. It also serves as an exploration of where to concentrate an optimization process

Simulation Experiments in Practice: Statistical Design and Regression Analysis

In practice, simulation analysts often change only one factor at a time, and use graphical analysis of the resulting Input/Output (I/O) data. The goal of this article is to change these traditional, naïve methods of design and analysis, because statistical theory proves that more information is obtained when applying Design Of Experiments (DOE) and linear regression analysis. Unfortunately, classic DOE and regression analysis assume a single simulation response that is normally and independently distributed with a constant variance; moreover, the regression (meta)model of the simulation model’s I/O behaviour is assumed to have residuals with zero means. This article addresses the following practical questions: (i) How realistic are these assumptions, in practice? (ii) How can these assumptions be tested? (iii) If assumptions are violated, can the simulation's I/O data be transformed such that the assumptions do hold? (iv) If not, which alternative statistical methods can then be applied?metamodel;experimental design;jackknife;bootstrap;common random numbers;validation

An artificial neural network for dimensions and cost modelling of internal micro-channels fabricated in PMMA using Nd:YVO4 laser

For micro-channel fabrication using laser micro-machining processing, estimation techniques are normally utilised to develop an approach for the system behaviour evaluation. Design of Experiments (DOE) and the Artificial Neural Networks (ANN) are two methodologies that can be used as estimation techniques. These techniques help in finding a set of laser processing parameters that provides the required micro-channel dimensions and in finding the optimal solutions in terms reducing the product development time, power consumption and of least cost. In this work, an integrated methodology is presented in which the ANN training experiments were obtained by the statistical software DoE to improve the developed models in ANN. A 33 factorial design of experiments (DoE) was used to get the experimental set. Laser power, P; pulse repetition frequency, PRF; and sample translation speed, U were the ANN inputs. The channel width and the produced micro-channel operating cost per metre were the measured responses. Four Artificial Neural Networks (ANNs) models were developed to be applied to internal micro-channels machined in PMMA using a Nd:YVO4 laser. These models were varied in terms of the selection and the quantity of training data set and constructed using a multi-layered, feed-forward structure with a the back-propagation algorithm. The responses were adequately estimated by the ANN models within the set micro-machining parameters limits. Moreover the effect of changing the selection and the quantity of training data on the approximation capability of the developed ANN model was discussed

Sensitivity analysis and optimization of system dynamics models:Regression analysis and statistical design of experiments

This tutorial discusses what-if analysis and optimization of System Dynamics models. These problems are solved, using the statistical techniques of regression analysis and design of experiments (DOE). These issues are illustrated by applying the statistical techniques to a System Dynamics model for coal transportation, taken from Wolstenholme's book "System Enquiry: a System Dynamics Approach" (1990). The regression analysis uses the least squares algorithm. DOE uses classic designs, namely, fractional factorials and central composite designs. Compared with intuitive approaches, DOE is more efficient: DOE gives more accurate estimators of input effects. Moreover DOE is more effective: interactions are estimable too. The System Dynamics model is also optimized. A heuristic is derived, inspired by Response Surface Methodology (RSM) but accounting for constraints. Some remaining pertinent problems are briefly discussed, namely DOE for cases with many factors, and DOE for random System Dynamics models. Conclusions are presented for the case study, and general principles are derived. Finally 23 references are given for further study.

Model-Robust Design of Experiments for Sequential Identification of ODE Parameters

International audienceThis paper presents the idea of sequential model-robust Design of Experiments (DOE) for the identification of dynamic systems modeled with an Ordinary Differential Equation (ODE). The studied DOE problem consists in selecting sequentially the instants where the measures will be done in order to best estimate the system's parameter. The robustness is achieved by considering a statistical representation of the model error defined as the difference between the true ODE and the ODE used in the model. The idea of modeling the model error with a statistical representation has been widely explored in the DOE literature for the identification of static systems. However, there have been little previous works that apply this idea for the identification of dynamic systems. This paper initiates an exploration of this idea in the context of first-order ODE. The model error is modeled by using a kernel-based representation (Gaussian process). A new criterion for the instant selection is constructed and tested on an illustrative example. The design reached with the proposed sequential robust criterion is compared with the design reached with the non-robust version of criterion and with the classical uniform design

The Goldilocks Approach: A Review of Employing Design of Experiments in Prokaryotic Recombinant Protein Production

The production of high yields of soluble recombinant protein is one of the main objectives of protein biotechnology. Several factors, such as expression system, vector, host, media composition and induction conditions can influence recombinant protein yield. Identifying the most important factors for optimum protein expression may involve significant investment of time and considerable cost. To address this problem, statistical models such as Design of Experiments (DoE) have been used to optimise recombinant protein production. This review examines the application of DoE in the production of recombinant proteins in prokaryotic expression systems with specific emphasis on media composition and culture conditions. The review examines the most commonly used DoE screening and optimisation designs. It provides examples of DoE applied to optimisation of media and culture conditions

A User's Guide to the Brave New World of Designing Simulation Experiments

Many simulation practitioners can get more from their analyses by using the statistical theory on design of experiments (DOE) developed specifically for exploring computer models.In this paper, we discuss a toolkit of designs for simulationists with limited DOE expertise who want to select a design and an appropriate analysis for their computational experiments.Furthermore, we provide a research agenda listing problems in the design of simulation experiments -as opposed to real world experiments- that require more investigation.We consider three types of practical problems: (1) developing a basic understanding of a particular simulation model or system; (2) finding robust decisions or policies; and (3) comparing the merits of various decisions or policies.Our discussion emphasizes aspects that are typical for simulation, such as sequential data collection.Because the same problem type may be addressed through different design types, we discuss quality attributes of designs.Furthermore, the selection of the design type depends on the metamodel (response surface) that the analysts tentatively assume; for example, more complicated metamodels require more simulation runs.For the validation of the metamodel estimated from a specific design, we present several procedures.

Deterministic versus Stochastic Sensitivity Analysis in Investment Problems: An Environmental Case Study

Sensitivity analysis in investment problems is an important tool to determine which factors can jeopardize the future of the investment.Information on the probability distribution of those factors that affect the investment is mostly lacking.In those situations the analysts have two options: (i) apply a method that does not require knowledge of that distribution, or (ii) make assumptions about the distribution.In both approaches sensitivity analysis should result in practical information about the actual importance of potential factors.For approach (i) we apply statistical design of experiments (DOE) in combination with regression analysis or meta-modeling.For approach (ii) we investigate five types of relationships between the model output and each individual factor; Pearson's p, Spearman's rank correlation, and location, dispersion, and statistical dependence.We introduce two distribution types popular with practitioners: uniform and triangular.In an environmental case study both approaches identify the same factors as important.sensitivity analysis;experimental design;investment analysis;simulation