Decision Support for Software Projects: The Role of SPC and Simulation Metamodeling

While many researchers have attempted to directly apply statistical process control (SPC) to the software domain, several difficulties in applying SPC to software development, in particular, the inability to compute meaningful control limits for the process. In this research, we propose a framework for applying SPC to software projects. The framework integrates SPC concepts and simulation metamodeling to create meaningful control limits on process and project inputs. The framework is demonstrated using a case study

Sensitivity analysis and optimization of system dynamics models: Regression analysis and statistical design of experiments

This tutorial discusses what-if analysis and optimization of System Dynamics models. These problems are solved, using the statistical techniques of regression analysis and design of experiments (DOE). These issues are illustrated by applying the statistical techniques to a System Dynamics model for coal transportation, taken from Wolstenholme's book "System Enquiry: a System Dynamics Approach" (1990). The regression analysis uses the least squares algorithm. DOE uses classic designs, namely, fractional factorials and central composite designs. Compared with intuitive approaches, DOE is more efficient: DOE gives more accurate estimators of input effects. Moreover DOE is more effective: interactions are estimable too. The System Dynamics model is also optimized. A heuristic is derived, inspired by Response Surface Methodology (RSM) but accounting for constraints. Some remaining pertinent problems are briefly discussed, namely DOE for cases with many factors, and DOE for random System Dynamics models. Conclusions are presented for the case study, and general principles are derived. Finally 23 references are given for further study.Regression Analysis;Experimental Design;System Dynamics Models;statistics

Manufacturing Lead Time Estimation with the Combination of Simulation and Statistical Learning Methods

In the paper, a novel method is introduced for selecting tuning parameters improving accuracy and robustness for multi-model based prediction of manufacturing lead times. Prediction is made by setting up models using statistical learning methods (multivariate regression); trained, validated and tested on log data gathered by manufacturing execution systems (MES). Relevant features, i.e., the predictors most contributing to the response, are selected from a wider range of system parameters. The proposed method is tested on data provided by a discrete event simulation model (as a part of a simulation-based prediction framework) of a small-sized flow-shop system. Accordingly, log data are generated by simulation experiments, substituting the function of a MES system, while considering several different system settings (e.g., job arrival rate, test rejection rate). By inserting the prediction models into a simulation-based decision support system, prospective simulations anticipating near-future deviations and/or disturbances, could be supported. Consequently, simulation could be applied for reactive, disturbance-handling purposes, and, moreover, for training the prediction models. (C) 2015 The Authors. Published by Elsevier B.V

A comprehensive literature classification of simulation optimisation methods

Simulation Optimization (SO) provides a structured approach to the system design and configuration when analytical expressions for input/output relationships are unavailable. Several excellent surveys have been written on this topic. Each survey concentrates on only few classification criteria. This paper presents a literature survey with all classification criteria on techniques for SO according to the problem of characteristics such as shape of the response surface (global as compared to local optimization), objective functions (single or multiple objectives) and parameter spaces (discrete or continuous parameters). The survey focuses specifically on the SO problem that involves single per-formance measureSimulation Optimization, classification methods, literature survey

Characterization and valuation of uncertainty of calibrated parameters in stochastic decision models

We evaluated the implications of different approaches to characterize uncertainty of calibrated parameters of stochastic decision models (DMs) in the quantified value of such uncertainty in decision making. We used a microsimulation DM of colorectal cancer (CRC) screening to conduct a cost-effectiveness analysis (CEA) of a 10-year colonoscopy screening. We calibrated the natural history model of CRC to epidemiological data with different degrees of uncertainty and obtained the joint posterior distribution of the parameters using a Bayesian approach. We conducted a probabilistic sensitivity analysis (PSA) on all the model parameters with different characterizations of uncertainty of the calibrated parameters and estimated the value of uncertainty of the different characterizations with a value of information analysis. All analyses were conducted using high performance computing resources running the Extreme-scale Model Exploration with Swift (EMEWS) framework. The posterior distribution had high correlation among some parameters. The parameters of the Weibull hazard function for the age of onset of adenomas had the highest posterior correlation of -0.958. Considering full posterior distributions and the maximum-a-posteriori estimate of the calibrated parameters, there is little difference on the spread of the distribution of the CEA outcomes with a similar expected value of perfect information (EVPI) of \$653 and \$685, respectively, at a WTP of \$66,000/QALY. Ignoring correlation on the posterior distribution of the calibrated parameters, produced the widest distribution of CEA outcomes and the highest EVPI of \$809 at the same WTP. Different characterizations of uncertainty of calibrated parameters have implications on the expect value of reducing uncertainty on the CEA. Ignoring inherent correlation among calibrated parameters on a PSA overestimates the value of uncertainty.Comment: 17 pages, 6 figures, 3 table

A near-stationary subspace for ridge approximation

Response surfaces are common surrogates for expensive computer simulations in engineering analysis. However, the cost of fitting an accurate response surface increases exponentially as the number of model inputs increases, which leaves response surface construction intractable for high-dimensional, nonlinear models. We describe ridge approximation for fitting response surfaces in several variables. A ridge function is constant along several directions in its domain, so fitting occurs on the coordinates of a low-dimensional subspace of the input space. We review essential theory for ridge approximation---e.g., the best mean-squared approximation and an optimal low-dimensional subspace---and we prove that the gradient-based active subspace is near-stationary for the least-squares problem that defines an optimal subspace. Motivated by the theory, we propose a computational heuristic that uses an estimated active subspace as an initial guess for a ridge approximation fitting problem. We show a simple example where the heuristic fails, which reveals a type of function for which the proposed approach is inappropriate. We then propose a simple alternating heuristic for fitting a ridge function, and we demonstrate the effectiveness of the active subspace initial guess applied to an airfoil model of drag as a function of its 18 shape parameters

The Effect of Systematic Error in Forced Oscillation Testing

One of the fundamental problems in flight dynamics is the formulation of aerodynamic forces and moments acting on an aircraft in arbitrary motion. Classically, conventional stability derivatives are used for the representation of aerodynamic loads in the aircraft equations of motion. However, for modern aircraft with highly nonlinear and unsteady aerodynamic characteristics undergoing maneuvers at high angle of attack and/or angular rates the conventional stability derivative model is no longer valid. Attempts to formulate aerodynamic model equations with unsteady terms are based on several different wind tunnel techniques: for example, captive, wind tunnel single degree-of-freedom, and wind tunnel free-flying techniques. One of the most common techniques is forced oscillation testing. However, the forced oscillation testing method does not address the systematic and systematic correlation errors from the test apparatus that cause inconsistencies in the measured oscillatory stability derivatives. The primary objective of this study is to identify the possible sources and magnitude of systematic error in representative dynamic test apparatuses. Sensitivities of the longitudinal stability derivatives to systematic errors are computed, using a high fidelity simulation of a forced oscillation test rig, and assessed using both Design of Experiments and Monte Carlo methods

Meta Modelle - Neue Planungswerkzeuge für Materialflußsysteme

Meta-Modelle sind Rechenmodelle, die das Verhalten technischer Systeme näherungsweise beschreiben oder nachbilden. Sie werden aus Beobachtungen von Simulationsmodellen der technischen Systeme abgeleitet. Es handelt sich also um Modelle von Modellen, um Meta-Modelle. Meta-Modelle unterscheiden sich grundsätzlich von analytischen Ansätzen zur Systembeschreibung. Während analytische Ansätze in ihrer mathematischen Struktur die tatsächlichen Gegebenheiten des betrachteten Systems wiedergeben, sind Meta-Modelle stets Näherungen. Der Vorteil von Meta-Modellen liegt in ihrer einfachen Form. Sie sind leicht zu bilden und anzuwenden. Ihr Nachteil ist die nur annähernde und u.U. unvollständige Beschreibung des Systemverhaltens. Im folgenden wird die Bildung von Meta-Modellen anhand eines Bediensystems dargestellt. Zuerst werden die Möglichkeiten einer analytischen Beschreibung bewertet. Danach werden zwei unterschiedliche Meta-Modelle, Polynome und neuronale Netze, vorgestellt. Möglichkeiten und Grenzen dieser Formen der Darstellung des Systemverhaltens werden diskutiert. Abschließend werden praktische Einsatzfelder von Meta-Modellen in der Materialflußplanung und -simulation aufgezeigt