Verification and validation of simulation models

Simulation Models;econometrics

Statistical validation of simulation models: A case study

Rigorous statistical validation requires that the responses of the model and the real system have the same expected values. However, the modeled and actual responses are not comparable if they are obtained under different scenarios (environmental conditions). Moreover, data on the real system may be unavailable; sensitivity analysis can then be applied to find out whether the model inputs have effects on the model outputs that agree with the experts' intuition. Not only the total model, but also its modules may be submitted to such sensitivity analyses. This article illustrates these issues through a case study, namely a simulation model for the use of sonar to search for mines on the sea bottom. The methodology, however, applies to models in general.Simulation Models;Statistical Validation;statistics

Validation of simulation models: Mine-hunting case-study

Simulation Models

Experimental Design for Sensitivity Analysis, Optimization and Validation of Simulation Models

This chapter gives a survey on the use of statistical designs for what-if analysis in simula- tion, including sensitivity analysis, optimization, and validation/verification. Sensitivity analysis is divided into two phases. The first phase is a pilot stage, which consists of screening or searching for the important factors among (say) hundreds of potentially important factors. A novel screening technique is presented, namely sequential bifurcation. The second phase uses regression analysis to approximate the input/output transformation that is implied by the simulation model; the resulting regression model is also known as a metamodel or a response surface. Regression analysis gives better results when the simu- lation experiment is well designed, using either classical statistical designs (such as frac- tional factorials) or optimal designs (such as pioneered by Fedorov, Kiefer, and Wolfo- witz). To optimize the simulated system, the analysts may apply Response Surface Metho- dology (RSM); RSM combines regression analysis, statistical designs, and steepest-ascent hill-climbing. To validate a simulation model, again regression analysis and statistical designs may be applied. Several numerical examples and case-studies illustrate how statisti- cal techniques can reduce the ad hoc character of simulation; that is, these statistical techniques can make simulation studies give more general results, in less time. Appendix 1 summarizes confidence intervals for expected values, proportions, and quantiles, in termi- nating and steady-state simulations. Appendix 2 gives details on four variance reduction techniques, namely common pseudorandom numbers, antithetic numbers, control variates or regression sampling, and importance sampling. Appendix 3 describes jackknifing, which may give robust confidence intervals.least squares;distribution-free;non-parametric;stopping rule;run-length;Von Neumann;median;seed;likelihood ratio

Data analysis with R in an experimental physics environment

A software package has been developed to bridge the R analysis model with the conceptual analysis environment typical of radiation physics experiments. The new package has been used in the context of a project for the validation of simulation models, where it has demonstrated its capability to satisfy typical requirements pertinent to the problem domain.Comment: IEEE Nuclear Science Symposium 201

Validation of simulation models in the context of railway vehicle acceptance

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.The evaluation of a reliable validation method, criteria and limit values suitable for model validation in the context of vehicle acceptance was one of the objectives of the DynoTRAIN project. The presented investigations represent a unique amount of testing, simulations, comparisons with measurements, and validation evaluations. The on-track measurements performed in four European countries included several different vehicles on a test train equipped to simultaneously record track irregularities and rail profiles. The simulations were performed using vehicle models built with the use of different simulation tools by different partners. The comparisons between simulation and measurement results were conducted for over 1000 simulations using a set of the same test sections for all vehicle models. The results were assessed by three different validation approaches: comparing values according to EN 14363; by subjective engineering judgement by project partners; and using so-called validation metrics, i.e. computable measures developed with the aim of increasing objectivity while still maintaining the level of agreement with engineering judgement. The proposed validation method uses the values computed by analogy with EN 14363 and provides validation limits that can be applied to a set of deviations between simulation and measurement values