On automated sequential steady-state simulation.

The credibility of the final results from stochastic simulation has had limited discussion in the simulation literature so far. However, it is important that the final results from any simulations be credible. To achieve this, validation, which determines whether the conceptual simulation model is an accurate representation of the system under study, has to be done carefully. Additionally, a proper statistical analysis of simulation output data, including a confidence interval or other assessment of statistical errors, has to be conducted before any valid inferences or conclusions about the performance of simulated dynamic systems, such as for example telecommunication networks, are made. There are many other issues, such as choice of a good pseudo-random number generator, elimination of initialisation bias in steady-state simulations, and consideration of auto correlations in collected observations, which have to be appropriately addressed for the final results to be credible. However, many of these issues are not trivial, particularly for simulation users who may not be experts in these areas. As a consequence, a fully-automated simulation package, which can control all important aspects of stochastic simulation, is needed. This dissertation focuses on the following contributions to such a package for steady-state simulation: properties of confidence intervals (CIs) used in coverage analysis, heuristic rules for improving the coverage of the final CIs in practical applications, automated sequential analysis of mean values by the method of regenerative cycles, automatic detection of the initial transient period for steady-state quantile estimation, and sequential steady-state quantile estimation with the automated detection of the length of initial transient period. One difficulty in obtaining precise estimates of a system using stochastic simulation can be the cost of the computing time needed to collect the large amount of output data required. Indeed there are situations, such as estimation of rare events, where, even assuming an appropriate statistical analysis procedure is available, the cost of collecting the number of observations needed by the analysis procedure can be prohibitively large. Fortunately, inexpensive computer network resources enable computationally intensive simulations by allowing us to run parallel and distributed simulations. Therefore, where possible, we extend the contributions to the distributed stochastic simulation scenario known as the Multiple Replications In Parallel (MRIP), in which multiple processors run their own independent replications of the simulated system but cooperate with central analysers that collect data to estimate the final results

Empirically derived methods for analysing simulation model output.

Often in simulation procedures are not proposed unless they are supported by a strong mathematical background. As will be shown in this thesis, this approach does not always give good results when the procedures are applied to complex simulation models, especially on output analysis. For this reason we have used an empirical rather than a theoretical approach for dealing with some of the output problems of simulation. The research carried out has dealt mainly with queuing networks. The first problem we address is that of the identification of possible unstable queues. We also deal with the problem of the identification of queues that may require a long simulation run length to reach the steady state. The method of replications is used for the estimation of terminating and sometimes of steady state parameters. In this thesis we study the relationship that exists between the number of replications used in the simulation and the simulation run length required for the parameter being estimated to reach the steady state. We also study the influence of the random number streams on the values of the mean estimates as a function of the number of replications. One of the most commonly discussed problems related to the estimation of steady state parameters is that of the initialisation bias problem. Two methods are proposed in this thesis to deal with this problem. In one of the methods we propose an effective procedure that can be used for the estimation of the number of initial observations that are to be deleted. The second method, is based on a basic forecasting technique called weighted averages and does not require the elimination of any of the initial observations. Another topic that has been studied in this thesis is the batch means method which is employed for the estimation of steady state parameters based on a single but very long simulation run. We show how a new sampling method called Descriptive Sampling is well suited for the estimation of steady state parameters with the batch means method. We also show how some of the procedures proposed in the literature for use in the batch means method do not work well in simulation models for which no analytical answer exists. The thesis demonstrates that empirically derived methods can be practically effective and could form future theoretical research

A Renewal Theoretic Approach to Bias Reduction in Regenerative Simulations

The special structure of regenerative processes is exploited to derive a new point estimate with very low bias for steady state quantities of regenerative simulations. If the simulation run length is t units of tune, the bias of the new estimate is of order 1/t 2 as opposed to the bias of order 1/t associated with more standard estimates. The bias reduction is achieved by continuing the simulation until the first regeneration after time t and then forming the ratio estimate which involves the random number of regenerative cycles observed during the simulation. Empirical results for several queueing models demonstrate that the bias reduction can be substantial for small values of t.simulation, regenerative, bias reduction, renewal theory, cumulative process, ratio estimator