Development Approaches Coupled with Verification and Validation Methodologies for Agent-Based Mission-Level Analytical Combat Simulations

This research investigated the applicability of agent-based combat simulations to real-world combat operations. An agent-based simulation of the Allied offensive search for German U-Boats in the Bay of Biscay during World War II was constructed, extending the state-of-the-art in agent-based combat simulations, bridging the gap between the current level of agent-like combat simulations and the concept of agent-based simulations found in the broader literature. The proposed simulation advances agent-based combat simulations to “validateable” mission-level military operations. Simulation validation is a complex task with numerous, diverse techniques available and levels of validation differing significantly among simulations and applications. This research presents a verification and validation taxonomy based on face validity, empirical validity, and theoretical validity, extending the verification and validation knowledge-base to include techniques specific to agent-based models. The verification and validation techniques are demonstrated in a Bay of Biscay case study. Validating combat operations pose particular problems due to the infrequency of real-world occurrences to serve as simulation validation cases; often just a single validation comparison can be made. This means comparisons to the underlying stochastic process are not possible without significant loss of statistical confidence. This research also presents a statistical validation methodology based on re-sampling historical outcomes, which when coupled with the traditional nonparametric sign test, allows comparison between a simulation and historic operation providing an improved validation indicator beyond the single pass or fail test

Design and optimization under uncertainty of Energy Systems

In many engineering design and optimisation problems, the presence of uncertainty in data and parameters is a central and critical issue. The analysis and design of advanced complex energy systems is generally performed starting from a single operating condition and assuming a series of design and operating parameters as fixed values. However, many of the variables on which the design is based are subject to uncertainty because they are not determinable with an adequate precision and they can affect both performance and cost. Uncertainties stem naturally from our limitations in measurements, predictions and manufacturing, and we can say that any system used in engineering is subject to some degree of uncertainty. Different fields of engineering use different ways to describe this uncertainty and adopt a variety of techniques to approach the problem. The past decade has seen a significant growth of research and development in uncertainty quantification methods to analyse the propagation of uncertain inputs through the systems. One of the main challenges in this field are identifying sources of uncertainty that potentially affect the outcomes and the efficiency in propagating these uncertainties from the sources to the quantities of interest, especially when there are many sources of uncertainties. Hence, the level of rigor in uncertainty analysis depends on the quality of uncertainty quantification method. The main obstacle of this analysis is often the computational effort, because the representative model is typically highly non-linear and complex. Therefore, it is necessary to have a robust tool that can perform the uncertainty propagation through a non-intrusive approach with as few evaluations as possible. The primary goal of this work is to show a robust method for uncertainty quantification applied to energy systems. The first step in this direction was made doing a work on the analysis of uncertainties on a recuperator for micro gas turbines, making use of the Monte Carlo and Response Sensitivity Analysis methodologies to perform this study. However, when considering more complex energy systems, one of the main weaknesses of uncertainty quantification methods arises: the extremely high computational effort needed. For this reason, the application of a so-called metamodel was found necessary and useful. This approach was applied to perform a complete analysis under uncertainty of a solid oxide fuel cell hybrid system, starting from the evaluation of the impact of several uncertainties on the system up to a robust design including a multi-objective optimization. The response surfaces have allowed the authors to consider the uncertainties in the system when performing an acceptable number of simulations. These response were then used to perform a Monte Carlo simulation to evaluate the impact of the uncertainties on the monitored outputs, giving an insight on the spread of the resulting probability density functions and so on the outputs which should be considered more carefully during the design phase. Finally, the analysis of a complex combined cycle with a flue gas condesing heat pump subject to market uncertainties was performed. To consider the uncertainties in the electrical price, which would impact directly the revenues of the system, a statistical study on the behaviour of such price along the years was performed. From the data obtained it was possible to create a probability density function for each hour of the day which would represent its behaviour, and then those distributions were used to analyze the variability of the system in terms of revenues and emissions

Reliability assessment of nuclear power plant fault-diagnostic systems using artificial neural networks

The assurance of the diagnosis obtained from a nuclear power plant (NPP) fault-diagnostic advisor based on artificial neural networks (ANNs) is essential for the practical implementation of the advisor to transient detection and identification. The objectives of this study are to develop a validation and verification technique suitable for ANNs and apply it to the fault-diagnostic advisor. The validation and verification is realized by estimating error bounds on the advisor\u27s diagnoses. The two different partition criteria are developed to create computationally effective partitions for generating the error information associated with the advisor performance. The bootstap partition criterion (BPC) and the modified bootstap partition criterion (MBPC) can alleviate the computational requirements significantly. In addition, a new error-bound prediction scheme called error estimation by series association (EESA) is constructed not only to infer error-bounds but also to alleviate the training complexity of an error predictor neural network. The EESA scheme is applied to validate the outputs of the ANNs modeled for a simple nonlinear mapping and more complicated NPP fault-diagnostic problems. Two independent sets of data simulated at San Onofre Nuclear Generating Station, a pressurized water reactor, and Duane Arnold Energy Center, a boiling water reactor, are used to design the fault-diagnostic advisor systems and to perform the reliability assessment of the advisor systems. The results of this research show that the fault-diagnostic systems developed using ANNs with EESA are effective at producing proper diagnoses with predicted error even when degraded by noise. In general, EESA can also be used to verify an ANN system by indicating that the ANN system requires training on more data in order to increase generalization. The EESA scheme developed in this study can be implemented to any ANN system regardless of ANN learning paradigm

Combining statistical methods with dynamical insight to improve nonlinear estimation

Physical processes such as the weather are usually modelled using nonlinear dynamical systems. Statistical methods are found to be difficult to draw the dynamical information from the observations of nonlinear dynamics. This thesis is focusing on combining statistical methods with dynamical insight to improve the nonlinear estimate of the initial states, parameters and future states. In the perfect model scenario (PMS), method based on the Indistin-guishable States theory is introduced to produce initial conditions that are consistent with both observations and model dynamics. Our meth-ods are demonstrated to outperform the variational method, Four-dimensional Variational Assimilation, and the sequential method, En-semble Kalman Filter. Problem of parameter estimation of deterministic nonlinear models is considered within the perfect model scenario where the mathematical structure of the model equations are correct, but the true parameter values are unknown. Traditional methods like least squares are known to be not optimal as it base on the wrong assumption that the distribu-tion of forecast error is Gaussian IID. We introduce two approaches to address the shortcomings of traditional methods. The first approach forms the cost function based on probabilistic forecasting; the second approach focuses on the geometric properties of trajectories in short term while noting the global behaviour of the model in the long term. Both methods are tested on a variety of nonlinear models, the true parameter values are well identified. Outside perfect model scenario, to estimate the current state of the model one need to account the uncertainty from both observatiOnal noise and model inadequacy. Methods assuming the model is perfect are either inapplicable or unable to produce the optimal results. It is almost certain that no trajectory of the model is consistent with an infinite series of observations. There are pseudo-orbits, however, that are consistent with observations and these can be used to estimate the model states. Applying the Indistinguishable States Gradient De-scent algorithm with certain stopping criteria is introduced to find rel-evant pseudo-orbits. The difference between Weakly Constraint Four-dimensional Variational Assimilation (WC4DVAR) method and Indis-tinguishable States Gradient Descent method is discussed. By testing on two system-model pairs, our method is shown to produce more consistent results than the WC4DVAR method. Ensemble formed from the pseudo-orbit generated by Indistinguishable States Gradient Descent method is shown to outperform the Inverse Noise ensemble in estimating the current states. Outside perfect model scenario, we demonstrate that forecast with relevant adjustment can produce better forecast than ignoring the existence of model error and using the model directly to make fore-casts. Measurement based on probabilistic forecast skill is suggested to measure the predictability outside PMS

A Framework for Noise Analysis and Verification of Analog Circuits

Analog circuit design and verification face significant challenges due to circuit complexity and short market windows. In particular, the influence of technology parameters on circuits, noise modeling and verification still remain a priority for many applications. Noise could be due to unwanted interaction between the various circuit blocks or it could be inherited from the circuit elements. Current industrial designs rely heavily on simulation techniques, but ensuring the correctness of such designs under all circumstances usually becomes impractically expensive. In this PhD thesis, we propose a methodology for modeling and verification of analog designs in the presence of noise and process variation using run-time verification methods. Verification based on run-time techniques employs logical or statistical monitors to check if an execution (simulation) of the design model violates the design specifications (properties). In order to study the random behavior of noise, we propose an approach based on modeling the designs using stochastic differential equations (SDE) in the time domain. Then, we define assertion and statistical verification methods in a MATLAB SDE simulation framework for monitoring properties of interest in order to detect errors. In order to overcome some of the drawbacks associated with monitoring techniques, we define a pattern matching based verification method for qualitative estimation of the simulation traces. We illustrate the efficiency of the proposed methods on different benchmark circuits