TOWARD A MODEL-DRIVEN ENGINEERING FRAMEWORK FOR REPRODUCIBLE SIMULATION EXPERIMENT LIFECYCLE MANAGEMENT

Goal-directed reproducible experimentation with simulation models is still a significant challenge. The underutilization of design of experiments, limited transparency in the collection and analysis of results, and ad-hoc adaptation of experiments as learning takes place continue to hamper reproducibility and hence cause a credibility gap. In this study, we propose a strategy that leverages the synergies between model-driven engineering, intelligent agent technology, and variability modeling to support the management of the lifecycle of a simulation experiment. Experiment design and workflow models are introduced for configurable experiment synthesis and execution. Feature-based variability modeling is used to design a family of experiments, which can be leveraged by ontology-driven software agents to configure, execute, and reproduce experiments. Online experiment adaptation is proposed as a strategy to facilitate dynamic experiment model updating as objectives shift from validation to variable screening, understanding, and optimization

Reusing simulation experiments for model composition and extension

This thesis aims to reuse simulation experiments to support developing models via model reuse, with a focus on validating the resulting model. Individual models are annotated with their simulation experiments. Upon reuse of those models for building new ones, the associated simulation experiments are also reused and executed with the new model, to inspect whether the key behavior exhibited by the original models is preserved or not in the new model. Hence, the changes of model behavior resulting from the model reuse are revealed, and insights into validity of the new model are provided