Tailoring the Statistical Experimental Design Process for LVC Experiments

The use of Live, Virtual and Constructive (LVC) Simulation environments are increasingly being examined for potential analytical use particularly in test and evaluation. The LVC simulation environments provide a mechanism for conducting joint mission testing and system of systems testing when scale and resource limitations prevent the accumulation of the necessary density and diversity of assets required for these complex and comprehensive tests. The statistical experimental design process is re-examined for potential application to LVC experiments and several additional considerations are identified to augment the experimental design process for use with LVC. This augmented statistical experimental design process is demonstrated by a case study involving a series of tests on an experimental data link for strike aircraft using LVC simulation for the test environment. The goal of these tests is to assess the usefulness of information being presented to aircrew members via different datalink capabilities. The statistical experimental design process is used to structure the experiment leading to the discovery of faulty assumptions and planning mistakes that could potentially wreck the results of the experiment. Lastly, an aggressive sequential experimentation strategy is presented for LVC experiments when test resources are limited. This strategy depends on a foldover algorithm that we developed for nearly orthogonal arrays to rescue LVC experiments when important factor effects are confounded

Optimal exact designs of experiments via Mixed Integer Nonlinear Programming

Optimal exact designs are problematic to find and study because there is no unified theory for determining them and studyingtheir properties. Each has its own challenges and when a method exists to confirm the design optimality, it is invariablyapplicable to the particular problem only.We propose a systematic approach to construct optimal exact designs by incorporatingthe Cholesky decomposition of the Fisher Information Matrix in a Mixed Integer Nonlinear Programming formulation. Asexamples, we apply the methodology to find D- and A-optimal exact designs for linear and nonlinear models using global orlocal optimizers. Our examples include design problems with constraints on the locations or the number of replicates at theoptimal design points

Listing Unique Fractional Factorial Designs

Fractional factorial designs are a popular choice in designing experiments for studying the effects of multiple factors simultaneously. The first step in planning an experiment is the selection of an appropriate fractional factorial design. An appro- priate design is one that has the statistical properties of interest of the experimenter and has a small number of runs. This requires that a catalog of candidate designs be available (or be possible to generate) for searching for the "good" design. In the attempt to generate the catalog of candidate designs, the problem of design isomor- phism must be addressed. Two designs are isomorphic to each other if one can be obtained from the other by some relabeling of factor labels, level labels of each factor and reordering of runs. Clearly, two isomorphic designs are statistically equivalent. Design catalogs should therefore contain only designs unique up to isomorphism. There are two computational challenges in generating such catalogs. Firstly, testing two designs for isomorphism is computationally hard due to the large number of possible relabelings, and, secondly, the number of designs increases very rapidly with the number of factors and run-size, making it impractical to compare all designs for isomorphism. In this dissertation we present a new approach for tackling both these challenging problems. We propose graph models for representing designs and use this relationship to develop efficient algorithms. We provide a new efficient iso- morphism check by modeling the fractional factorial design isomorphism problem as graph isomorphism problem. For generating the design catalogs efficiently we extend a result in graph isomorphism literature to improve the existing sequential design catalog generation algorithm. The potential of the proposed methods is reflected in the results. For 2-level regular fractional factorial designs, we could generate complete design catalogs of run sizes up to 4096 runs, while the largest designs generated in literature are 512 run designs. Moreover, compared to the next best algorithms, the computation times for our algorithm are 98% lesser in most cases. Further, the generic nature of the algorithms makes them widely applicable to a large class of designs. We give details of graph models and prove the results for two classes of designs, namely, 2-level regular fractional factorial designs and 2-level regular fractional factorial split-plot designs, and provide discussions for extensions, with graph models, for more general classes of designs

Bayesian D-Optimal Design of Experiments with Quantitative and Qualitative Responses

Systems with both quantitative and qualitative responses are widely encountered in many applications. Design of experiment methods are needed when experiments are conducted to study such systems. Classic experimental design methods are unsuitable here because they often focus on one type of response. In this paper, we develop a Bayesian D-optimal design method for experiments with one continuous and one binary response. Both noninformative and conjugate informative prior distributions on the unknown parameters are considered. The proposed design criterion has meaningful interpretations regarding the D-optimality for the models for both types of responses. An efficient point-exchange search algorithm is developed to construct the local D-optimal designs for given parameter values. Global D-optimal designs are obtained by accumulating the frequencies of the design points in local D-optimal designs, where the parameters are sampled from the prior distributions. The performances of the proposed methods are evaluated through two examples