Replication or exploration? Sequential design for stochastic simulation experiments

We investigate the merits of replication, and provide methods for optimal design (including replicates), with the goal of obtaining globally accurate emulation of noisy computer simulation experiments. We first show that replication can be beneficial from both design and computational perspectives, in the context of Gaussian process surrogate modeling. We then develop a lookahead based sequential design scheme that can determine if a new run should be at an existing input location (i.e., replicate) or at a new one (explore). When paired with a newly developed heteroskedastic Gaussian process model, our dynamic design scheme facilitates learning of signal and noise relationships which can vary throughout the input space. We show that it does so efficiently, on both computational and statistical grounds. In addition to illustrative synthetic examples, we demonstrate performance on two challenging real-data simulation experiments, from inventory management and epidemiology.Comment: 34 pages, 9 figure

Kriging-Based Design of Experiments for Multi-Source Exposure-Response Studies in Nanotoxicology

One of the major challenges with toxicology studies of nanomaterials (NMs), compared to traditional materials or chemicals, lies in the large NM variety (or sources) caused by their various physico-chemical properties. How to efficiently design multi-source biological experiments for the toxicity characterization of NMs in terms of their exposure-response profiles? This work intends to address this question by a two-stage experimental design procedure, which is developed based on the statistical model, stochastic kriging with qualitative factors (SKQ). With a given experimental budget, the SKQ-based design method aims at achieving the highest-quality SKQ, which synergistically models the exposure-response data from multiple sources (e.g., NM types). The method determines the experimental design (that is, the sampling location as well as allocation) in such a way that the resulting sampling data allow SKQ to realize its maximum potential to pool information across multiple sources for efficient modeling. Built in a two-stage framework, which enables a learning process of the target exposure-response relationships, the SKQ-based design procedure also inherits the general advantages of stochastic kriging in the sense that the design is particularly tailored to model the possibly nonlinear and complex relationships and heterogeneous data variances. Through simulation studies, the efficiency of the SKQ-based procedure for multi-source experiments is demonstrated over the two alternative design methods

A New Gaussian Process Method For Modeling and Design of Multi-Source Data in Exposure-Response Toxicology Studies

One of the most fundamental steps in risk assessment is to quantify the exposure-response relationship for the material/chemical of interest. This work develops a new statistical method, referred to as SKQ (stochastic kriging with qualitative factors), to synergistically model exposure-response data, which often arise from multiple sources (e.g., laboratories, animal providers, and shapes of nanomaterials) in toxicology studies. Compared to the existing methods, SKQ has several distinct features. First of all, SKQ integrates data across multiple sources, and allows for the derivation of more accurate information from limited data. Second, SKQ is highly flexible and able to model practically any continuous response surfaces (e.g., dose-time-response surface). Third, SKQ is able to accommodate variance heterogeneity across experimental conditions, and to provide valid statistical inference (i.e., quantify uncertainties of the model estimates). Through empirical studies, we have demonstrated SKQ\u27s ability to efficiently model exposure-response surfaces by pooling information across multiple data sources.;Based on the SKQ modeling and inference, a design of experiments (DOE) procedure is developed to guide biological experiments for the efficient quantification of exposure-response relationships. Built on SKQ, the DOE procedure inherits the advantages of SKQ and is particularly tailored for experimental data arising from multiple sources, with non-normality and variance heterogeneity, and mapping nonlinear exposure-response relationships. The design procedure is built in a sequential two-stage paradigm that allows for a learning process: In the first stage, preliminary experiments are performed to gain information regarding the underlying exposure-response curve and variance structure; in the second stage, the prior information obtained from the previous stage is utilized to guide the second-stage experiments. Matlab\u27s global optimization function MultiStart is employed to search for optimal designs that will lead to exposure-response models of the highest quality.;SKQ and SKQ-based DOE fit into the mosaic of efficient decision-making methods for assessing the risk of a tremendously large variety of nanomaterials, and helps to alleviate the sustainability concerns regarding the enormous new nanomaterials