Comparison of Gaussian process modeling software

Gaussian process fitting, or kriging, is often used to create a model from a set of data. Many available software packages do this, but we show that very different results can be obtained from different packages even when using the same data and model. We describe the parameterization, features, and optimization used by eight different fitting packages that run on four different platforms. We then compare these eight packages using various data functions and data sets, revealing that there are stark differences between the packages. In addition to comparing the prediction accuracy, the predictive variance--which is important for evaluating precision of predictions and is often used in stopping criteria--is also evaluated

A Spatial Approach to Addressing Weather Derivative Basis Risk: A Drought Insurance Example

Risk and Uncertainty,

Simulation-Optimization via Kriging and Bootstrapping:A Survey (Revision of CentER DP 2011-064)

Abstract: This article surveys optimization of simulated systems. The simulation may be either deterministic or random. The survey reflects the author’s extensive experience with simulation-optimization through Kriging (or Gaussian process) metamodels. The analysis of these metamodels may use parametric bootstrapping for deterministic simulation or distribution-free bootstrapping (or resampling) for random simulation. The survey covers: (1) Simulation-optimization through "efficient global optimization" (EGO) using "expected improvement" (EI); this EI uses the Kriging predictor variance, which can be estimated through parametric bootstrapping accounting for estimation of the Kriging parameters. (2) Optimization with constraints for multiple random simulation outputs and deterministic inputs through mathematical programming applied to Kriging metamodels validated through distribution-free bootstrapping. (3) Taguchian robust optimization for uncertain environments, using mathematical programming— applied to Kriging metamodels— and distribution- free bootstrapping to estimate the variability of the Kriging metamodels and the resulting robust solution. (4) Bootstrapping for improving convexity or preserving monotonicity of the Kriging metamodel.

Replication or exploration? Sequential design for stochastic simulation experiments

We investigate the merits of replication, and provide methods for optimal design (including replicates), with the goal of obtaining globally accurate emulation of noisy computer simulation experiments. We first show that replication can be beneficial from both design and computational perspectives, in the context of Gaussian process surrogate modeling. We then develop a lookahead based sequential design scheme that can determine if a new run should be at an existing input location (i.e., replicate) or at a new one (explore). When paired with a newly developed heteroskedastic Gaussian process model, our dynamic design scheme facilitates learning of signal and noise relationships which can vary throughout the input space. We show that it does so efficiently, on both computational and statistical grounds. In addition to illustrative synthetic examples, we demonstrate performance on two challenging real-data simulation experiments, from inventory management and epidemiology.Comment: 34 pages, 9 figure

Exposure and Exposure Modeling

Exposure to contaminants in the environment is quantified through the ecological risk assessment (ERA) process which provides a framework for the development and implementation of environmental management decisions. The ERA uses available toxicological and ecological information to estimate the probability of occurrence for a specified undesired ecological event or endpoint. The level for these endpoints depends on the objectives and the constraints imposed upon the risk assessment process; therefore, multiple endpoints at different scales may be necessary. ERAs Ecotoxicology | Exposure and Exposure Assessment 1527Author\u27s personal copy often rely on the link between these undesired endpoints to a threshold of exposure to specific toxicants and toxicant mixtures. Oral reference doses (RfD), inhalation reference concentrations (RfC), and carcinogenicity assessments are the usual way these links are expressed in the ERA, and unfortunately most of these thresholds have been developed for human health assessments and not ecosystem integrity. However, since these studies often use animal models, in many cases the original empirical data can be used when trying to apply these findings to ecological consequences or to establish ecological screening values (ESVs). The ecological exposure assessment often begins by comparing constituent concentrations in media (surface water, sediment, soil) to ESVs. The ESVs are derived from ecologically relevant criteria and standards. For example, in the United States the United States Environmental Protection Agency (USEPA) Screening Values and National Ambient Water Quality Criteria (NAWQC) are often used based on ‘no observed adverse effect levels’ (NOAELs) or ‘lowest observed adverse effect levels’ (LOAELs) derived from literature to assess exposure. Radionuclide comparisons for ecological screening are typically dose-based for population level effects. In addition to the ecological threshold comparison, constituents that may bioaccumulate/bioconcentrate are identified during initial screening processes. This is done to account for toxicants that may not be present at levels exceeding ESVs, but must be considered due to trophic transfer of toxicants that may concentrate in higher-trophic-level organisms. Constituents that exceed ESV comparisons (present with means, maximums, or 95% upper confidence levels (UCLs)) are evaluated using a lines-of-evidence approach based on (1) a background evaluation, (2) a bioaccumulation/ bioconcentration potential and ecotoxicity evaluation, (3) a frequency and pattern-of-exceedances evaluation based on review of exceedances to the ESVs, and (4) an evaluation of existing biological data. From this information, ecosystems can be prioritized in terms of risk and focused for proper exposure assessments. This article presents a scientific overview and review of how toxicant exposure is estimated and applied to assess ecosystem integrity