Design and Analysis of Monte Carlo Experiments

monte carlo experiments;simulation models;mathematical analysis;sensitivity analysis;experimental design

Driving efficiency in design for rare events using metamodeling and optimization

Rare events have very low probability of occurrence but can have significant impact. Earthquakes, volcanoes, and stock market crashes can have devastating impact on those affected. In industry, engineers evaluate rare events to design better high-reliability systems. The objective of this work is to increase efficiency in design optimization for rare events using metamodeling and variance reduction techniques. Opportunity exists to increase deterministic optimization efficiency by leveraging Design of Experiments to build an accurate metamodel of the system which is less resource intensive to evaluate than the real system. For computationally expensive models, running many trials will impede fast design iteration. Accurate metamodels can be used in place of these expensive models to probabilistically optimize the system for efficient quantification of rare event risk. Monte Carlo is traditionally used for this risk quantification but variance reduction techniques such as importance sampling allow accurate quantification with fewer model evaluations. Metamodel techniques are the thread that tie together deterministic optimization using Design of Experiments and probabilistic optimization using Monte Carlo and variance reduction. This work will explore metamodeling theory and implementation, and outline a framework for efficient deterministic and probabilistic system optimization. The overall conclusion is that deterministic and probabilistic simulation can be combined through metamodeling and used to drive efficiency in design optimization. Applications are demonstrated on a gas turbine combustion autoignition application where user controllable independent variables are optimized in mean and variance to maximize system performance while observing a constraint on allowable probability of a rare autoignition event

Expected Improvement in Efficient Global Optimization Through Bootstrapped Kriging - Replaces CentER DP 2010-62

This article uses a sequentialized experimental design to select simulation input com- binations for global optimization, based on Kriging (also called Gaussian process or spatial correlation modeling); this Kriging is used to analyze the input/output data of the simulation model (computer code). This design and analysis adapt the clas- sic "expected improvement" (EI) in "efficient global optimization" (EGO) through the introduction of an unbiased estimator of the Kriging predictor variance; this estimator uses parametric bootstrapping. Classic EI and bootstrapped EI are com- pared through various test functions, including the six-hump camel-back and several Hartmann functions. These empirical results demonstrate that in some applications bootstrapped EI finds the global optimum faster than classic EI does; in general, however, the classic EI may be considered to be a robust global optimizer.Simulation;Optimization;Kriging;Bootstrap

Clustering based Multiple Anchors High-Dimensional Model Representation

In this work, a cut high-dimensional model representation (cut-HDMR) expansion based on multiple anchors is constructed via the clustering method. Specifically, a set of random input realizations is drawn from the parameter space and grouped by the centroidal Voronoi tessellation (CVT) method. Then for each cluster, the centroid is set as the reference, thereby the corresponding zeroth-order term can be determined directly. While for non-zero order terms of each cut-HDMR, a set of discrete points is selected for each input component, and the Lagrange interpolation method is applied. For a new input, the cut-HDMR corresponding to the nearest centroid is used to compute its response. Numerical experiments with high-dimensional integral and elliptic stochastic partial differential equation as backgrounds show that the CVT based multiple anchors cut-HDMR can alleviate the negative impact of a single inappropriate anchor point, and has higher accuracy than the average of several expansions

A Recommendation System for Meta-modeling: A Meta-learning Based Approach

Various meta-modeling techniques have been developed to replace computationally expensive simulation models. The performance of these meta-modeling techniques on different models is varied which makes existing model selection/recommendation approaches (e.g., trial-and-error, ensemble) problematic. To address these research gaps, we propose a general meta-modeling recommendation system using meta-learning which can automate the meta-modeling recommendation process by intelligently adapting the learning bias to problem characterizations. The proposed intelligent recommendation system includes four modules: (1) problem module, (2) meta-feature module which includes a comprehensive set of meta-features to characterize the geometrical properties of problems, (3) meta-learner module which compares the performance of instance-based and model-based learning approaches for optimal framework design, and (4) performance evaluation module which introduces two criteria, Spearman\u27s ranking correlation coefficient and hit ratio, to evaluate the system on the accuracy of model ranking prediction and the precision of the best model recommendation, respectively. To further improve the performance of meta-learning for meta-modeling recommendation, different types of feature reduction techniques, including singular value decomposition, stepwise regression and ReliefF, are studied. Experiments show that our proposed framework is able to achieve 94% correlation on model rankings, and a 91% hit ratio on best model recommendation. Moreover, the computational cost of meta-modeling recommendation is significantly reduced from an order of minutes to seconds compared to traditional trial-and-error and ensemble process. The proposed framework can significantly advance the research in meta-modeling recommendation, and can be applied for data-driven system modeling

Non-Linear Metamodeling Extensions to the Robust Parameter Design of Computer Simulations

Robust parameter design (RPD) is used to identify a systems control settings that offer a compromise between obtaining desired mean responses and minimizing the variability about those responses. Two popular combined-array strategies the response surface model (RSM) approach and the emulator approach are limited when applied to simulations. In the former case, the mean and variance models can be inadequate due to a high level of non-linearity within many simulations. In the latter case, precise mean and variance approximations are developed at the expense of extensive Monte Carlo sampling. This research combines the RSM approach\u27s efficiency with the emulator approach\u27s accuracy. Non-linear metamodeling extensions, namely through Kriging and radial basis function neural networks, are made to the RSM approach. The mean and variance of second-order Taylor series approximations of these metamodels are generated via the Multivariate Delta Method and subsequent optimization problems employing these approximations are solved. Results show that improved prediction models can be attained through the proposed approach at a reduced computational cost. Additionally, a multi-response RPD problem solving technique based on desirability functions is presented to produce a solution that is mutually robust across all responses. Lastly, quality measures are developed to provide a holistic assessment of several competing RPD strategies