Designs for computer experiments and uncertainty quantification

Computer experiments are widely-used in analysis of real systems where physical experiments are infeasible or unaffordable. Computer models are usually complex and computationally demanding, consequently, time consuming to run. Therefore, surrogate models, also known as emulators, are fitted to approximate these computationally intensive computer models. Since emulators are easy-to-evaluate they may replace computer models in the actual analysis of the systems. Experimental design for computer simulations and modeling of simulated outputs are two important aspects of building accurate emulators. This thesis consists of three chapters, covering topics in design of computer experiments and uncertainty quantification of complex computer models. The first chapter proposes a new type of space-filling designs for computer experiments, and the second chapter develops an emulator-based approach for uncertainty quantification of machining processes using their computer simulations. Finally, third chapter extends the experimental designs proposed in the first chapter and enables to generate designs with both quantitative and qualitative factors. In design of computer experiments, space-filling properties are important. The traditional maximin and minimax distance designs consider only space-fillingness in the full-dimensional space which can result in poor projections onto lower-dimensional spaces, which is undesirable when only a few factors are active. On the other hand, restricting maximin distance design to the class of Latin hypercubes can improve one-dimensional projections but cannot guarantee good space-filling properties in larger subspaces. In the first chapter, we propose designs that maximize space-filling properties on projections to all subsets of factors. Proposed designs are called maximum projection designs. Maximum projection designs have better space-filling properties in their projections compared to other widely-used space-filling designs. They also provide certain advantages in Gaussian process modeling. More importantly, the design criterion can be computed at a cost no more than that of a design criterion which ignores projection properties. In the second chapter, we develop an uncertainty quantification methodology for machining processes with uncertain input factors. Understanding the uncertainty in a machining process using the simulation outputs is important for careful decision making. However, Monte Carlo-based methods cannot be used for evaluating the uncertainty when the simulations are computationally expensive. An alternative approach is to build an easy-to-evaluate emulator to approximate the computer model and run the Monte Carlo simulations on the emulator. Although this approach is very promising, it becomes inefficient when the computer model is highly nonlinear and the region of interest is large. Most machining simulations are of this kind because the output is affected by a large number of parameters including the workpiece material properties, cutting tool parameters, and process parameters. Building an accurate emulator that works for different kinds of materials, tool designs, tool paths, etc. is not an easy task. We propose a new approach, called in-situ emulator, to overcome this problem. The idea is to build an emulator in a local region defined by the user-specified input uncertainty distribution. We use maximum projection designs and Gaussian process modeling techniques for constructing the in-situ emulator. On two solid end milling processes, we show that the in-situ emulator methodology is efficient and accurate in uncertainty quantification and has apparent advantages over other conventional tools. Computer experiments with quantitative and qualitative factors are prevalent. In the third chapter, we extend maximum projection designs so that they can accommodate qualitative factors as well. Proposed designs maintain an economic run size and they are flexible in run size, number of quantitative and qualitative factors and factor levels. Their construction is not restricted to a special design class and does not impose any design configuration. A general construction algorithm, which utilizes orthogonal arrays, is developed. We have shown on several simulations that maximum projection designs with both quantitative and qualitative factors have attractive space-filling properties for all of their projections. Their advantages are also illustrated on optimization of a solid end milling process simulation. Finally, we propose a methodology for sequential construction of maximum projection designs which ensures efficient analysis of systems within financial cost and time constraints. The performance of the sequential construction methodology is demonstrated using the optimization of a solid end milling process.Ph.D

Sliced rotated sphere packing designs

Space-filling designs are popular choices for computer experiments. A sliced design is a design that can be partitioned into several subdesigns. We propose a new type of sliced space-filling design called sliced rotated sphere packing designs. Their full designs and subdesigns are rotated sphere packing designs. They are constructed by rescaling, rotating, translating and extracting the points from a sliced lattice. We provide two fast algorithms to generate such designs. Furthermore, we propose a strategy to use sliced rotated sphere packing designs adaptively. Under this strategy, initial runs are uniformly distributed in the design space, follow-up runs are added by incorporating information gained from initial runs, and the combined design is space-filling for any local region. Examples are given to illustrate its potential application

An empirical learning-based validation procedure for simulation workflow

Simulation workflow is a top-level model for the design and control of simulation process. It connects multiple simulation components with time and interaction restrictions to form a complete simulation system. Before the construction and evaluation of the component models, the validation of upper-layer simulation workflow is of the most importance in a simulation system. However, the methods especially for validating simulation workflow is very limit. Many of the existing validation techniques are domain-dependent with cumbersome questionnaire design and expert scoring. Therefore, this paper present an empirical learning-based validation procedure to implement a semi-automated evaluation for simulation workflow. First, representative features of general simulation workflow and their relations with validation indices are proposed. The calculation process of workflow credibility based on Analytic Hierarchy Process (AHP) is then introduced. In order to make full use of the historical data and implement more efficient validation, four learning algorithms, including back propagation neural network (BPNN), extreme learning machine (ELM), evolving new-neuron (eNFN) and fast incremental gaussian mixture model (FIGMN), are introduced for constructing the empirical relation between the workflow credibility and its features. A case study on a landing-process simulation workflow is established to test the feasibility of the proposed procedure. The experimental results also provide some useful overview of the state-of-the-art learning algorithms on the credibility evaluation of simulation models

G\mathcal{G}-SELC: Optimization by sequential elimination of level combinations using genetic algorithms and Gaussian processes

Identifying promising compounds from a vast collection of feasible compounds is an important and yet challenging problem in the pharmaceutical industry. An efficient solution to this problem will help reduce the expenditure at the early stages of drug discovery. In an attempt to solve this problem, Mandal, Wu and Johnson [Technometrics 48 (2006) 273--283] proposed the SELC algorithm. Although powerful, it fails to extract substantial information from the data to guide the search efficiently, as this methodology is not based on any statistical modeling. The proposed approach uses Gaussian Process (GP) modeling to improve upon SELC, and hence named $\mathcal{G}$-SELC. The performance of the proposed methodology is illustrated using four and five dimensional test functions. Finally, we implement the new algorithm on a real pharmaceutical data set for finding a group of chemical compounds with optimal properties.Comment: Published in at http://dx.doi.org/10.1214/08-AOAS199 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Towards Visually Explaining Variational Autoencoders

Recent advances in Convolutional Neural Network (CNN) model interpretability have led to impressive progress in visualizing and understanding model predictions. In particular, gradient-based visual attention methods have driven much recent effort in using visual attention maps as a means for visual explanations. A key problem, however, is these methods are designed for classification and categorization tasks, and their extension to explaining generative models, e.g. variational autoencoders (VAE) is not trivial. In this work, we take a step towards bridging this crucial gap, proposing the first technique to visually explain VAEs by means of gradient-based attention. We present methods to generate visual attention from the learned latent space, and also demonstrate such attention explanations serve more than just explaining VAE predictions. We show how these attention maps can be used to localize anomalies in images, demonstrating state-of-the-art performance on the MVTec-AD dataset. We also show how they can be infused into model training, helping bootstrap the VAE into learning improved latent space disentanglement, demonstrated on the Dsprites dataset