Data-Efficient Design and Analysis Methodologies for Computer and Physical Experiments

Data science for experimentation, including the rapidly growing area of the design and analysis of computer experiments, aims to use statistical approaches to collect and analyze (physical or virtual) experimental responses and facilitate decision-making. The cost for each run of an experiment can be expensive. This dissertation proposes novel data-efficient methodologies to tackle three different challenges in this field. The first two are regarding computer experiments, and the third one is regarding physical experiments. The first work aims to reconstruct the input-output relationship (surrogate model) given by the computer code via scattered evaluations with small sizes based on Gaussian process regression. Traditional isotropic Gaussian process models suffer from the curse of dimensionality when the input dimension is relatively high given limited data points. Gaussian process models with additive correlation functions are scalable to dimensionality, but they are more restrictive as they only work for additive functions. In the first work, we consider a projection pursuit model in which the nonparametric part is driven by an additive Gaussian process regression. We choose the dimension of the additive function higher than the original input dimension and call this strategy “dimension expansion”. We show that dimension expansion can help approximate more complex functions. A gradient descent algorithm is proposed for model training based on the maximum likelihood estimation. Simulation studies show that the proposed method outperforms the traditional Gaussian process models. The second work focuses on the designs of experiments (DoE) of multi-fidelity computer experiments with fixed budget. We consider the autoregressive Gaussian process model and the optimal nested design that maximizes the prediction accuracy subject to the budget constraint. An approximate solution is identified through the idea of multilevel approximation and recent error bounds of Gaussian process regression. The proposed (approximately) optimal designs admit a simple analytical form. We prove that, to achieve the same prediction accuracy, the proposed optimal multi-fidelity design requires much lower computational cost than any single-fidelity design in the asymptotic sense. The last work is proposed to model complex experiments when the distributions of training and testing input features are different (referred to as domain adaptation). In this work, we propose a novel transfer learning algorithm called Renewing Iterative Self-labeling Domain Adaptation (Re-ISDA) to tackle the domain adaptation problem. The learning problem is formulated as a dynamic programming model, and the latter is then solved by an efficient greedy algorithm by adding a renewing step to the original ISDA algorithm. This renewing step helps avoid a potential issue of the ISDA that the possible mis-labeled samples by a weak predictor in the initial stage of the iterative learning can cause serious harm to the subsequent learning process. Numerical studies show that the proposed method outperforms prevailing transfer learning methods. The proposed method also achieves high prediction accuracy for a cervical spine motion problem

A CFD-informed model for subchannel resolution crud prediction

A physics-directed, statistically based, surrogate model of the small scale flow fea- tures that impact Chalk River unidentified deposit (crud) growth is presented in this work. The objective of the surrogate is to provide additional details of the rod surface temperature, heat flux, and near-wall turbulent kinetic energy fields which cannot be explicitly captured by a subchannel code. Operating as a mapping from the high fidelity computational fluid dynamics (CFD) data to the low fidelity subchannel grid (hi2lo), the model provides CFD-informed bound- ary conditions to the crud model executed on the subchannel pin surface mesh. The surface temperature, heat flux, and turbulent kinetic energy, henceforth referred to as the fields of interest (FOI), govern the growth rate of crud on the surface of the rod and the precipitation of boron in the porous crud layer. Therefore the model predicts the behavior of the FOIs as a function of position in the core and local thermal-hydraulic (TH) conditions. The subchannel code produces an estimate for all crud-relevant TH quantities at a coarse spatial resolution everywhere in the core and executes substantially faster than CFD. In the hi2lo approach, the solution provided by the subchannel code is augmented by a predicted stochastic component of the FOI informed by CFD results to provide a more detailed description of the target FOIs than subchannel can provide alone. To this end, a novel method based on the marriage of copula and gradient boosting techniques is proposed. This methodology forgoes a spatial interpolation procedure for a statistically driven approach, which predicts the fractional area of a rod’s surface in excess of some critical temperature but not precisely where such maxima occur on the rod surface. The resultant model retains the ability to account for the presence of hot and cold spots on the rod surface induced by turbulent flow downstream of spacer grids when producing crud estimates. Sklar’s theorem is leveraged to decompose multivariate probability densities of the FOI into independent copula and marginal models. The free parameters within the copula model are predicted using a combination of supervised regression and classification machine learning techniques with training data sets supplied by a suite of precomputed CFD results spanning a typical pressurized water reactor TH envelope. Results show that compared to the subchannel standalone case, the hi2lo method more accurately preserves the influence of spacer grids on the crud growth rate. Or more precisely, the hi2lo method recovers key statistical properties of the FOI which impact crud growth. Compared to gold standard high fidelity CFD/crud coupled results in a single assembly test case, the hi2lo model produced a relative total crud mass difference of -8.9% compared to the standalone subchannel relative crud mass difference of 192.1%.Mechanical Engineerin

Surrogate Models Coupled with Machine Learning to Approximate Complex Physical Phenomena Involving Aerodynamic and Aerothermal Simulations

Numerical simulations provide a key element in aircraft design process, complementing physical tests and flight tests. They could take advantage of innovative methods, such as artificial intelligence technologies spreading in aviation. Simulating the full flight mission for various disciplines pose important problems due to significant computational cost coupled to varying operating conditions. Moreover, complex physical phenomena can occur. For instance, the aerodynamic field on the wing takes different shapes and can encounter shocks, while aerothermal simulations around nacelle and pylon are sensitive to the interaction between engine flows and external flows. Surrogate models can be used to substitute expensive high-fidelitysimulations by mathematical and statistical approximations in order to reduce overall computation cost and to provide a data-driven approach. In this thesis, we propose two developments: (i) machine learning-based surrogate models capable of approximating aerodynamic experiments and (ii) integrating more classical surrogate models into industrial aerothermal process. The first approach mitigates aerodynamic issues by separating solutions with very different shapes into several subsets using machine learning algorithms. Moreover, a resampling technique takes advantage of the subdomain decomposition by adding extra information in relevant regions. The second development focuses on pylon sizing by building surrogate models substitutingaerothermal simulations. The two approaches are applied to aircraft configurations in order to bridge the gap between academic methods and real-world applications. Significant improvements are highlighted in terms of accuracy and cost gain

New strategies for the aerodynamic design optimization of aeronautical configurations through soft-computing techniques

Premio Extraordinario de Doctorado de la UAH en 2013Lozano Rodríguez, Carlos, codir.This thesis deals with the improvement of the optimization process in the aerodynamic design of aeronautical configurations. Nowadays, this topic is of great importance in order to allow the European aeronautical industry to reduce their development and operational costs, decrease the time-to-market for new aircraft, improve the quality of their products and therefore maintain their competitiveness. Within this thesis, a study of the state-of-the-art of the aerodynamic optimization tools has been performed, and several contributions have been proposed at different levels: -One of the main drawbacks for an industrial application of aerodynamic optimization tools is the huge requirement of computational resources, in particular, for complex optimization problems, current methodological approaches would need more than a year to obtain an optimized aircraft. For this reason, one proposed contribution of this work is focused on reducing the computational cost by the use of different techniques as surrogate modelling, control theory, as well as other more software-related techniques as code optimization and proper domain parallelization, all with the goal of decreasing the cost of the aerodynamic design process. -Other contribution is related to the consideration of the design process as a global optimization problem, and, more specifically, the use of evolutionary algorithms (EAs) to perform a preliminary broad exploration of the design space, due to their ability to obtain global optima. Regarding this, EAs have been hybridized with metamodels (or surrogate models), in order to substitute expensive CFD simulations. In this thesis, an innovative approach for the global aerodynamic optimization of aeronautical configurations is proposed, consisting of an Evolutionary Programming algorithm hybridized with a Support Vector regression algorithm (SVMr) as a metamodel. Specific issues as precision, dataset training size, geometry parameterization sensitivity and techniques for design of experiments are discussed and the potential of the proposed approach to achieve innovative shapes that would not be achieved with traditional methods is assessed. -Then, after a broad exploration of the design space, the optimization process is continued with local gradient-based optimization techniques for a finer improvement of the geometry. Here, an automated optimization framework is presented to address aerodynamic shape design problems. Key aspects of this framework include the use of the adjoint methodology to make the computational requirements independent of the number of design variables, and Computer Aided Design (CAD)-based shape parameterization, which uses the flexibility of Non-Uniform Rational B-Splines (NURBS) to handle complex configurations. The mentioned approach is applied to the optimization of several test cases and the improvements of the proposed strategy and its ability to achieve efficient shapes will complete this study

New strategies for the aerodynamic design optimization of aeronautical configurations through soft-computing techniques

Premio Extraordinario de Doctorado de la UAH en 2013Lozano Rodríguez, Carlos, codir.This thesis deals with the improvement of the optimization process in the aerodynamic design of aeronautical configurations. Nowadays, this topic is of great importance in order to allow the European aeronautical industry to reduce their development and operational costs, decrease the time-to-market for new aircraft, improve the quality of their products and therefore maintain their competitiveness. Within this thesis, a study of the state-of-the-art of the aerodynamic optimization tools has been performed, and several contributions have been proposed at different levels: -One of the main drawbacks for an industrial application of aerodynamic optimization tools is the huge requirement of computational resources, in particular, for complex optimization problems, current methodological approaches would need more than a year to obtain an optimized aircraft. For this reason, one proposed contribution of this work is focused on reducing the computational cost by the use of different techniques as surrogate modelling, control theory, as well as other more software-related techniques as code optimization and proper domain parallelization, all with the goal of decreasing the cost of the aerodynamic design process. -Other contribution is related to the consideration of the design process as a global optimization problem, and, more specifically, the use of evolutionary algorithms (EAs) to perform a preliminary broad exploration of the design space, due to their ability to obtain global optima. Regarding this, EAs have been hybridized with metamodels (or surrogate models), in order to substitute expensive CFD simulations. In this thesis, an innovative approach for the global aerodynamic optimization of aeronautical configurations is proposed, consisting of an Evolutionary Programming algorithm hybridized with a Support Vector regression algorithm (SVMr) as a metamodel. Specific issues as precision, dataset training size, geometry parameterization sensitivity and techniques for design of experiments are discussed and the potential of the proposed approach to achieve innovative shapes that would not be achieved with traditional methods is assessed. -Then, after a broad exploration of the design space, the optimization process is continued with local gradient-based optimization techniques for a finer improvement of the geometry. Here, an automated optimization framework is presented to address aerodynamic shape design problems. Key aspects of this framework include the use of the adjoint methodology to make the computational requirements independent of the number of design variables, and Computer Aided Design (CAD)-based shape parameterization, which uses the flexibility of Non-Uniform Rational B-Splines (NURBS) to handle complex configurations. The mentioned approach is applied to the optimization of several test cases and the improvements of the proposed strategy and its ability to achieve efficient shapes will complete this study