Quantifying uncertainty with ensembles of surrogates for blackbox optimization

This work is in the context of blackbox optimization where the functions defining the problem are expensive to evaluate and where no derivatives are available. A tried and tested technique is to build surrogates of the objective and the constraints in order to conduct the optimization at a cheaper computational cost. This work proposes different uncertainty measures when using ensembles of surrogates. The resulting combination of an ensemble of surrogates with our measures behaves as a stochastic model and allows the use of efficient Bayesian optimization tools. The method is incorporated in the search step of the mesh adaptive direct search (MADS) algorithm to improve the exploration of the search space. Computational experiments are conducted on seven analytical problems, two multi-disciplinary optimization problems and two simulation problems. The results show that the proposed approach solves expensive simulation-based problems at a greater precision and with a lower computational effort than stochastic models.Comment: 36 pages, 11 figures, submitte

System Architecture Optimization: An Open Source Multidisciplinary Aircraft Jet Engine Architecting Problem

Decisions regarding the system architecture are important and taken early in the design process, however suffer from large design spaces and expert bias. Systematic design space exploration techniques, like optimization, can be applied to system architecting. Realistic engineering benchmark problems are needed to enable development of optimization algorithms that can successfully solve these black-box, hierarchical, mixed-discrete, multi-objective architecture optimization problems. Such benchmark problems support the development of more capable optimization algorithms, more suitable methods for modeling system architecture design space, and educating engineers and other stakeholders on system architecture optimization in general. In this paper, an engine architecting benchmark problem is presented that exhibits all this behavior and is based on the open-source simulation tools pyCycle and OpenMDAO. Next to thermodynamic cycle analysis, the proposed benchmark problem includes modules for the estimation of engine weight, length, diameter, noise and NOx emissions. The problem is defined using modular interfaces, allowing to tune the complexity of the problem, by varying the number of design variables, objectives and constraints. The benchmark problem is validated by comparing to pyCycle example cases and existing engine performance data, and demonstrated using both a simple and a realistic problem formulation, solved using the multi-objective NSGA-II algorithm. It is shown that realistic results can be obtained, even though the design space is subject to hidden constraints due to the engine evaluation not converging for all design points

System Architecture Optimization: An Open Source Multidisciplinary Aircraft Jet Engine Architecting Problem

Decisions regarding the system architecture are important and taken early in the design process, however suffer from large design spaces and expert bias. Systematic design space exploration techniques, like optimization, can be applied to system architecting. Realistic engineering benchmark problems are needed to enable development of optimization algorithms that can successfully solve these black-box, hierarchical, mixed-discrete, multi-objective architecture optimization problems. Such benchmark problems support the development of more capable optimization algorithms, more suitable methods for modeling system architecture design space, and educating engineers and other stakeholders on system architecture optimization in general. In this paper, an engine architecting benchmark problem is presented that exhibits all this behavior and is based on the open-source simulation tools pyCycle and OpenMDAO. Next to thermodynamic cycle analysis, the proposed benchmark problem includes modules for the estimation of engine weight, length, diameter, noise and NOx emissions. The problem is defined using modular interfaces, allowing to tune the complexity of the problem, by varying the number of design variables, objectives and constraints. The benchmark problem is validated by comparing to pyCycle example cases and existing engine performance data, and demonstrated using both a simple and a realistic problem formulation, solved using the multi-objective NSGA-II algorithm. It is shown that realistic results can be obtained, even though the design space is subject to hidden constraints due to the engine evaluation not converging for all design points

Effectiveness of Surrogate-Based Optimization Algorithms for System Architecture Optimization

The design of complex system architectures brings with it a number of challenging issues, among others large combinatorial design spaces. Optimization can be applied to explore the design space, however gradient-based optimization algorithms cannot be applied due to the mixed-discrete nature of the design variables. It is investigated how effective surrogate-based optimization algorithms are for solving the black-box, hierarchical, mixed-discrete, multi-objective system architecture optimization problems. Performance is compared to the NSGA-II multi-objective evolutionary algorithm. An analytical benchmark problem that exhibits most important characteristics of architecture optimization is defined. First, an investigation into algorithm effectiveness is performed by measuring how accurately a known Pareto-front can be approximated for a fixed number of function evaluations. Then, algorithm efficiency is investigated by applying various multi-objective convergence criteria to the algorithms and establishing the possible trade-off between result quality and function evaluations needed. Finally, the impact of hidden constraints on algorithm performance is investigated. The code used for this paper has been published