16,648 research outputs found
MATSuMoTo: The MATLAB Surrogate Model Toolbox For Computationally Expensive Black-Box Global Optimization Problems
MATSuMoTo is the MATLAB Surrogate Model Toolbox for computationally
expensive, black-box, global optimization problems that may have continuous,
mixed-integer, or pure integer variables. Due to the black-box nature of the
objective function, derivatives are not available. Hence, surrogate models are
used as computationally cheap approximations of the expensive objective
function in order to guide the search for improved solutions. Due to the
computational expense of doing a single function evaluation, the goal is to
find optimal solutions within very few expensive evaluations. The multimodality
of the expensive black-box function requires an algorithm that is able to
search locally as well as globally. MATSuMoTo is able to address these
challenges. MATSuMoTo offers various choices for surrogate models and surrogate
model mixtures, initial experimental design strategies, and sampling
strategies. MATSuMoTo is able to do several function evaluations in parallel by
exploiting MATLAB's Parallel Computing Toolbox.Comment: 13 pages, 7 figure
Fast stable direct fitting and smoothness selection for Generalized Additive Models
Existing computationally efficient methods for penalized likelihood GAM
fitting employ iterative smoothness selection on working linear models (or
working mixed models). Such schemes fail to converge for a non-negligible
proportion of models, with failure being particularly frequent in the presence
of concurvity. If smoothness selection is performed by optimizing `whole model'
criteria these problems disappear, but until now attempts to do this have
employed finite difference based optimization schemes which are computationally
inefficient, and can suffer from false convergence. This paper develops the
first computationally efficient method for direct GAM smoothness selection. It
is highly stable, but by careful structuring achieves a computational
efficiency that leads, in simulations, to lower mean computation times than the
schemes based on working-model smoothness selection. The method also offers a
reliable way of fitting generalized additive mixed models
Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations
We construct a new framework for accelerating Markov chain Monte Carlo in
posterior sampling problems where standard methods are limited by the
computational cost of the likelihood, or of numerical models embedded therein.
Our approach introduces local approximations of these models into the
Metropolis-Hastings kernel, borrowing ideas from deterministic approximation
theory, optimization, and experimental design. Previous efforts at integrating
approximate models into inference typically sacrifice either the sampler's
exactness or efficiency; our work seeks to address these limitations by
exploiting useful convergence characteristics of local approximations. We prove
the ergodicity of our approximate Markov chain, showing that it samples
asymptotically from the \emph{exact} posterior distribution of interest. We
describe variations of the algorithm that employ either local polynomial
approximations or local Gaussian process regressors. Our theoretical results
reinforce the key observation underlying this paper: when the likelihood has
some \emph{local} regularity, the number of model evaluations per MCMC step can
be greatly reduced without biasing the Monte Carlo average. Numerical
experiments demonstrate multiple order-of-magnitude reductions in the number of
forward model evaluations used in representative ODE and PDE inference
problems, with both synthetic and real data.Comment: A major update of the theory and example
Towards efficient multiobjective optimization: multiobjective statistical criterions
The use of Surrogate Based Optimization (SBO) is widely spread in engineering design to reduce the number of computational expensive simulations. However, "real-world" problems often consist of multiple, conflicting objectives leading to a set of equivalent solutions (the Pareto front). The objectives are often aggregated into a single cost function to reduce the computational cost, though a better approach is to use multiobjective optimization methods to directly identify a set of Pareto-optimal solutions, which can be used by the designer to make more efficient design decisions (instead of making those decisions upfront). Most of the work in multiobjective optimization is focused on MultiObjective Evolutionary Algorithms (MOEAs). While MOEAs are well-suited to handle large, intractable design spaces, they typically require thousands of expensive simulations, which is prohibitively expensive for the problems under study. Therefore, the use of surrogate models in multiobjective optimization, denoted as MultiObjective Surrogate-Based Optimization (MOSBO), may prove to be even more worthwhile than SBO methods to expedite the optimization process. In this paper, the authors propose the Efficient Multiobjective Optimization (EMO) algorithm which uses Kriging models and multiobjective versions of the expected improvement and probability of improvement criterions to identify the Pareto front with a minimal number of expensive simulations. The EMO algorithm is applied on multiple standard benchmark problems and compared against the well-known NSGA-II and SPEA2 multiobjective optimization methods with promising results
Optimizing Photonic Nanostructures via Multi-fidelity Gaussian Processes
We apply numerical methods in combination with finite-difference-time-domain
(FDTD) simulations to optimize transmission properties of plasmonic mirror
color filters using a multi-objective figure of merit over a five-dimensional
parameter space by utilizing novel multi-fidelity Gaussian processes approach.
We compare these results with conventional derivative-free global search
algorithms, such as (single-fidelity) Gaussian Processes optimization scheme,
and Particle Swarm Optimization---a commonly used method in nanophotonics
community, which is implemented in Lumerical commercial photonics software. We
demonstrate the performance of various numerical optimization approaches on
several pre-collected real-world datasets and show that by properly trading off
expensive information sources with cheap simulations, one can more effectively
optimize the transmission properties with a fixed budget.Comment: NIPS 2018 Workshop on Machine Learning for Molecules and Materials.
arXiv admin note: substantial text overlap with arXiv:1811.0075
How to Solve Classification and Regression Problems on High-Dimensional Data with a Supervised Extension of Slow Feature Analysis
Supervised learning from high-dimensional data, e.g., multimedia data, is a challenging task. We propose an extension of slow feature analysis (SFA) for supervised dimensionality reduction called graph-based SFA (GSFA). The algorithm extracts a label-predictive low-dimensional set of features that can be post-processed by typical supervised algorithms to generate the final label or class estimation. GSFA is trained with a so-called training graph, in which the vertices are the samples and the edges represent similarities of the corresponding labels. A new weighted SFA optimization problem is introduced, generalizing the notion of slowness from sequences of samples to such training graphs. We show that GSFA computes an optimal solution to this problem in the considered function space, and propose several types of training graphs. For classification, the most straightforward graph yields features equivalent to those of (nonlinear) Fisher discriminant analysis. Emphasis is on regression, where four different graphs were evaluated experimentally with a subproblem of face detection on photographs. The method proposed is promising particularly when linear models are insufficient, as well as when feature selection is difficult
- …