851,426 research outputs found
GLOBAL OPTIMIZATION METHODS
Training a neural network is a difficult optimization problem because of numerous local minimums. Many global search algorithms have been used to train neural networks. However, local search algorithms are more efficient with computational resources, and therefore numerous random restarts with a local algorithm may be more effective than a global algorithm. This study uses Monte-Carlo simulations to determine the relative efficiency of a local search algorithm to 9 stochastic global algorithms. The computational requirements of the global algorithms are several times higher than the local algorithm and there is little gain in using the global algorithms to train neural networks.Research Methods/ Statistical Methods,
Active Bayesian Optimization: Minimizing Minimizer Entropy
The ultimate goal of optimization is to find the minimizer of a target
function.However, typical criteria for active optimization often ignore the
uncertainty about the minimizer. We propose a novel criterion for global
optimization and an associated sequential active learning strategy using
Gaussian processes.Our criterion is the reduction of uncertainty in the
posterior distribution of the function minimizer. It can also flexibly
incorporate multiple global minimizers. We implement a tractable approximation
of the criterion and demonstrate that it obtains the global minimizer
accurately compared to conventional Bayesian optimization criteria
- …