16 research outputs found
Bayesian Optimization with Unknown Constraints
Recent work on Bayesian optimization has shown its effectiveness in global
optimization of difficult black-box objective functions. Many real-world
optimization problems of interest also have constraints which are unknown a
priori. In this paper, we study Bayesian optimization for constrained problems
in the general case that noise may be present in the constraint functions, and
the objective and constraints may be evaluated independently. We provide
motivating practical examples, and present a general framework to solve such
problems. We demonstrate the effectiveness of our approach on optimizing the
performance of online latent Dirichlet allocation subject to topic sparsity
constraints, tuning a neural network given test-time memory constraints, and
optimizing Hamiltonian Monte Carlo to achieve maximal effectiveness in a fixed
time, subject to passing standard convergence diagnostics.Comment: 14 pages, 3 figure
Theoretical Analysis of Bayesian Optimisation with Unknown Gaussian Process Hyper-Parameters
Bayesian optimisation has gained great popularity as a tool for optimising
the parameters of machine learning algorithms and models. Somewhat ironically,
setting up the hyper-parameters of Bayesian optimisation methods is notoriously
hard. While reasonable practical solutions have been advanced, they can often
fail to find the best optima. Surprisingly, there is little theoretical
analysis of this crucial problem in the literature. To address this, we derive
a cumulative regret bound for Bayesian optimisation with Gaussian processes and
unknown kernel hyper-parameters in the stochastic setting. The bound, which
applies to the expected improvement acquisition function and sub-Gaussian
observation noise, provides us with guidelines on how to design hyper-parameter
estimation methods. A simple simulation demonstrates the importance of
following these guidelines.Comment: 16 pages, 1 figur
A new hybrid meta-heuristic algorithm for solving single machine scheduling problems
A dissertation submitted in partial ful lment of the
degree of Master of Science in Engineering (Electrical) (50/50)
in the
Faculty of Engineering and the Built Environment
Department of Electrical and Information Engineering
May 2017Numerous applications in a wide variety of elds has resulted in a rich history of research
into optimisation for scheduling. Although it is a fundamental form of the problem, the
single machine scheduling problem with two or more objectives is known to be NP-hard.
For this reason we consider the single machine problem a good test bed for solution
algorithms. While there is a plethora of research into various aspects of scheduling
problems, little has been done in evaluating the performance of the Simulated Annealing
algorithm for the fundamental problem, or using it in combination with other techniques.
Speci cally, this has not been done for minimising total weighted earliness and tardiness,
which is the optimisation objective of this work.
If we consider a mere ten jobs for scheduling, this results in over 3.6 million possible
solution schedules. It is thus of de nite practical necessity to reduce the search space in
order to nd an optimal or acceptable suboptimal solution in a shorter time, especially
when scaling up the problem size. This is of particular importance in the application
area of packet scheduling in wireless communications networks where the tolerance for
computational delays is very low. The main contribution of this work is to investigate
the hypothesis that inserting a step of pre-sampling by Markov Chain Monte Carlo
methods before running the Simulated Annealing algorithm on the pruned search space
can result in overall reduced running times.
The search space is divided into a number of sections and Metropolis-Hastings Markov
Chain Monte Carlo is performed over the sections in order to reduce the search space for
Simulated Annealing by a factor of 20 to 100. Trade-o s are found between the run time
and number of sections of the pre-sampling algorithm, and the run time of Simulated
Annealing for minimising the percentage deviation of the nal result from the optimal
solution cost. Algorithm performance is determined both by computational complexity
and the quality of the solution (i.e. the percentage deviation from the optimal). We
nd that the running time can be reduced by a factor of 4.5 to ensure a 2% deviation
from the optimal, as compared to the basic Simulated Annealing algorithm on the full
search space. More importantly, we are able to reduce the complexity of nding the
optimal from O(n:n!) for a complete search to O(nNS) for Simulated Annealing to
O(n(NMr +NS)+m) for the input variables n jobs, NS SA iterations, NM Metropolis-
Hastings iterations, r inner samples and m sections.MT 201