Black-box Mixed-Variable Optimisation using a Surrogate Model that Satisfies Integer Constraints

A challenging problem in both engineering and computer science is that of minimising a function for which we have no mathematical formulation available, that is expensive to evaluate, and that contains continuous and integer variables, for example in automatic algorithm configuration. Surrogate-based algorithms are very suitable for this type of problem, but most existing techniques are designed with only continuous or only discrete variables in mind. Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) is a surrogate-based algorithm that uses a linear combination of rectified linear units, defined in such a way that (local) optima satisfy the integer constraints. This method outperforms the state of the art on several synthetic benchmarks with up to 238 continuous and integer variables, and achieves competitive performance on two real-life benchmarks: XGBoost hyperparameter tuning and Electrostatic Precipitator optimisation.Comment: Ann Math Artif Intell (2020

Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints

A challenging problem in both engineering and computer science is that of minimising a function for which we have no mathematical formulation available, that is expensive to evaluate, and that contains continuous and integer variables, for example in automatic algorithm configuration. Surrogate-based algorithms are very suitable for this type of problem, but most existing techniques are designed with only continuous or only discrete variables in mind. Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) is a surrogate-based algorithm that uses a linear combination of rectified linear units, defined in such a way that (local) optima satisfy the integer constraints. Unlike other methods, it also has a constant run-time per iteration. This method outperforms the state of the art on several synthetic benchmarks with up to 238 continuous and integer variables, and achieves competitive performance on two real-life benchmarks: XG-Boost hyperparameter tuning and Electrostatic Precipitator optimisation. Cyber SecurityAlgorithmic

EXPObench: Benchmarking Surrogate-based Optimisation Algorithms on Expensive Black-box Functions

Surrogate algorithms such as Bayesian optimisation are especially designed for black-box optimisation problems with expensive objectives, such as hyperparameter tuning or simulation-based optimisation. In the literature, these algorithms are usually evaluated with synthetic benchmarks which are well established but have no expensive objective, and only on one or two real-life applications which vary wildly between papers. There is a clear lack of standardisation when it comes to benchmarking surrogate algorithms on real-life, expensive, black-box objective functions. This makes it very difficult to draw conclusions on the effect of algorithmic contributions. A new benchmark library, EXPObench, provides first steps towards such a standardisation. The library is used to provide an extensive comparison of six different surrogate algorithms on four expensive optimisation problems from different real-life applications. This has led to new insights regarding the relative importance of exploration, the evaluation time of the objective, and the used model. A further contribution is that we make the algorithms and benchmark problem instances publicly available, contributing to more uniform analysis of surrogate algorithms. Most importantly, we include the performance of the six algorithms on all evaluated problem instances. This results in a unique new dataset that lowers the bar for researching new methods as the number of expensive evaluations required for comparison is significantly reduced.Comment: 13 page

Algorithms for Self-Optimising Chemical Platforms

The appreciable interest in machine learning has stimulated the development of self-optimising chemical platforms. The power of harnessing computer aided design, coupled with the desire for improved process sustainability and economics, has led to self-optimising systems being applied to the optimisation of reaction screening and chemical synthesis. The algorithms used in these systems have largely been limited to a select few, with little focus paid to the development of optimisation algorithms specifically for chemical systems. The expanding digitisation of the process development pipeline necessitates the further development of algorithms to tackle the diverse array of chemistries and systems .Improvements and expansion to the available algorithmic portfolio will enable the wider adoption of automated optimisation systems, with novel algorithms required to match the previously unmet domain specific demands and improve upon classical designed experiment procedures which may offer a reduction in optimisation efficiency. The work in this thesis looks to develop novel approaches, targeting areas currently lacking or under developed in automated chemical system optimisations. This includes development and application of hybrid approaches looking at improving the robustness of optimisation and increasing the users understanding of the optimum region, as well as expanding multi-objective algorithms to the mixed variable domain, enabling the wider application of efficient optimisation and data acquisition methodologies