Bayesian optimisation for automated machine learning

In this thesis, we develop a rich family of efficient and performant Bayesian optimisation (BO) methods to tackle various AutoML tasks. We first introduce a fast information-theoretic BO method, FITBO, that overcomes the computation bottleneck of information-theoretic acquisition functions while maintaining their competitiveness on the noisy optimisation problems frequently encountered in AutoML. We then improve on the idea of local penalisation and develop an asynchronous batch BO solution, PLAyBOOK, to enable more efficient use of parallel computing resources when evaluation runtime varies across configurations. In view of the fact that many practical AutoML problems involve a mixture of multiple continuous and multiple categorical variables, we propose a new framework, named Continuous and Categorical BO (CoCaBO) to handle such mixed-type input spaces. CoCaBO merges the strengths of multi-armed bandits on categorical inputs and that of BO on continuous space, and uses a tailored kernel to permit information sharing across different categorical variables. We also extend CoCaBO by harnessing the concept of local trust region to achieve competitive performance on high-dimensional optimisation problems with mixed input types. Beyond hyper-parameter tuning, we also investigate the novel use of BO on two important AutoML applications: black-box adversarial attack and neural architecture search. For the former (adversarial attack), we introduce the first BO-based attacks on image and graph classifiers; by actively querying the unknown victim classifier, our BO attacks can successfully find adversarial perturbations with many fewer attempts than competing baselines. They can thus serve as efficient tools for assessing the robustness of models suggested by AutoML. For the latter (neural architecture search), we leverage the Weisfeiler-Lehamn graph kernel to empower our BO search strategy, NAS-BOWL, to naturally handle the directed acyclic graph representation of architectures. Besides achieving superior query efficiency, our NAS-BOWL also returns interpretable sub-features that help explain the architecture performance, thus marking the first step towards interpretable neural architecture search. Finally, we examine the most computation-intense step in AutoML pipeline: generalisation performance evaluation for a new configuration. We propose a cheap yet reliable test performance estimator based on a simple measure of training speed. It consistently outperforms various existing estimators on on a wide range of architecture search spaces and and can be easily incorporated into different search strategies, including BO, to improve the cost efficiency

Study on incentive mechanisms of smes crowdsourcing contest innovation.

Lighterness, Paul - Associate SupervisorDealing with insufficient resources is a common challenge yet practical reality for many project managers working within SMEs. With the rise of Web 2.0, crowdsourcing contest innovation (CCI) it is now possible for project managers to use online platforms as a way to collaborate with external agents to fill this resource gap and thus improve innovation. This research uses agent-based modelling to prognosticate the efficacy of crowdsourcing contest innovation with a particular focus on the project manager ‘seeker’ within an SME initiating competitive crowdsourced contest teams made up of individual ‘solver’ participants. The contribution of knowledge will benefit the open innovation community to better understand the main motivational incentives to obtain maximum productivity of a team with limited project management resources. In pursuit of this, the social exchange theory is challenged, this thesis explores the motivation factors that influence solvers to participate in SMEs CCI from the perspectives of benefit perception and cost perception. The results found that non-material factors such as knowledge acquisition and sharing, reputation can stimulate solvers to participate in SMEs CCI more than material (physical money) rewards. Meanwhile, risks such as intellectual property risks and waste of resources are significant participation obstacles. Based on this, the principal- agent theory is used to design the models of team collaboration material incentive mechanism, dynamic reputation incentive mechanism and knowledge sharing incentive mechanism, and the performance of each incentive mechanism is analysed. At last, according to the principles of sample selection, Zbj.com, the China’s most successful crowdsourcing platform of which the main clients are SMEs, is chosen as the research object, and the effectiveness of the incentive mechanisms designed in this thesis is verified. It is found that the material and non-material incentives have been partially applied on the platform, and the explicit, implicit and synergistic effects of incentives are preliminarily achieved. According to the research results, it is suggested that the guarantee measures of the incentive mechanisms should be further developed, such as optimising pricing services and refining task allocation rules.PhD in Water, including Desig