Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels

Current neural architecture search (NAS) strategies focus only on finding a single, good, architecture. They offer little insight into why a specific network is performing well, or how we should modify the architecture if we want further improvements. We propose a Bayesian optimisation (BO) approach for NAS that combines the Weisfeiler-Lehman graph kernel with a Gaussian process surrogate. Our method optimises the architecture in a highly data-efficient manner: it is capable of capturing the topological structures of the architectures and is scalable to large graphs, thus making the high-dimensional and graph-like search spaces amenable to BO. More importantly, our method affords interpretability by discovering useful network features and their corresponding impact on the network performance. Indeed, we demonstrate empirically that our surrogate model is capable of identifying useful motifs which can guide the generation of new architectures. We finally show that our method outperforms existing NAS approaches to achieve the state of the art on both closed- and open-domain search spaces.Comment: ICLR 2021. 9 pages, 5 figures, 1 table (23 pages, 14 figures and 3 tables including references and appendices

Multi-Fidelity Bayesian Optimization for Efficient Materials Design

Materials design is a process of identifying compositions and structures to achieve desirable properties. Usually, costly experiments or simulations are required to evaluate the objective function for a design solution. Therefore, one of the major challenges is how to reduce the cost associated with sampling and evaluating the objective. Bayesian optimization is a new global optimization method which can increase the sampling efficiency with the guidance of the surrogate of the objective. In this work, a new acquisition function, called consequential improvement, is proposed for simultaneous selection of the solution and fidelity level of sampling. With the new acquisition function, the subsequent iteration is considered for potential selections at low-fidelity levels, because evaluations at the highest fidelity level are usually required to provide reliable objective values. To reduce the number of samples required to train the surrogate for molecular design, a new recursive hierarchical similarity metric is proposed. The new similarity metric quantifies the differences between molecules at multiple levels of hierarchy simultaneously based on the connections between multiscale descriptions of the structures. The new methodologies are demonstrated with simulation-based design of materials and structures based on fully atomistic and coarse-grained molecular dynamics simulations, and finite-element analysis. The new similarity metric is demonstrated in the design of tactile sensors and biodegradable oligomers. The multi-fidelity Bayesian optimization method is also illustrated with the multiscale design of a piezoelectric transducer by concurrently optimizing the atomic composition of the aluminum titanium nitride ceramic and the device’s porous microstructure at the micrometer scale.Ph.D