8 research outputs found
Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels
Current neural architecture search (NAS) strategies focus only on finding a
single, good, architecture. They offer little insight into why a specific
network is performing well, or how we should modify the architecture if we want
further improvements. We propose a Bayesian optimisation (BO) approach for NAS
that combines the Weisfeiler-Lehman graph kernel with a Gaussian process
surrogate. Our method optimises the architecture in a highly data-efficient
manner: it is capable of capturing the topological structures of the
architectures and is scalable to large graphs, thus making the high-dimensional
and graph-like search spaces amenable to BO. More importantly, our method
affords interpretability by discovering useful network features and their
corresponding impact on the network performance. Indeed, we demonstrate
empirically that our surrogate model is capable of identifying useful motifs
which can guide the generation of new architectures. We finally show that our
method outperforms existing NAS approaches to achieve the state of the art on
both closed- and open-domain search spaces.Comment: ICLR 2021. 9 pages, 5 figures, 1 table (23 pages, 14 figures and 3
tables including references and appendices
Natural Language Processing: Emerging Neural Approaches and Applications
This Special Issue highlights the most recent research being carried out in the NLP field to discuss relative open issues, with a particular focus on both emerging approaches for language learning, understanding, production, and grounding interactively or autonomously from data in cognitive and neural systems, as well as on their potential or real applications in different domains