17,952 research outputs found
Progressive Neural Architecture Search
We propose a new method for learning the structure of convolutional neural
networks (CNNs) that is more efficient than recent state-of-the-art methods
based on reinforcement learning and evolutionary algorithms. Our approach uses
a sequential model-based optimization (SMBO) strategy, in which we search for
structures in order of increasing complexity, while simultaneously learning a
surrogate model to guide the search through structure space. Direct comparison
under the same search space shows that our method is up to 5 times more
efficient than the RL method of Zoph et al. (2018) in terms of number of models
evaluated, and 8 times faster in terms of total compute. The structures we
discover in this way achieve state of the art classification accuracies on
CIFAR-10 and ImageNet.Comment: To appear in ECCV 2018 as oral. The code and checkpoint for PNASNet-5
trained on ImageNet (both Mobile and Large) can now be downloaded from
https://github.com/tensorflow/models/tree/master/research/slim#Pretrained.
Also see https://github.com/chenxi116/PNASNet.TF for refactored and
simplified TensorFlow code; see https://github.com/chenxi116/PNASNet.pytorch
for exact conversion to PyTorc
Federated Neural Architecture Search
To preserve user privacy while enabling mobile intelligence, techniques have
been proposed to train deep neural networks on decentralized data. However,
training over decentralized data makes the design of neural architecture quite
difficult as it already was. Such difficulty is further amplified when
designing and deploying different neural architectures for heterogeneous mobile
platforms. In this work, we propose an automatic neural architecture search
into the decentralized training, as a new DNN training paradigm called
Federated Neural Architecture Search, namely federated NAS. To deal with the
primary challenge of limited on-client computational and communication
resources, we present FedNAS, a highly optimized framework for efficient
federated NAS. FedNAS fully exploits the key opportunity of insufficient model
candidate re-training during the architecture search process, and incorporates
three key optimizations: parallel candidates training on partial clients, early
dropping candidates with inferior performance, and dynamic round numbers.
Tested on large-scale datasets and typical CNN architectures, FedNAS achieves
comparable model accuracy as state-of-the-art NAS algorithm that trains models
with centralized data, and also reduces the client cost by up to two orders of
magnitude compared to a straightforward design of federated NAS
- …