155 research outputs found
Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search
Neural Architecture Search (NAS) has shown great success in automating the
design of neural networks, but the prohibitive amount of computations behind
current NAS methods requires further investigations in improving the sample
efficiency and the network evaluation cost to get better results in a shorter
time. In this paper, we present a novel scalable Monte Carlo Tree Search (MCTS)
based NAS agent, named AlphaX, to tackle these two aspects. AlphaX improves the
search efficiency by adaptively balancing the exploration and exploitation at
the state level, and by a Meta-Deep Neural Network (DNN) to predict network
accuracies for biasing the search toward a promising region. To amortize the
network evaluation cost, AlphaX accelerates MCTS rollouts with a distributed
design and reduces the number of epochs in evaluating a network by transfer
learning, which is guided with the tree structure in MCTS. In 12 GPU days and
1000 samples, AlphaX found an architecture that reaches 97.84\% top-1 accuracy
on CIFAR-10, and 75.5\% top-1 accuracy on ImageNet, exceeding SOTA NAS methods
in both the accuracy and sampling efficiency. Particularly, we also evaluate
AlphaX on NASBench-101, a large scale NAS dataset; AlphaX is 3x and 2.8x more
sample efficient than Random Search and Regularized Evolution in finding the
global optimum. Finally, we show the searched architecture improves a variety
of vision applications from Neural Style Transfer, to Image Captioning and
Object Detection.Comment: To appear in the Thirty-Fourth AAAI conference on Artificial
Intelligence (AAAI-2020
Optimizing Neural Architecture Search using Limited GPU Time in a Dynamic Search Space: A Gene Expression Programming Approach
Efficient identification of people and objects, segmentation of regions of
interest and extraction of relevant data in images, texts, audios and videos
are evolving considerably in these past years, which deep learning methods,
combined with recent improvements in computational resources, contributed
greatly for this achievement. Although its outstanding potential, development
of efficient architectures and modules requires expert knowledge and amount of
resource time available. In this paper, we propose an evolutionary-based neural
architecture search approach for efficient discovery of convolutional models in
a dynamic search space, within only 24 GPU hours. With its efficient search
environment and phenotype representation, Gene Expression Programming is
adapted for network's cell generation. Despite having limited GPU resource time
and broad search space, our proposal achieved similar state-of-the-art to
manually-designed convolutional networks and also NAS-generated ones, even
beating similar constrained evolutionary-based NAS works. The best cells in
different runs achieved stable results, with a mean error of 2.82% in CIFAR-10
dataset (which the best model achieved an error of 2.67%) and 18.83% for
CIFAR-100 (best model with 18.16%). For ImageNet in the mobile setting, our
best model achieved top-1 and top-5 errors of 29.51% and 10.37%, respectively.
Although evolutionary-based NAS works were reported to require a considerable
amount of GPU time for architecture search, our approach obtained promising
results in little time, encouraging further experiments in evolutionary-based
NAS, for search and network representation improvements.Comment: Accepted for presentation at the IEEE Congress on Evolutionary
Computation (IEEE CEC) 202
- …