20,144 research outputs found
SCANN: Synthesis of Compact and Accurate Neural Networks
Deep neural networks (DNNs) have become the driving force behind recent
artificial intelligence (AI) research. An important problem with implementing a
neural network is the design of its architecture. Typically, such an
architecture is obtained manually by exploring its hyperparameter space and
kept fixed during training. This approach is time-consuming and inefficient.
Another issue is that modern neural networks often contain millions of
parameters, whereas many applications and devices require small inference
models. However, efforts to migrate DNNs to such devices typically entail a
significant loss of classification accuracy. To address these challenges, we
propose a two-step neural network synthesis methodology, called DR+SCANN, that
combines two complementary approaches to design compact and accurate DNNs. At
the core of our framework is the SCANN methodology that uses three basic
architecture-changing operations, namely connection growth, neuron growth, and
connection pruning, to synthesize feed-forward architectures with arbitrary
structure. SCANN encapsulates three synthesis methodologies that apply a
repeated grow-and-prune paradigm to three architectural starting points.
DR+SCANN combines the SCANN methodology with dataset dimensionality reduction
to alleviate the curse of dimensionality. We demonstrate the efficacy of SCANN
and DR+SCANN on various image and non-image datasets. We evaluate SCANN on
MNIST and ImageNet benchmarks. In addition, we also evaluate the efficacy of
using dimensionality reduction alongside SCANN (DR+SCANN) on nine small to
medium-size datasets. We also show that our synthesis methodology yields neural
networks that are much better at navigating the accuracy vs. energy efficiency
space. This would enable neural network-based inference even on
Internet-of-Things sensors.Comment: 13 pages, 8 figure
Evolving neural networks with genetic algorithms to study the String Landscape
We study possible applications of artificial neural networks to examine the
string landscape. Since the field of application is rather versatile, we
propose to dynamically evolve these networks via genetic algorithms. This means
that we start from basic building blocks and combine them such that the neural
network performs best for the application we are interested in. We study three
areas in which neural networks can be applied: to classify models according to
a fixed set of (physically) appealing features, to find a concrete realization
for a computation for which the precise algorithm is known in principle but
very tedious to actually implement, and to predict or approximate the outcome
of some involved mathematical computation which performs too inefficient to
apply it, e.g. in model scans within the string landscape. We present simple
examples that arise in string phenomenology for all three types of problems and
discuss how they can be addressed by evolving neural networks from genetic
algorithms.Comment: 17 pages, 7 figures, references added, typos corrected, extended
introductory sectio
Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing
This work presents a new algorithm called evolutionary exploration of
augmenting convolutional topologies (EXACT), which is capable of evolving the
structure of convolutional neural networks (CNNs). EXACT is in part modeled
after the neuroevolution of augmenting topologies (NEAT) algorithm, with
notable exceptions to allow it to scale to large scale distributed computing
environments and evolve networks with convolutional filters. In addition to
multithreaded and MPI versions, EXACT has been implemented as part of a BOINC
volunteer computing project, allowing large scale evolution. During a period of
two months, over 4,500 volunteered computers on the Citizen Science Grid
trained over 120,000 CNNs and evolved networks reaching 98.32% test data
accuracy on the MNIST handwritten digits dataset. These results are even
stronger as the backpropagation strategy used to train the CNNs was fairly
rudimentary (ReLU units, L2 regularization and Nesterov momentum) and these
were initial test runs done without refinement of the backpropagation
hyperparameters. Further, the EXACT evolutionary strategy is independent of the
method used to train the CNNs, so they could be further improved by advanced
techniques like elastic distortions, pretraining and dropout. The evolved
networks are also quite interesting, showing "organic" structures and
significant differences from standard human designed architectures.Comment: 17 pages, 13 figures. Submitted to the 2017 Genetic and Evolutionary
Computation Conference (GECCO 2017
Multilayered feed forward Artificial Neural Network model to predict the average summer-monsoon rainfall in India
In the present research, possibility of predicting average summer-monsoon
rainfall over India has been analyzed through Artificial Neural Network models.
In formulating the Artificial Neural Network based predictive model, three
layered networks have been constructed with sigmoid non-linearity. The models
under study are different in the number of hidden neurons. After a thorough
training and test procedure, neural net with three nodes in the hidden layer is
found to be the best predictive model.Comment: 19 pages, 1 table, 3 figure
Neural Networks for Modeling and Control of Particle Accelerators
We describe some of the challenges of particle accelerator control, highlight
recent advances in neural network techniques, discuss some promising avenues
for incorporating neural networks into particle accelerator control systems,
and describe a neural network-based control system that is being developed for
resonance control of an RF electron gun at the Fermilab Accelerator Science and
Technology (FAST) facility, including initial experimental results from a
benchmark controller.Comment: 21 p
Limited Evaluation Cooperative Co-evolutionary Differential Evolution for Large-scale Neuroevolution
Many real-world control and classification tasks involve a large number of
features. When artificial neural networks (ANNs) are used for modeling these
tasks, the network architectures tend to be large. Neuroevolution is an
effective approach for optimizing ANNs; however, there are two bottlenecks that
make their application challenging in case of high-dimensional networks using
direct encoding. First, classic evolutionary algorithms tend not to scale well
for searching large parameter spaces; second, the network evaluation over a
large number of training instances is in general time-consuming. In this work,
we propose an approach called the Limited Evaluation Cooperative
Co-evolutionary Differential Evolution algorithm (LECCDE) to optimize
high-dimensional ANNs.
The proposed method aims to optimize the pre-synaptic weights of each
post-synaptic neuron in different subpopulations using a Cooperative
Co-evolutionary Differential Evolution algorithm, and employs a limited
evaluation scheme where fitness evaluation is performed on a relatively small
number of training instances based on fitness inheritance. We test LECCDE on
three datasets with various sizes, and our results show that cooperative
co-evolution significantly improves the test error comparing to standard
Differential Evolution, while the limited evaluation scheme facilitates a
significant reduction in computing time
- ā¦