34 research outputs found
PyPop7: A Pure-Python Library for Population-Based Black-Box Optimization
In this paper, we present a pure-Python open-source library, called PyPop7,
for black-box optimization (BBO). It provides a unified and modular interface
for more than 60 versions and variants of different black-box optimization
algorithms, particularly population-based optimizers, which can be classified
into 12 popular families: Evolution Strategies (ES), Natural Evolution
Strategies (NES), Estimation of Distribution Algorithms (EDA), Cross-Entropy
Method (CEM), Differential Evolution (DE), Particle Swarm Optimizer (PSO),
Cooperative Coevolution (CC), Simulated Annealing (SA), Genetic Algorithms
(GA), Evolutionary Programming (EP), Pattern Search (PS), and Random Search
(RS). It also provides many examples, interesting tutorials, and full-fledged
API documentations. Through this new library, we expect to provide a
well-designed platform for benchmarking of optimizers and promote their
real-world applications, especially for large-scale BBO. Its source code and
documentations are available at
https://github.com/Evolutionary-Intelligence/pypop and
https://pypop.readthedocs.io/en/latest, respectively.Comment: 5 page
Limited Evaluation Cooperative Co-evolutionary Differential Evolution for Large-scale Neuroevolution
Many real-world control and classification tasks involve a large number of
features. When artificial neural networks (ANNs) are used for modeling these
tasks, the network architectures tend to be large. Neuroevolution is an
effective approach for optimizing ANNs; however, there are two bottlenecks that
make their application challenging in case of high-dimensional networks using
direct encoding. First, classic evolutionary algorithms tend not to scale well
for searching large parameter spaces; second, the network evaluation over a
large number of training instances is in general time-consuming. In this work,
we propose an approach called the Limited Evaluation Cooperative
Co-evolutionary Differential Evolution algorithm (LECCDE) to optimize
high-dimensional ANNs.
The proposed method aims to optimize the pre-synaptic weights of each
post-synaptic neuron in different subpopulations using a Cooperative
Co-evolutionary Differential Evolution algorithm, and employs a limited
evaluation scheme where fitness evaluation is performed on a relatively small
number of training instances based on fitness inheritance. We test LECCDE on
three datasets with various sizes, and our results show that cooperative
co-evolution significantly improves the test error comparing to standard
Differential Evolution, while the limited evaluation scheme facilitates a
significant reduction in computing time
Coevolution of Generative Adversarial Networks
Generative adversarial networks (GAN) became a hot topic, presenting
impressive results in the field of computer vision. However, there are still
open problems with the GAN model, such as the training stability and the
hand-design of architectures. Neuroevolution is a technique that can be used to
provide the automatic design of network architectures even in large search
spaces as in deep neural networks. Therefore, this project proposes COEGAN, a
model that combines neuroevolution and coevolution in the coordination of the
GAN training algorithm. The proposal uses the adversarial characteristic
between the generator and discriminator components to design an algorithm using
coevolution techniques. Our proposal was evaluated in the MNIST dataset. The
results suggest the improvement of the training stability and the automatic
discovery of efficient network architectures for GANs. Our model also partially
solves the mode collapse problem.Comment: Published in EvoApplications 201
Towards the Evolution of Multi-Layered Neural Networks: A Dynamic Structured Grammatical Evolution Approach
Current grammar-based NeuroEvolution approaches have several shortcomings. On
the one hand, they do not allow the generation of Artificial Neural Networks
(ANNs composed of more than one hidden-layer. On the other, there is no way to
evolve networks with more than one output neuron. To properly evolve ANNs with
more than one hidden-layer and multiple output nodes there is the need to know
the number of neurons available in previous layers. In this paper we introduce
Dynamic Structured Grammatical Evolution (DSGE): a new genotypic representation
that overcomes the aforementioned limitations. By enabling the creation of
dynamic rules that specify the connection possibilities of each neuron, the
methodology enables the evolution of multi-layered ANNs with more than one
output neuron. Results in different classification problems show that DSGE
evolves effective single and multi-layered ANNs, with a varying number of
output neurons
Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction
Cooperative coevolution decomposes a problem into subcomponents and employs evolutionary algorithms for solving them. Cooperative coevolution has been effective for evolving neural networks. Different problem decomposition methods in cooperative coevolution determine how a neural network is decomposed and encoded which affects its performance. A good problem decomposition method should provide enough diversity and also group interacting variables which are the synapses in the neural network. Neural networks have shown promising results in chaotic time series prediction. This work employs two problem decomposition methods for training Elman recurrent neural networks on chaotic time series problems. The Mackey-Glass, Lorenz and Sunspot time series are used to demonstrate the performance of the cooperative neuro-evolutionary methods. The results show improvement in performance in terms of accuracy when compared to some of the methods from literature
Safe Crossover of Neural Networks Through Neuron Alignment
One of the main and largely unexplored challenges in evolving the weights of
neural networks using genetic algorithms is to find a sensible crossover
operation between parent networks. Indeed, naive crossover leads to
functionally damaged offspring that do not retain information from the parents.
This is because neural networks are invariant to permutations of neurons,
giving rise to multiple ways of representing the same solution. This is often
referred to as the competing conventions problem. In this paper, we propose a
two-step safe crossover(SC) operator. First, the neurons of the parents are
functionally aligned by computing how well they correlate, and only then are
the parents recombined. We compare two ways of measuring relationships between
neurons: Pairwise Correlation (PwC) and Canonical Correlation Analysis (CCA).
We test our safe crossover operators (SC-PwC and SC-CCA) on MNIST and CIFAR-10
by performing arithmetic crossover on the weights of feed-forward neural
network pairs. We show that it effectively transmits information from parents
to offspring and significantly improves upon naive crossover. Our method is
computationally fast,can serve as a way to explore the fitness landscape more
efficiently and makes safe crossover a potentially promising operator in future
neuroevolution research and applications