57,212 research outputs found
A Hybrid Differential Evolution Approach to Designing Deep Convolutional Neural Networks for Image Classification
Convolutional Neural Networks (CNNs) have demonstrated their superiority in
image classification, and evolutionary computation (EC) methods have recently
been surging to automatically design the architectures of CNNs to save the
tedious work of manually designing CNNs. In this paper, a new hybrid
differential evolution (DE) algorithm with a newly added crossover operator is
proposed to evolve the architectures of CNNs of any lengths, which is named
DECNN. There are three new ideas in the proposed DECNN method. Firstly, an
existing effective encoding scheme is refined to cater for variable-length CNN
architectures; Secondly, the new mutation and crossover operators are developed
for variable-length DE to optimise the hyperparameters of CNNs; Finally, the
new second crossover is introduced to evolve the depth of the CNN
architectures. The proposed algorithm is tested on six widely-used benchmark
datasets and the results are compared to 12 state-of-the-art methods, which
shows the proposed method is vigorously competitive to the state-of-the-art
algorithms. Furthermore, the proposed method is also compared with a method
using particle swarm optimisation with a similar encoding strategy named IPPSO,
and the proposed DECNN outperforms IPPSO in terms of the accuracy.Comment: Accepted by The Australasian Joint Conference on Artificial
Intelligence 201
Safe Mutations for Deep and Recurrent Neural Networks through Output Gradients
While neuroevolution (evolving neural networks) has a successful track record
across a variety of domains from reinforcement learning to artificial life, it
is rarely applied to large, deep neural networks. A central reason is that
while random mutation generally works in low dimensions, a random perturbation
of thousands or millions of weights is likely to break existing functionality,
providing no learning signal even if some individual weight changes were
beneficial. This paper proposes a solution by introducing a family of safe
mutation (SM) operators that aim within the mutation operator itself to find a
degree of change that does not alter network behavior too much, but still
facilitates exploration. Importantly, these SM operators do not require any
additional interactions with the environment. The most effective SM variant
capitalizes on the intriguing opportunity to scale the degree of mutation of
each individual weight according to the sensitivity of the network's outputs to
that weight, which requires computing the gradient of outputs with respect to
the weights (instead of the gradient of error, as in conventional deep
learning). This safe mutation through gradients (SM-G) operator dramatically
increases the ability of a simple genetic algorithm-based neuroevolution method
to find solutions in high-dimensional domains that require deep and/or
recurrent neural networks (which tend to be particularly brittle to mutation),
including domains that require processing raw pixels. By improving our ability
to evolve deep neural networks, this new safer approach to mutation expands the
scope of domains amenable to neuroevolution
Tile Pattern KL-Divergence for Analysing and Evolving Game Levels
This paper provides a detailed investigation of using the Kullback-Leibler
(KL) Divergence as a way to compare and analyse game-levels, and hence to use
the measure as the objective function of an evolutionary algorithm to evolve
new levels. We describe the benefits of its asymmetry for level analysis and
demonstrate how (not surprisingly) the quality of the results depends on the
features used. Here we use tile-patterns of various sizes as features.
When using the measure for evolution-based level generation, we demonstrate
that the choice of variation operator is critical in order to provide an
efficient search process, and introduce a novel convolutional mutation operator
to facilitate this. We compare the results with alternative generators,
including evolving in the latent space of generative adversarial networks, and
Wave Function Collapse. The results clearly show the proposed method to provide
competitive performance, providing reasonable quality results with very fast
training and reasonably fast generation.Comment: 8 pages plus references. Proceedings of GECCO 201
A generic optimising feature extraction method using multiobjective genetic programming
In this paper, we present a generic, optimising feature extraction method using multiobjective genetic programming. We re-examine the feature extraction problem and show that effective feature extraction can significantly enhance the performance of pattern recognition systems with simple classifiers. A framework is presented to evolve optimised feature extractors that transform an input pattern space into a decision space in which maximal class separability is obtained. We have applied this method to real world datasets from the UCI Machine Learning and StatLog databases to verify our approach and compare our proposed method with other reported results. We conclude that our algorithm is able to produce classifiers of superior (or equivalent) performance to the conventional classifiers examined, suggesting removal of the need to exhaustively evaluate a large family of conventional classifiers on any new problem. (C) 2010 Elsevier B.V. All rights reserved
- …