167 research outputs found
Multi-criteria Evolution of Neural Network Topologies: Balancing Experience and Performance in Autonomous Systems
Majority of Artificial Neural Network (ANN) implementations in autonomous
systems use a fixed/user-prescribed network topology, leading to sub-optimal
performance and low portability. The existing neuro-evolution of augmenting
topology or NEAT paradigm offers a powerful alternative by allowing the network
topology and the connection weights to be simultaneously optimized through an
evolutionary process. However, most NEAT implementations allow the
consideration of only a single objective. There also persists the question of
how to tractably introduce topological diversification that mitigates
overfitting to training scenarios. To address these gaps, this paper develops a
multi-objective neuro-evolution algorithm. While adopting the basic elements of
NEAT, important modifications are made to the selection, speciation, and
mutation processes. With the backdrop of small-robot path-planning
applications, an experience-gain criterion is derived to encapsulate the amount
of diverse local environment encountered by the system. This criterion
facilitates the evolution of genes that support exploration, thereby seeking to
generalize from a smaller set of mission scenarios than possible with
performance maximization alone. The effectiveness of the single-objective
(optimizing performance) and the multi-objective (optimizing performance and
experience-gain) neuro-evolution approaches are evaluated on two different
small-robot cases, with ANNs obtained by the multi-objective optimization
observed to provide superior performance in unseen scenarios
Evolving developmental, recurrent and convolutional neural networks for deliberate motion planning in sparse reward tasks
Motion planning algorithms have seen a diverse set of approaches in a variety of disciplines. In the domain of artificial evolutionary systems, motion planning has been included in models to achieve sophisticated deliberate behaviours. These algorithms rely on fixed rules or little evolutionary influence which compels behaviours to conform within those specific policies, rather than allowing the model to establish its own specialised behaviour. In order to further these models, the constraints imposed by planning algorithms must be removed to grant greater evolutionary control over behaviours. That is the focus of this thesis.
An examination of prevailing neuroevolution methods led to the use of two distinct approaches, NEAT and HyperNEAT. Both were used to gain an understanding of the components necessary to create neuroevolution planning. The findings accumulated in the formation of a novel convolutional neural network architecture with a recurrent convolution process. The architecture’s goal was to iteratively disperse local activations to greater regions of the feature space. Experimentation showed significantly improved robustness over contemporary neuroevolution techniques as well as an efficiency increase over a static rule set. Greater evolutionary responsibility is given to the model with multiple network combinations; all of which continually demonstrated the necessary behaviours. In comparison, these behaviours were shown to be difficult to achieve in a state-of-the-art deep convolutional network.
Finally, the unique use of recurrent convolution is relocated to a larger convolutional architecture on an established benchmarking platform. Performance improvements are seen on a number of domains which illustrates that this recurrent mechanism can be exploited in alternative areas outside of planning. By presenting a viable neuroevolution method for motion planning a potential emerges for further systems to adopt and examine the capability of this work in prospective domains, as well as further avenues of experimentation in convolutional architectures
Evolutionary design of deep neural networks
Mención Internacional en el título de doctorFor three decades, neuroevolution has applied evolutionary computation to the optimization of
the topology of artificial neural networks, with most works focusing on very simple architectures.
However, times have changed, and nowadays convolutional neural networks are the industry and
academia standard for solving a variety of problems, many of which remained unsolved before the
discovery of this kind of networks.
Convolutional neural networks involve complex topologies, and the manual design of these
topologies for solving a problem at hand is expensive and inefficient. In this thesis, our aim is to
use neuroevolution in order to evolve the architecture of convolutional neural networks.
To do so, we have decided to try two different techniques: genetic algorithms and grammatical
evolution. We have implemented a niching scheme for preserving the genetic diversity, in order
to ease the construction of ensembles of neural networks. These techniques have been validated
against the MNIST database for handwritten digit recognition, achieving a test error rate of 0.28%,
and the OPPORTUNITY data set for human activity recognition, attaining an F1 score of 0.9275.
Both results have proven very competitive when compared with the state of the art. Also, in all
cases, ensembles have proven to perform better than individual models.
Later, the topologies learned for MNIST were tested on EMNIST, a database recently introduced
in 2017, which includes more samples and a set of letters for character recognition. Results have
shown that the topologies optimized for MNIST perform well on EMNIST, proving that architectures
can be reused across domains with similar characteristics.
In summary, neuroevolution is an effective approach for automatically designing topologies for
convolutional neural networks. However, it still remains as an unexplored field due to hardware
limitations. Current advances, however, should constitute the fuel that empowers the emergence of
this field, and further research should start as of today.This Ph.D. dissertation has been partially supported by the Spanish Ministry of Education, Culture and Sports under FPU fellowship with identifier FPU13/03917.
This research stay has been partially co-funded by the Spanish Ministry of Education, Culture and Sports under FPU short stay grant with identifier EST15/00260.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: María Araceli Sanchís de Miguel.- Secretario: Francisco Javier Segovia Pérez.- Vocal: Simon Luca
- …