795 research outputs found
Safe Crossover of Neural Networks Through Neuron Alignment
One of the main and largely unexplored challenges in evolving the weights of
neural networks using genetic algorithms is to find a sensible crossover
operation between parent networks. Indeed, naive crossover leads to
functionally damaged offspring that do not retain information from the parents.
This is because neural networks are invariant to permutations of neurons,
giving rise to multiple ways of representing the same solution. This is often
referred to as the competing conventions problem. In this paper, we propose a
two-step safe crossover(SC) operator. First, the neurons of the parents are
functionally aligned by computing how well they correlate, and only then are
the parents recombined. We compare two ways of measuring relationships between
neurons: Pairwise Correlation (PwC) and Canonical Correlation Analysis (CCA).
We test our safe crossover operators (SC-PwC and SC-CCA) on MNIST and CIFAR-10
by performing arithmetic crossover on the weights of feed-forward neural
network pairs. We show that it effectively transmits information from parents
to offspring and significantly improves upon naive crossover. Our method is
computationally fast,can serve as a way to explore the fitness landscape more
efficiently and makes safe crossover a potentially promising operator in future
neuroevolution research and applications
Recommended from our members
Neural Diagrammatic Reasoning
Diagrams have been shown to be effective tools for humans to represent and reason about
complex concepts. They have been widely used to represent concepts in science teaching, to
communicate workflow in industries and to measure human fluid intelligence. Mechanised
reasoning systems typically encode diagrams into symbolic representations that can be
easily processed with rule-based expert systems. This relies on human experts to define the
framework of diagram-to-symbol mapping and the set of rules to reason with the symbols.
This means the reasoning systems cannot be easily adapted to other diagrams without
a new set of human-defined representation mapping and reasoning rules. Moreover such
systems are not able to cope with diagram inputs as raw and possibly noisy images. The
need for human input and the lack of robustness to noise significantly limit the applications
of mechanised diagrammatic reasoning systems.
A key research question then arises: can we develop human-like reasoning systems that
learn to reason robustly without predefined reasoning rules? To answer this question, I
propose Neural Diagrammatic Reasoning, a new family of diagrammatic reasoning
systems which does not have the drawbacks of mechanised reasoning systems. The new
systems are based on deep neural networks, a recently popular machine learning method
that achieved human-level performance on a range of perception tasks such as object
detection, speech recognition and natural language processing. The proposed systems are
able to learn both diagram to symbol mapping and implicit reasoning rules only from data,
with no prior human input about symbols and rules in the reasoning tasks. Specifically I
developed EulerNet, a novel neural network model that solves Euler diagram syllogism
tasks with 99.5% accuracy. Experiments show that EulerNet learns useful representations
of the diagrams and tasks, and is robust to noise and deformation in the input data. I
also developed MXGNet, a novel multiplex graph neural architecture that solves Raven
Progressive Matrices (RPM) tasks. MXGNet achieves state-of-the-art accuracies on two
popular RPM datasets. In addition, I developed Discrete-AIR, an unsupervised learning
architecture that learns semi-symbolic representations of diagrams without any labels.
Lastly I designed a novel inductive bias module that can be readily used in today’s deep
neural networks to improve their generalisation capability on relational reasoning tasks.EPSRC Studentship and Cambridge Trust Scholarshi
- …