Evolving Artificial Neural Networks using Cartesian Genetic Programming

NeuroEvolution is the application of Evolutionary Algorithms to the training of Artificial Neural Networks. NeuroEvolution is thought to possess many benefits over traditional training methods including: the ability to train recurrent network structures, the capability to adapt network topology, being able to create heterogeneous networks of arbitrary transfer functions, and allowing application to reinforcement as well as supervised learning tasks. This thesis presents a series of rigorous empirical investigations into many of these perceived advantages of NeuroEvolution. In this work it is demonstrated that the ability to simultaneously adapt network topology along with connection weights represents a significant advantage of many NeuroEvolutionary methods. It is also demonstrated that the ability to create heterogeneous networks comprising a range of transfer functions represents a further significant advantage. This thesis also investigates many potential benefits and drawbacks of NeuroEvolution which have been largely overlooked in the literature. This includes the presence and role of genetic redundancy in NeuroEvolution's search and whether program bloat is a limitation. The investigations presented focus on the use of a recently developed NeuroEvolution method based on Cartesian Genetic Programming. This thesis extends Cartesian Genetic Programming such that it can represent recurrent program structures allowing for the creation of recurrent Artificial Neural Networks. Using this newly developed extension, Recurrent Cartesian Genetic Programming, and its application to Artificial Neural Networks, are demonstrated to be extremely competitive in the domain of series forecasting

Evolving Graphs by Graph Programming

Graphs are a ubiquitous data structure in computer science and can be used to represent solutions to difficult problems in many distinct domains. This motivates the use of Evolutionary Algorithms to search over graphs and efficiently find approximate solutions. However, existing techniques often represent and manipulate graphs in an ad-hoc manner. In contrast, rule-based graph programming offers a formal mechanism for describing relations over graphs. This thesis proposes the use of rule-based graph programming for representing and implementing genetic operators over graphs. We present the Evolutionary Algorithm Evolving Graphs by Graph Programming and a number of its extensions which are capable of learning stateful and stateless digital circuits, symbolic expressions and Artificial Neural Networks. We demonstrate that rule-based graph programming may be used to implement new and effective constraint-respecting mutation operators and show that these operators may strictly generalise others found in the literature. Through our proposal of Semantic Neutral Drift, we accelerate the search process by building plateaus into the fitness landscape using domain knowledge of equivalence. We also present Horizontal Gene Transfer, a mechanism whereby graphs may be passively recombined without disrupting their fitness. Through rigorous evaluation and analysis of over 20,000 independent executions of Evolutionary Algorithms, we establish numerous benefits of our approach. We find that on many problems, Evolving Graphs by Graph Programming and its variants may significantly outperform other approaches from the literature. Additionally, our empirical results provide further evidence that neutral drift aids the efficiency of evolutionary search

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN). These methods play a crucial role in the success or failure of the DNN for most problems and applications. Evolutionary Algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimisation and training of DNNs. Neuroevolution is a term which describes these processes of automated configuration and training of DNNs using EAs. While many works exist in the literature, no comprehensive surveys currently exist focusing exclusively on the strengths and limitations of using neuroevolution approaches in DNNs. Prolonged absence of such surveys can lead to a disjointed and fragmented field preventing DNNs researchers potentially adopting neuroevolutionary methods in their own research, resulting in lost opportunities for improving performance and wider application within real-world deep learning problems. This paper presents a comprehensive survey, discussion and evaluation of the state-of-the-art works on using EAs for architectural configuration and training of DNNs. Based on this survey, the paper highlights the most pertinent current issues and challenges in neuroevolution and identifies multiple promising future research directions.Comment: 20 pages (double column), 2 figures, 3 tables, 157 reference

Discrete and fuzzy dynamical genetic programming in the XCSF learning classifier system

A number of representation schemes have been presented for use within learning classifier systems, ranging from binary encodings to neural networks. This paper presents results from an investigation into using discrete and fuzzy dynamical system representations within the XCSF learning classifier system. In particular, asynchronous random Boolean networks are used to represent the traditional condition-action production system rules in the discrete case and asynchronous fuzzy logic networks in the continuous-valued case. It is shown possible to use self-adaptive, open-ended evolution to design an ensemble of such dynamical systems within XCSF to solve a number of well-known test problems

Uncertainty Quantification And Economic Dispatch Models For The Power Grid

The modern power grid is constrained by several challenges, such as increased penetration of Distributed Energy Resources (DER), rising demand for Electric Vehicle (EV) integration, and the need to schedule resources in real-time accurately. To address the above challenges, this dissertation offers solutions through data-driven forecasting models, topology-aware economic dispatch models, and efficient optional power flow calculations for large scale grids. Particularly, in chapter 2, a novel microgrid decomposition scheme is proposed to divide the large scale power grids into smaller microgrids. Here, a two-stage Nearest-Generator Girvan-Newman (NGGN) algorithm, a graphicalclustering-based approach, followed by a distributed economic dispatch model, is deployed to yield a 12.64% cost savings. In chapter 3, a deep-learning-based scheduling scheme is intended for the EVs in a household community that uses forecasted demand, consumer preferences and Time-of-use (TOU) pricing scheme to reduce electricity costs for the consumers and peak shaving for the utilities. In chapter 4, a hybrid machine learning model using GLM with other methods was designed to forecast wind generation data. Finally, in chapter 5, multiple formulations for Alternating Current Optimal Power Flow (ACOPF) were designed for large scale grids in a high-performance computing environment. The ACOPF formulations, namely, power balance polar, power balance Cartesian, and current balance Cartesian, are tested on bus systems ranging from a 9-bus to 25,000. The current balance Cartesian formulation had an average of 23% faster computational time than two other formulations on a 25,000 bus system

Forecasting price movements in betting exchanges using Cartesian Genetic Programming and ANN

Since the introduction of betting exchanges in 2000, there has been increased interest of ways to monetize on the new technology. Betting exchange markets are fairly similar to the financial markets in terms of their operation. Due to the lower market share and newer technology, there are very few tools available for automated trading for betting exchanges. The in-depth analysis of features available in commercial software demonstrates that there is no commercial software that natively supports machine learned strategy development. Furthermore, previously published academic software products are not publicly obtainable. Hence, this work concentrates on developing a full-stack solution from data capture, back-testing to automated Strategy Agent development for betting exchanges. Moreover, work also explores ways to forecast price movements within betting exchange using new machine learned trading strategies based on Artificial Neuron Networks (ANN) and Cartesian Genetic Programming (CGP). Automatically generated strategies can then be deployed on a server and require no human interaction. Data explored in this work were captured from 1st of January 2016 to 17th of May 2016 for all GB WIN Horse Racing markets (total of 204GB of data processing). Best found Strategy agent shows promising 83% Return on Investment (ROI) during simulated historical validation period of one month (15th of April 2016 to 16th of May 2016)