A genetic-algorithms based evolutionary computational neural network for modelling spatial interaction data

Building a feedforward computational neural network model (CNN) involves two distinct tasks: determination of the network topology and weight estimation. The specification of a problem adequate network topology is a key issue and the primary focus of this contribution. Up to now, this issue has been either completely neglected in spatial application domains, or tackled by search heuristics (see Fischer and Gopal 1994). With the view of modelling interactions over geographic space, this paper considers this problem as a global optimization problem and proposes a novel approach that embeds backpropagation learning into the evolutionary paradigm of genetic algorithms. This is accomplished by interweaving a genetic search for finding an optimal CNN topology with gradient-based backpropagation learning for determining the network parameters. Thus, the model builder will be relieved of the burden of identifying appropriate CNN-topologies that will allow a problem to be solved with simple, but powerful learning mechanisms, such as backpropagation of gradient descent errors. The approach has been applied to the family of three inputs, single hidden layer, single output feedforward CNN models using interregional telecommunication traffic data for Austria, to illustrate its performance and to evaluate its robustness.

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

GLOBAL OPTIMIZATION METHODS

Training a neural network is a difficult optimization problem because of numerous local minimums. Many global search algorithms have been used to train neural networks. However, local search algorithms are more efficient with computational resources, and therefore numerous random restarts with a local algorithm may be more effective than a global algorithm. This study uses Monte-Carlo simulations to determine the relative efficiency of a local search algorithm to 9 stochastic global algorithms. The computational requirements of the global algorithms are several times higher than the local algorithm and there is little gain in using the global algorithms to train neural networks.Research Methods/ Statistical Methods,

Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature

This report briefly motivates current research on evolutionary design of neural architectures (EDNA) and presents a short overview of major research issues in this area. It also includes a preliminary taxonomy of research on EDNA and an extensive bibliography of publications on this topic. The taxonomy is an attempt to categorize current research on EDNA in terms of major research issues addressed and approaches pursued. It is our hope that this will help identify open research questions as well as promising directions for further research on EDNA. The report also includes an appendix that provides some suggestions for effective use of the electronic version of the bibliography

NEUROEVOLUTION AND AN APPLICATION OF AN AGENT BASED MODEL FOR FINANCIAL MARKET

Market prediction is one of the most difficult problems for the machine learning community. Even though, successful trading strategies can be found for the training data using various optimization methods, these strategies usually do not perform well on the test data as expected. Therefore, selection of the correct strategy becomes problematic. In this study, we propose an evolutionary algorithm that produces a variation of trader agents ensuring that the trading strategies they use are different. We discuss that because the selection of the correct strategy is difficult, a variety of agents can be used simultaneously in order to reduce risk. We simulate trader agents on real market data and attempt to optimize their actions. Agent decisions are based on Echo State Networks. The agents take various market indicators as inputs and produce an action such as: buy or sell. We optimize the parameters of the echo state networks using evolutionary algorithms