Evolutionary Multi-objective Optimization for Simultaneous Generation of Signal-Type and Symbol-Type Representations

It has been a controversial issue in the research of cognitive science and artificial intelligence whether signal-type representations (typically connectionist networks) or symbol-type representations (e.g., semantic networks, production systems) should be used. Meanwhile, it has also been recognized that both types of information representations might exist in the human brain. In addition, symbol-type representations are often very helpful in gaining insights into unknown systems. For these reasons, comprehensible symbolic rules need to be extracted from trained neural networks. In this paper, an evolutionary multi-objective algorithm is employed to generate multiple models that facilitate the generation of signal-type and symbol-type representations simultaneously. It is argued that one main difference between signal-type and symbol-type representations lies in the fact that the signal-type representations are models of a higher complexity (fine representation), whereas symbol-type representations are models of a lower complexity (coarse representation). Thus, by generating models with a spectrum of model complexity, we are able to obtain a population of models of both signal-type and symbol-type quality, although certain post-processing is needed to get a fully symbol-type representation. An illustrative example is given on generating neural networks for the breast cancer diagnosis benchmark problem. © Springer-Verlag Berlin Heidelberg 2005

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Multi objective genetic algorithm for training three term backpropagation network

Multi Objective Evolutionary Algorithms has been applied for learning problem in Artificial Neural Networks to improve the generalization of the training and testing unseen data.This paper proposes the simultaneous optimization method for training Three Term Back Propagation Network (TTBPN) learning using Multi Objective Genetic Algorithm.The Non-dominated Sorting Genetic Algorithm II is applied to optimize the TTBPN structure by simultaneously reducing the error and complexity in terms of number of hidden nodes of the network for better accuracy in classification problem.This methodology is applied in two kinds of multiclasses data set obtained from the University of California at Irvine repository.The results obtained for training and testing on the datasets illustrate less network error and better classification accuracy, besides having simple architecture for the TTBPN

Bacterial Foraging Based Channel Equalizers

A channel equalizer is one of the most important subsystems in any digital communication receiver. It is also the subsystem that consumes maximum computation time in the receiver. Traditionally maximum-likelihood sequence estimation (MLSE) was the most popular form of equalizer. Owing to non-stationary characteristics of the communication channel MLSE receivers perform poorly. Under these circumstances ‘Maximum A-posteriori Probability (MAP)’ receivers also called Bayesian receivers perform better. Natural selection tends to eliminate animals with poor “foraging strategies” and favor the propagation of genes of those animals that have successful foraging strategies since they are more likely to enjoy reproductive success. After many generations, poor foraging strategies are either eliminated or shaped into good ones (redesigned). Logically, such evolutionary principles have led scientists in the field of “foraging theory” to hypothesize that it is appropriate to model the activity of foraging as an optimization process. This thesis presents an investigation on design of bacterial foraging based channel equalizer for digital communication. Extensive simulation studies shows that the performance of the proposed receiver is close to optimal receiver for variety of channel conditions. The proposed receiver also provides near optimal performance when channel suffers from nonlinearities

Backpropagation Neural Network Based on Local Search Strategy and Enhanced Multi-objective Evolutionary Algorithm for Breast Cancer Diagnosis

The role of intelligence techniques is becoming more significant in detecting and diagnosis of medical data. However, the performance of such methods is based on the algorithms or technique. In this paper, we develop an intelligent technique using multiobjective evolutionary method hybrid with a local search approach to enhance the backpropagation neural network. First, we enhance the famous multiobjective evolutionary algorithms, which is a non-dominated sorting genetic algorithm (NSGA-II). Then, we hybrid the enhanced algorithm with the local search strategy to ensures the acceleration of the convergence speed to the non-dominated front. In addition, such hybridization get the solutions achieved are well spread over it. As a result of using a local search method the quality of the Pareto optimal solutions are increased and all individuals in the population are enhanced. The key notion of the proposed algorithm was to  show a new technique to settle automaticly artificial neural network design problem. The empirical results generated by the proposed intelligent technique evaluated by applying to the breast cancer dataset and emphasize the capability of the proposed algorithm to improve the results. The network size and accuracy results of the proposed method are better than the previous methods. Therefore, the method is then capable of finding a proper number of hidden neurons and error rates of the BP algorithm

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

A variety of methods have been applied to the architectural configuration and learning or training of artificial deep neural networks (DNN). These methods play a crucial role in the success or failure of the DNN for most problems and applications. Evolutionary Algorithms (EAs) are gaining momentum as a computationally feasible method for the automated optimisation and training of DNNs. Neuroevolution is a term which describes these processes of automated configuration and training of DNNs using EAs. While many works exist in the literature, no comprehensive surveys currently exist focusing exclusively on the strengths and limitations of using neuroevolution approaches in DNNs. Prolonged absence of such surveys can lead to a disjointed and fragmented field preventing DNNs researchers potentially adopting neuroevolutionary methods in their own research, resulting in lost opportunities for improving performance and wider application within real-world deep learning problems. This paper presents a comprehensive survey, discussion and evaluation of the state-of-the-art works on using EAs for architectural configuration and training of DNNs. Based on this survey, the paper highlights the most pertinent current issues and challenges in neuroevolution and identifies multiple promising future research directions.Comment: 20 pages (double column), 2 figures, 3 tables, 157 reference