Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Differential evolution with an evolution path: a DEEP evolutionary algorithm

Utilizing cumulative correlation information already existing in an evolutionary process, this paper proposes a predictive approach to the reproduction mechanism of new individuals for differential evolution (DE) algorithms. DE uses a distributed model (DM) to generate new individuals, which is relatively explorative, whilst evolution strategy (ES) uses a centralized model (CM) to generate offspring, which through adaptation retains a convergence momentum. This paper adopts a key feature in the CM of a covariance matrix adaptation ES, the cumulatively learned evolution path (EP), to formulate a new evolutionary algorithm (EA) framework, termed DEEP, standing for DE with an EP. Without mechanistically combining two CM and DM based algorithms together, the DEEP framework offers advantages of both a DM and a CM and hence substantially enhances performance. Under this architecture, a self-adaptation mechanism can be built inherently in a DEEP algorithm, easing the task of predetermining algorithm control parameters. Two DEEP variants are developed and illustrated in the paper. Experiments on the CEC'13 test suites and two practical problems demonstrate that the DEEP algorithms offer promising results, compared with the original DEs and other relevant state-of-the-art EAs

Meta-heuristic algorithms in car engine design: a literature survey

Meta-heuristic algorithms are often inspired by natural phenomena, including the evolution of species in Darwinian natural selection theory, ant behaviors in biology, flock behaviors of some birds, and annealing in metallurgy. Due to their great potential in solving difficult optimization problems, meta-heuristic algorithms have found their way into automobile engine design. There are different optimization problems arising in different areas of car engine management including calibration, control system, fault diagnosis, and modeling. In this paper we review the state-of-the-art applications of different meta-heuristic algorithms in engine management systems. The review covers a wide range of research, including the application of meta-heuristic algorithms in engine calibration, optimizing engine control systems, engine fault diagnosis, and optimizing different parts of engines and modeling. The meta-heuristic algorithms reviewed in this paper include evolutionary algorithms, evolution strategy, evolutionary programming, genetic programming, differential evolution, estimation of distribution algorithm, ant colony optimization, particle swarm optimization, memetic algorithms, and artificial immune system

Application of biosignal-driven intelligent systems for multifunction prosthesis control

University of Technology, Sydney. Faculty of Engineering and Information Technology.Prosthetic devices aim to provide an artificial alternative to missing limbs. The controller for such devices is usually driven by the biosignals generated by the human body, particularly Electromyogram (EMG) or Electroencephalogram (EEG) signals. Such a controller utilizes a pattern recognition approach to classify the EMG signal recorded from the human muscles or the EEG signal from the brain. The aim of this thesis is to improve the EMG and EEG pattern classification accuracy. Due to the fact that the success of pattern recognition based biosignal driven systems highly depends on the quality of extracted features, a number of novel, robust, hybrid and innovative methods are proposed to achieve better performance. These methods are developed to effectively tackle many of the limitations of existing systems, in particular feature representation and dimensionality reduction. A set of knowledge extraction methods that can accurately and rapidly identify the most important attributes for classifying the arm movements are formulated. This is accomplished through the following: 1. Developing a new feature extraction technique that can identify the most important features from the high-dimensional time-frequency representation of the multichannel EMG and EEG signals. For this task, an information content estimation method using fuzzy entropies and fuzzy mutual information is proposed to identify the optimal wravelet packet transform decomposition for classification. 2. Developing a powerful variable (feature or channel) selection paradigm to improve the performance of multi-channel EMG and EEG driven systems. This will eventually lead to the development of a combined channel and feature selection technique as one possible scheme for dimensionality reduction. Two novel feature selection methods are developed under this scheme utilizing the ant colony arid differential evolution optimization techniques. The differential evolution optimization technique is further modified in a novel attempt in employing a float optimizer for the combinatorial task of feature selection, proving powerful performance by both methods. 3. Developing two feature projection techniques that extract a small subset of highly informative discriminant features, thus acting as an alternative scheme for dimensionality reduction. The two methods represent novel variations to fuzzy discriminant analysis based projection techniques. In addition, an extension to the non-linear discriminant analysis is proposed based on a mixture of differential evolution and fuzzy discriminant analysis. The testing and verification process of the proposed methods on different EMG and EEG datasets provides very encouraging results

Cloud computing resource scheduling and a survey of its evolutionary approaches

A disruptive technology fundamentally transforming the way that computing services are delivered, cloud computing offers information and communication technology users a new dimension of convenience of resources, as services via the Internet. Because cloud provides a finite pool of virtualized on-demand resources, optimally scheduling them has become an essential and rewarding topic, where a trend of using Evolutionary Computation (EC) algorithms is emerging rapidly. Through analyzing the cloud computing architecture, this survey first presents taxonomy at two levels of scheduling cloud resources. It then paints a landscape of the scheduling problem and solutions. According to the taxonomy, a comprehensive survey of state-of-the-art approaches is presented systematically. Looking forward, challenges and potential future research directions are investigated and invited, including real-time scheduling, adaptive dynamic scheduling, large-scale scheduling, multiobjective scheduling, and distributed and parallel scheduling. At the dawn of Industry 4.0, cloud computing scheduling for cyber-physical integration with the presence of big data is also discussed. Research in this area is only in its infancy, but with the rapid fusion of information and data technology, more exciting and agenda-setting topics are likely to emerge on the horizon