Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Optimal Microgrid Topology Design and Siting of Distributed Generation Sources Using a Multi-Objective Substrate Layer Coral Reefs Optimization Algorithm

n this work, a problem of optimal placement of renewable generation and topology design for a Microgrid (MG) is tackled. The problem consists of determining the MG nodes where renewable energy generators must be optimally located and also the optimization of the MG topology design, i.e., deciding which nodes should be connected and deciding the lines’ optimal cross-sectional areas (CSA). For this purpose, a multi-objective optimization with two conflicting objectives has been used, utilizing the cost of the lines, C, higher as the lines’ CSA increases, and the MG energy losses, E, lower as the lines’ CSA increases. To characterize generators and loads connected to the nodes, on-site monitored annual energy generation and consumption profiles have been considered. Optimization has been carried out by using a novel multi-objective algorithm, the Multi-objective Substrate Layers Coral Reefs Optimization algorithm (Mo-SL-CRO). The performance of the proposed approach has been tested in a realistic simulation of a MG with 12 nodes, considering photovoltaic generators and micro-wind turbines as renewable energy generators, as well as the consumption loads from different commercial and industrial sites. We show that the proposed Mo-SL-CRO is able to solve the problem providing good solutions, better than other well-known multi-objective optimization techniques, such as NSGA-II or multi-objective Harmony Search algorithm.This research was partially funded by Ministerio de Economía, Industria y Competitividad, project number TIN2017-85887-C2-1-P and TIN2017-85887-C2-2-P, and by the Comunidad Autónoma de Madrid, project number S2013ICE-2933_02

Self-adaptive global best harmony search algorithm for training neural networks

AbstractThis paper addresses the application of Self-adaptive Global Best Harmony Search (SGHS) algorithm for the supervised training of feed-forward neural networks (NNs). A structure suitable to data representation of NNs is adapted to SGHS algorithm. The technique is empirically tested and verified by training NNs on two classification benchmarking problems. Overall training time, sum of squared errors, training and testing accuracies of SGHS algorithm is compared with other harmony search algorithms and the standard back-propagation algorithm. The experiments presented that the proposed algorithm lends itself very well to training of NNs and it is also highly competitive with the compared methods

Improved Adaptive Harmony Search algorithm for the resource levelling problem with minimal lags

The resource leveling problem (RLP) aims to provide the most efficient resource consumption as well as minimize the resource fluctuations without increasing the prescribed makespan of the construction project. Resource fluctuations are impractical, inefficient and costly when they happen on construction sites. Therefore, previous research has tried to find an efficient way to solve this problem. Metaheuristics using Harmony Search seem to be faster and more efficient than others, but present the same problem of premature convergence closing around local optimums. In order to diminish this issue, this study introduces an innovative Improved and Adaptive Harmony Search (IAHS) algorithm to improve the solution of the RLP with multiple resources. This IAHS algorithm has been tested with the standard Project Scheduling Problem Library for four metrics that provide different levelled profiles from rectangular to bell shapes. The results have been compared with the benchmarks available in the literature presenting a complete discussion of results. Additionally, a case study of 71 construction activities contemplating the widest possible set of conditions including continuity and discontinuity of flow relationships has been solved as example of application for real life construction projects. Finally, a visualizer tool has been developed to compare the effects of applying different metrics with an app for Excel. The IAHS algorithm is faster with better overall results than other metaheuristics. Results also show that the IAHS algorithm is especially fitted for the Sum of Squares Optimization metric. The proposed IAHS algorithm for the RLP is a starting point in order to develop user-friendly and practical computer applications to provide realistic, fast and good solutions for construction project managers.This research was partially supported by the FAPA program of Universidad de Los Andes, Colombia (code P14.246922.005/01). The authors would also like to thank the research group of Construction Engineering and Management (INgeco).Ponz Tienda, JL.; Salcedo-Bernal, A.; Pellicer Armiñana, E.; Benlloch Marco, J. (2017). Improved Adaptive Harmony Search algorithm for the resource levelling problem with minimal lags. Automation in Construction. 77:82-92. https://doi.org/10.1016/j.autcon.2017.01.018S82927