Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

A comprehensive survey on cultural algorithms

Peer reviewedPostprin

A Survey on Adaptation Strategies for Mutation and Crossover Rates of Differential Evolution Algorithm

Differential Evolution (DE), the well-known optimization algorithm, is a tool under the roof of Evolutionary Algorithms (EAs) for solving non-linear and non-differential optimization problems. DE has many qualities in its hand, which are attributing to its popularity. DE also is known for its simplicity in solving the given problem with few control parameters: the population size (NP), the mutation rate (F) and the crossover rate (Cr). To avoid the difficulty involved in setting of suitable values for NP, F and Cr many parameter adaptation strategies are proposed in the literature. This paper is to present the working principle of the parameter adaptation strategies of F and Cr. The adaptation strategies are categorized based on the logic used by the authors, and clear insights about all the categories are presented

Fuzzy adaptive teaching learning-based optimization for solving unconstrained numerical optimization problems

Teaching learning-based optimization is one of the widely accepted metaheuristic algorithms inspired by teaching and learning within classrooms. It has successfully addressed several real-world optimization problems, but it may still be trapped in local optima and may suffer from the problem of premature convergence in the case of solving some challenging optimization problems. To overcome these drawbacks and to achieve an appropriate percentage of exploitation and exploration, this study presents a new modified teaching learning-based optimization algorithm called the fuzzy adaptive teaching learning-based optimization algorithm. The proposed fuzzy adaptive teaching learning-based optimization algorithm uses three measures from the search space, namely, quality measure, diversification measure, and intensification measure. As the 50-50 probabilities for exploitation and exploration in the basic teaching learning-based optimization algorithm may be counterproductive, the Mamdani-type fuzzy inference system of the new algorithm takes these measures as a crisp inputs and generates selection as crisp output to choose either exploitation or exploration based on the current search requirement. This fuzzy-based adaptive selection helps to adequately balance global search or exploration and local search or exploitation operations during the search process as these operations are intrinsically dynamic. The performance of the fuzzy adaptive teaching learning-based optimization is evaluated against other metaheuristic algorithms including basic teaching learning-based optimization on 23 unconstrained global test functions. Moreover, adaptive teaching learning-based optimization is used to search for near-optimal values for the four parameters of the COCOMO II model, which are then tested for validity on a software project of NASA. Analysis and comparison of the obtained results indicate the efficiency and competitiveness of the proposed algorithm in addressing unconstrained continuous optimization tasks

Evolutionary Computation 2020

Intelligent optimization is based on the mechanism of computational intelligence to refine a suitable feature model, design an effective optimization algorithm, and then to obtain an optimal or satisfactory solution to a complex problem. Intelligent algorithms are key tools to ensure global optimization quality, fast optimization efficiency and robust optimization performance. Intelligent optimization algorithms have been studied by many researchers, leading to improvements in the performance of algorithms such as the evolutionary algorithm, whale optimization algorithm, differential evolution algorithm, and particle swarm optimization. Studies in this arena have also resulted in breakthroughs in solving complex problems including the green shop scheduling problem, the severe nonlinear problem in one-dimensional geodesic electromagnetic inversion, error and bug finding problem in software, the 0-1 backpack problem, traveler problem, and logistics distribution center siting problem. The editors are confident that this book can open a new avenue for further improvement and discoveries in the area of intelligent algorithms. The book is a valuable resource for researchers interested in understanding the principles and design of intelligent algorithms

Time-Cost Tradeoff and Resource-Scheduling Problems in Construction: A State-of-the-Art Review

Duration, cost, and resources are defined as constraints in projects. Consequently, Construction manager needs to balance between theses constraints to ensure that project objectives are met. Choosing the best alternative of each activity is one of the most significant problems in construction management to minimize project duration, project cost and also satisfies resources constraints as well as smoothing resources. Advanced computer technologies could empower construction engineers and project managers to make right, fast and applicable decisions based on accurate data that can be studied, optimized, and quantified with great accuracy. This article strives to find the recent improvements of resource-scheduling problems and time-cost trade off and the interacting between them which can be used in innovating new approaches in construction management. To achieve this goal, a state-of-the-art review, is conducted as a literature sample including articles implying three areas of research; time-cost trade off, constrained resources and unconstrained resources. A content analysis is made to clarify contributions and gaps of knowledge to help suggesting and specifying opportunities for future research

A Harris Hawks Optimization Based Single- and Multi-Objective Optimal Power Flow Considering Environmental Emission

The electric sector is majorly concerned about the greenhouse and non-greenhouse gas emissions generated from both conventional and renewable energy sources, as this is becoming a major issue globally. Thus, the utilities must adhere to certain environmental guidelines for sustainable power generation. Therefore, this paper presents a novel nature-inspired and population-based Harris Hawks Optimization (HHO) methodology for controlling the emissions from thermal generating sources by solving single and multi-objective Optimal Power Flow (OPF) problems. The OPF is a non-linear, non-convex, constrained optimization problem that primarily aims to minimize the fitness function by satisfying the equality and inequality constraints of the system. The cooperative behavior and dynamic chasing patterns of hawks to pounce on escaping prey is modeled mathematically to minimize the objective function. In this paper, fuel cost, real power loss and environment emissions are regarded as single and multi-objective functions for optimal adjustments of power system control variables. The different conflicting framed multi-objective functions have been solved using weighted sums using a no-preference method. The presented method is coded using MATLAB software and an IEEE (Institute of Electrical and Electronics Engineers) 30-bus. The system was used to demonstrate the effectiveness of selective objectives. The obtained results are compared with the other Artificial Intelligence (AI) techniques such as the Whale Optimization Algorithm (WOA), the Salp Swarm Algorithm (SSA), Moth Flame (MF) and Glow Warm Optimization (GWO). Additionally, the study on placement of Distributed Generation (DG) reveals that the system losses and emissions are reduced by an amount of 9.8355% and 26.2%, respectively

Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches

This paper surveys the field of multiagent deep reinforcement learning. The combination of deep neural networks with reinforcement learning has gained increased traction in recent years and is slowly shifting the focus from single-agent to multiagent environments. Dealing with multiple agents is inherently more complex as (a) the future rewards depend on the joint actions of multiple players and (b) the computational complexity of functions increases. We present the most common multiagent problem representations and their main challenges, and identify five research areas that address one or more of these challenges: centralised training and decentralised execution, opponent modelling, communication, efficient coordination, and reward shaping. We find that many computational studies rely on unrealistic assumptions or are not generalisable to other settings; they struggle to overcome the curse of dimensionality or nonstationarity. Approaches from psychology and sociology capture promising relevant behaviours such as communication and coordination. We suggest that, for multiagent reinforcement learning to be successful, future research addresses these challenges with an interdisciplinary approach to open up new possibilities for more human-oriented solutions in multiagent reinforcement learning.Comment: 37 pages, 6 figure