Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Streamflow forecasting using a hybrid LSTM-PSO approach: the case of Seyhan Basin

The conditions which affect the sustainability of water cause a number of serious environmental and hydrological problems. Effective and correct management of water resources constitutes an effective and important issue among scales. In this sense, a precise estimation of streamflow time series in rivers is one of the most important issues in optimal management of surface water resources. Therefore, a hybrid method combining particle swarm algorithm (PSO) and long short-term memory networks (LSTM) are proposed to predict flow with data obtained from different flow measurement stations. In this respect, the data gathered from three Flow Measurement Stations (FMS) from Zamanti and Eglence rivers located on Seyhan Basin are utilized. Besides, the proposed LSTM-PSO method is compared to an adaptive neuro-fuzzy inference system (ANFIS) and the LSTM benchmark model to demonstrate the performance achievement of proposed method. The prediction performances of the developed hybrid model and the others are tested on the determined stations. The forecasting performances of the models are determined with RMSE, MAE, MAPE, SD, and R-2 metrics. The comparison results indicated that the LSTM-PSO method provides highest results with values of R-2 (approximate to 0.9433), R-2 (approximate

Recent Development in Electricity Price Forecasting Based on Computational Intelligence Techniques in Deregulated Power Market

The development of artificial intelligence (AI) based techniques for electricity price forecasting (EPF) provides essential information to electricity market participants and managers because of its greater handling capability of complex input and output relationships. Therefore, this research investigates and analyzes the performance of different optimization methods in the training phase of artificial neural network (ANN) and adaptive neuro-fuzzy inference system (ANFIS) for the accuracy enhancement of EPF. In this work, a multi-objective optimization-based feature selection technique with the capability of eliminating non-linear and interacting features is implemented to create an efficient day-ahead price forecasting. In the beginning, the multi-objective binary backtracking search algorithm (MOBBSA)-based feature selection technique is used to examine various combinations of input variables to choose the suitable feature subsets, which minimizes, simultaneously, both the number of features and the estimation error. In the later phase, the selected features are transferred into the machine learning-based techniques to map the input variables to the output in order to forecast the electricity price. Furthermore, to increase the forecasting accuracy, a backtracking search algorithm (BSA) is applied as an efficient evolutionary search algorithm in the learning procedure of the ANFIS approach. The performance of the forecasting methods for the Queensland power market in the year 2018, which is well-known as the most competitive market in the world, is investigated and compared to show the superiority of the proposed methods over other selected methods.© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).fi=vertaisarvioitu|en=peerReviewed

A survey on financial applications of metaheuristics

Modern heuristics or metaheuristics are optimization algorithms that have been increasingly used during the last decades to support complex decision-making in a number of fields, such as logistics and transportation, telecommunication networks, bioinformatics, finance, and the like. The continuous increase in computing power, together with advancements in metaheuristics frameworks and parallelization strategies, are empowering these types of algorithms as one of the best alternatives to solve rich and real-life combinatorial optimization problems that arise in a number of financial and banking activities. This article reviews some of the works related to the use of metaheuristics in solving both classical and emergent problems in the finance arena. A non-exhaustive list of examples includes rich portfolio optimization, index tracking, enhanced indexation, credit risk, stock investments, financial project scheduling, option pricing, feature selection, bankruptcy and financial distress prediction, and credit risk assessment. This article also discusses some open opportunities for researchers in the field, and forecast the evolution of metaheuristics to include real-life uncertainty conditions into the optimization problems being considered.This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (TRA2013-48180-C3-P, TRA2015-71883-REDT), FEDER, and the Universitat Jaume I mobility program (E-2015-36)

Forecasting methods in energy planning models

Energy planning models (EPMs) play an indispensable role in policy formulation and energy sector development. The forecasting of energy demand and supply is at the heart of an EPM. Different forecasting methods, from statistical to machine learning have been applied in the past. The selection of a forecasting method is mostly based on data availability and the objectives of the tool and planning exercise. We present a systematic and critical review of forecasting methods used in 483 EPMs. The methods were analyzed for forecasting accuracy; applicability for temporal and spatial predictions; and relevance to planning and policy objectives. Fifty different forecasting methods have been identified. Artificial neural network (ANN) is the most widely used method, which is applied in 40% of the reviewed EPMs. The other popular methods, in descending order, are: support vector machine (SVM), autoregressive integrated moving average (ARIMA), fuzzy logic (FL), linear regression (LR), genetic algorithm (GA), particle swarm optimization (PSO), grey prediction (GM) and autoregressive moving average (ARMA). In terms of accuracy, computational intelligence (CI) methods demonstrate better performance than that of the statistical ones, in particular for parameters with greater variability in the source data. However, hybrid methods yield better accuracy than that of the stand-alone ones. Statistical methods are useful for only short and medium range, while CI methods are preferable for all temporal forecasting ranges (short, medium and long). Based on objective, most EPMs focused on energy demand and load forecasting. In terms geographical coverage, the highest number of EPMs were developed on China. However, collectively, more models were established for the developed countries than the developing ones. Findings would benefit researchers and professionals in gaining an appreciation of the forecasting methods, and enable them to select appropriate method(s) to meet their needs

Improving the sustainability of coal SC in both developed and developing countries by incorporating extended exergy accounting and different carbon reduction policies

In the age of Industry 4.0 and global warming, it is inevitable for decision-makers to change the way they view the coal supply chain (SC). In nature, energy is the currency, and nature is the source of energy for humankind. Coal is one of the most important sources of energy which provides much-needed electricity, as well as steel and cement production. This manuscript-based PhD thesis examines the coal SC network as well as the four carbon reduction strategies and plans to develop a comprehensive model for sustainable design. Thus, the Extended Exergy Accounting (EEA) method is incorporated into a coal SC under economic order quantity (EOQ) and economic production quantity (EPQs) in an uncertain environment. Using a real case study in coal SC in Iran, four carbon reduction policies such as carbon tax (Chapter 5), carbon trade (Chapter 6), carbon cap (Chapter 7), and carbon offset (Chapter 8) are examined. Additionally, all carbon policies are compared for sustainable performance of coal SCs in some developed and developing countries (the USA, China, India, Germany, Canada, Australia, etc.) with the world's most significant coal consumption. The objective function of the four optimization models under each carbon policy is to minimize the total exergy (in Joules as opposed to Dollars/Euros) of the coal SC in each country. The models have been solved using three recent metaheuristic algorithms, including Ant lion optimizer (ALO), Lion optimization algorithm (LOA), and Whale optimization algorithm (WOA), as well as three popular ones, such as Genetic algorithm (GA), Ant colony optimization (ACO), and Simulated annealing (SA), are suggested to determine a near-optimal solution to an exergy fuzzy nonlinear integer-programming (EFNIP). Moreover, the proposed metaheuristic algorithms are validated by using an exact method (by GAMS software) in small-size test problems. Finally, through a sensitivity analysis, this dissertation compares the effects of applying different percentages of exergy parameters (capital, labor, and environmental remediation) to coal SC models in each country. Using this approach, we can determine the best carbon reduction policy and exergy percentage that leads to the most sustainable performance (the lowest total exergy per Joule). The findings of this study may enhance the related research of sustainability assessment of SC as well as assist coal enterprises in making logical and measurable decisions

A Review of Hybrid Soft Computing and Data Pre-Processing Techniques to Forecast Freshwater Quality’s Parameters: Current Trends and Future Directions

Water quality has a significant influence on human health. As a result, water quality parameter modelling is one of the most challenging problems in the water sector. Therefore, the major factor in choosing an appropriate prediction model is accuracy. This research aims to analyse hybrid techniques and pre-processing data methods in freshwater quality modelling and forecasting. Hybrid approaches have generally been seen as a potential way of improving the accuracy of water quality modelling and forecasting compared with individual models. Consequently, recent studies have focused on using hybrid models to enhance forecasting accuracy. The modelling of dissolved oxygen is receiving more attention. From a review of relevant articles, it is clear that hybrid techniques are viable and precise methods for water quality prediction. Additionally, this paper presents future research directions to help researchers predict freshwater quality variables

A scattering and repulsive swarm intelligence algorithm for solving global optimization problems

The firefly algorithm (FA), as a metaheuristic search method, is useful for solving diverse optimization problems. However, it is challenging to use FA in tackling high dimensional optimization problems, and the random movement of FA has a high likelihood to be trapped in local optima. In this research, we propose three improved algorithms, i.e., Repulsive Firefly Algorithm (RFA), Scattering Repulsive Firefly Algorithm (SRFA), and Enhanced SRFA (ESRFA), to mitigate the premature convergence problem of the original FA model. RFA adopts a repulsive force strategy to accelerate fireflies (i.e. solutions) to move away from unpromising search regions, in order to reach global optimality in fewer iterations. SRFA employs a scattering mechanism along with the repulsive force strategy to divert weak neighbouring solutions to new search regions, in order to increase global exploration. Motivated by the survival tactics of hawk-moths, ESRFA incorporates a hovering-driven attractiveness operation, an exploration-driven evading mechanism, and a learning scheme based on the historical best experience in the neighbourhood to further enhance SRFA. Standard and CEC2014 benchmark optimization functions are used for evaluation of the proposed FA-based models. The empirical results indicate that ESRFA, SRFA and RFA significantly outperform the original FA model, a number of state-of-the-art FA variants, and other swarm-based algorithms, which include Simulated Annealing, Cuckoo Search, Particle Swarm, Bat Swarm, Dragonfly, and Ant-Lion Optimization, in diverse challenging benchmark functions

Short-term Traffic Flow Prediction Based on Genetic Artificial Neural Network and Exponential Smoothing

In order to improve the accuracy of short-term traffic flow prediction, a combined model composed of artificial neural network optimized by using Genetic Algorithm (GA) and Exponential Smoothing (ES) has been proposed. By using the metaheuristic optimal search ability of GA, the connection weight and threshold of the feedforward neural network trained by a backpropagation algorithm are optimized to avoid the feedforward neural network falling into local optimum, and the prediction model of Genetic Artificial Neural Network (GANN) is established. An ES prediction model is presented then. In order to take the advantages of the two models, the combined model is composed of a weighted average, while the weight of the combined model is determined according to the prediction mean square error of the single model. The road traffic flow data of Xuancheng, Anhui Province with an observation interval of 5 min are used for experimental verification. Additionally, the feedforward neural network model, GANN model, ES model and combined model are compared and analysed, respectively. The results show that the prediction accuracy of the optimized feedforward neural network is much higher than that before the optimization. The prediction accuracy of the combined model is higher than that of the two single models, which verifies the feasibility and effectiveness of the combined model