An Improved Bees Algorithm for Training Deep Recurrent Networks for Sentiment Classification

Recurrent neural networks (RNNs) are powerful tools for learning information from temporal sequences. Designing an optimum deep RNN is difficult due to configuration and training issues, such as vanishing and exploding gradients. In this paper, a novel metaheuristic optimisation approach is proposed for training deep RNNs for the sentiment classification task. The approach employs an enhanced Ternary Bees Algorithm (BA-3+), which operates for large dataset classification problems by considering only three individual solutions in each iteration. BA-3+ combines the collaborative search of three bees to find the optimal set of trainable parameters of the proposed deep recurrent learning architecture. Local learning with exploitative search utilises the greedy selection strategy. Stochastic gradient descent (SGD) learning with singular value decomposition (SVD) aims to handle vanishing and exploding gradients of the decision parameters with the stabilisation strategy of SVD. Global learning with explorative search achieves faster convergence without getting trapped at local optima to find the optimal set of trainable parameters of the proposed deep recurrent learning architecture. BA-3+ has been tested on the sentiment classification task to classify symmetric and asymmetric distribution of the datasets from different domains, including Twitter, product reviews, and movie reviews. Comparative results have been obtained for advanced deep language models and Differential Evolution (DE) and Particle Swarm Optimization (PSO) algorithms. BA-3+ converged to the global minimum faster than the DE and PSO algorithms, and it outperformed the SGD, DE, and PSO algorithms for the Turkish and English datasets. The accuracy value and F1 measure have improved at least with a 30–40% improvement than the standard SGD algorithm for all classification datasets. Accuracy rates in the RNN model trained with BA-3+ ranged from 80% to 90%, while the RNN trained with SGD was able to achieve between 50% and 60% for most datasets. The performance of the RNN model with BA-3+ has as good as for Tree-LSTMs and Recursive Neural Tensor Networks (RNTNs) language models, which achieved accuracy results of up to 90% for some datasets. The improved accuracy and convergence results show that BA-3+ is an efficient, stable algorithm for the complex classification task, and it can handle the vanishing and exploding gradients problem of deep RNNs

Urban Water Demand Prediction for a City that Suffers from Climate Change and Population Growth: Gauteng Province case study

The proper management of municipal water system is essential to sustain cities and support water security of societies. Urban water estimating has always been a challenging task for managers of water utilities and policymakers. This paper applies a novel methodology that includes data pre-processing and Artificial Neural Network (ANN) optimized with Backtracking Search Algorithm (BSA-ANN) to estimate monthly water demand in relation to previous water consumption. Historical data of monthly water consumption in the Gauteng Province, South Africa, for the period 2007–2016, were selected for the creation and evaluation of the methodology. Data pre-processing techniques played a crucial role in the enhancing of the quality of the data before creating the prediction model. The BSA-ANN model yielded the best result with a root mean square error and a coefficient of efficiency of 0.0099 mega liters and 0.979, respectively. Also, it proved more efficient and reliable than the Crow Search Algorithm (CSA-ANN), based on the scale of error. Overall, this paper presents a new application for the hybrid model BSA-ANN that can be successfully used to predict water demand with high accuracy, in a city that heavily suffers from the impact of climate change and population growth

The Application of Nature-inspired Metaheuristic Methods for Optimising Renewable Energy Problems and the Design of Water Distribution Networks

This work explores the technical challenges that emerge when applying bio-inspired optimisation methods to real-world engineering problems. A number of new heuristic algorithms were proposed and tested to deal with these challenges. The work is divided into three main dimensions: i) One of the most significant industrial optimisation problems is optimising renewable energy systems. Ocean wave energy is a promising technology for helping to meet future growth in global energy demand. However, the current technologies of wave energy converters (WECs) are not fully developed because of technical engineering and design challenges. This work proposes new hybrid heuristics consisting of cooperative coevolutionary frameworks and neuro-surrogate optimisation methods for optimising WECs problem in three domains, including position, control parameters, and geometric parameters. Our problem-specific algorithms perform better than existing approaches in terms of higher quality results and the speed of convergence. ii) The second part applies search methods to the optimization of energy output in wind farms. Wind energy has key advantages in terms of technological maturity, cost, and life-cycle greenhouse gas emissions. However, designing an accurate local wind speed and power prediction is challenging. We propose two models for wind speed and power forecasting for two wind farms located in Sweden and the Baltic Sea by a combination of recurrent neural networks and evolutionary search algorithms. The proposed models are superior to other applied machine learning methods. iii) Finally, we investigate the design of water distribution systems (WDS) as another challenging real-world optimisation problem. WDS optimisation is demanding because it has a high-dimensional discrete search space and complex constraints. A hybrid evolutionary algorithm is suggested for minimising the cost of various water distribution networks and for speeding up the convergence rate of search.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 202

Forecasting model for short-term wind speed using robust local mean decomposition, deep neural networks, intelligent algorithm, and error correction

Wind power generation has aroused widespread concern worldwide. Accurate prediction of wind speed is very important for the safe and economic operation of the power grid. This paper presents a short-term wind speed prediction model which includes data decomposition, deep learning, intelligent algorithm optimization, and error correction modules. First, the robust local mean decomposition (RLMD) is applied to the original wind speed data to reduce the non-stationarity of the data. Then, the salp swarm algorithm (SSA) is used to determine the optimal parameter combination of the bidirectional gated recurrent unit (BiGRU) to ensure prediction quality. In order to eliminate the predictable components of the error further, a correction module based on the improved salp swarm algorithm (ISSA) and deep extreme learning machine (DELM) is constructed. The exploration and exploitation capability of the original SSA is enhanced by introducing a crazy operator and dynamic learning strategy, and the input weights and thresholds in the DELM are optimized by the ISSA to improve the generalization ability of the model. The actual data of wind farms are used to verify the advancement of the proposed model. Compared with other models, the results show that the proposed model has the best prediction performance. As a powerful tool, the developed forecasting system is expected to be further used in the energy system

Flood Forecasting Using Machine Learning Methods

This book is a printed edition of the Special Issue Flood Forecasting Using Machine Learning Methods that was published in Wate

AI-big data analytics for building automation and management systems: a survey, actual challenges and future perspectives

In theory, building automation and management systems (BAMSs) can provide all the components and functionalities required for analyzing and operating buildings. However, in reality, these systems can only ensure the control of heating ventilation and air conditioning system systems. Therefore, many other tasks are left to the operator, e.g. evaluating buildings’ performance, detecting abnormal energy consumption, identifying the changes needed to improve efficiency, ensuring the security and privacy of end-users, etc. To that end, there has been a movement for developing artificial intelligence (AI) big data analytic tools as they offer various new and tailor-made solutions that are incredibly appropriate for practical buildings’ management. Typically, they can help the operator in (i) analyzing the tons of connected equipment data; and; (ii) making intelligent, efficient, and on-time decisions to improve the buildings’ performance. This paper presents a comprehensive systematic survey on using AI-big data analytics in BAMSs. It covers various AI-based tasks, e.g. load forecasting, water management, indoor environmental quality monitoring, occupancy detection, etc. The first part of this paper adopts a well-designed taxonomy to overview existing frameworks. A comprehensive review is conducted about different aspects, including the learning process, building environment, computing platforms, and application scenario. Moving on, a critical discussion is performed to identify current challenges. The second part aims at providing the reader with insights into the real-world application of AI-big data analytics. Thus, three case studies that demonstrate the use of AI-big data analytics in BAMSs are presented, focusing on energy anomaly detection in residential and office buildings and energy and performance optimization in sports facilities. Lastly, future directions and valuable recommendations are identified to improve the performance and reliability of BAMSs in intelligent buildings

Iberian Energy Market: Spot Price Forecast by Modelling Market Offers

Electricity is a very special commodity since it is economically non-storable, and thus requiring a constant balance between production and consumption. At the corporate level, electricity price forecasts have become a fundamental input to energy companies’ decision making mechanisms [22, 45]. Electric utilities are higly vulnerable to economical crisis, since they generally cannot pass their excess costs on the wholesale market to the retail consumers [77] and, since the price depends on variables like weather (temperature, wind speed, precipitation, etc.) and the intensity of business and everyday activities (on-peak vs. off-peak hours, weekdays vs. weekends, holidays and near-holidays, etc.) it shows specific dynamics not observed in any other market, exhibiting seasonality at the daily, weekly and annual levels, and abrupt, short-lived and generally unanticipated price spikes. These extreme price volatility make price forecasts from a few hours to a few months ahead to become of particular interest to power portfolio managers. An utility company or large industrial consumer who is able to accurately forecast the wholesale prices and it’s volatility, can adjust its bidding strategy and its own production/consumption schedule in order to reduce the risk or maximize the profits in day-ahead trading. In this work I discuss the dynamics of the Iberian electricity day-ahead market (OMIE), review the state-of-the-art forecasting techniques and introduce a new approach to Electricity Price Forecasting, by forecasting the underlying dynamics, the market demand/supply curves. With this method it is possible to predict not only the electricity prices for the next hours, but also the market curves, which can then be used for risk management and a more accurate schedule of generation units. I analyze the model results and benchmark them against other models in the industry.A eletricidade é uma commodity muito especial, uma vez que não é possível armazená-la, e por isso, requer um constante equilíbrio entre a produção e consumo. ao nível empresarial, a previsão de preços de eletricidade tornou-se um input fundamental para os mecanismos de tomada de decisão das companhias [22, 45]. As empresas de eletricidade são altamente vulneráveis a crises económicas, uma vez que, em geral, não conseguem passar os seus custos excessivos para o mercado retalhista [77] e, uma vez que o preço depende de variáveis como meteorologia (temperatura, velocidade do vento, precipitação, etc.) e da intensidade de negócio e das atividades do dia-a-dia (pico vs vazio, dias da semana vs fim-de-semana, feriados e pontes, etc.) apresenta uma dinâmica que não é observada em mais nenhum mercado, com sazonalidade diária, semanal e anual, e com picos de preço abruptos de pouca duração e, em termos gerais, impossíveis de antecipar. Esta volatilidade de preços torna a previsão de preços particularmente interessante para gestores de portfólio, seja a curto ou a longo prazo. Uma companhia de eletricidade ou grande consumidor industrial que seja capaz de prever corretamente os preços do mercado grossista e a sua volatilidade, pode ajustar a estratégia de oferta da sua produção/seu consumo de maneira a reduzir o risco ou maximizar os ganhos no mercado à vista. Neste trabalho abordo a dinâmica do mercado de eletricidade ibérico (Operador de Mercado Iberico - Polo Español (OMIE)), revendo o estado da arte dos métodos de previsão de preços de eletricidade, e introduzo uma nova técnica de previsão de preços de eletricidade, através da previsão da sua dinâmica subjacente, as curvas de mercado da procura e oferta. Com este método é possível prever, não só o preço de eletricidade para as próximas horas, mas também as próprias curvas de oferta, o que pode ser utilizado na gestão de risco ao melhor a capacidade de programar as suas unidades de geração.Os resultados do modelo são analisados e comparados com outros modelos já utilizados na industria

Efficient Optimization and Robust Value Quantification of Enhanced Oil Recovery Strategies

With an increasing demand for hydrocarbon reservoir produces such as oil, etc., and difficulties in finding green oil fields, the use of Enhanced Oil Recovery (EOR) methods such as polymer, Smart water, and solvent flooding for further development of existing fields can not be overemphasized. For reservoir profitability and reduced environmental impact, it is crucial to consider appropriate well control settings of EOR methods for given reservoir characterization. Moreover, finding appropriate well settings requires solving a constrained optimization problem with suitable numerical solution methods. Conventionally, the solution method requires many iterations involving several computationally demanding function evaluations before convergence to the appropriate near optimum. The major subject of this thesis is to develop an efficient and accurate solution method for constrained optimization problems associated with EOR methods for their value quantifications and ranking in the face of reservoir uncertainties. The first contribution of the thesis develops a solution method based on the inexact line search method (with Ensemble Based Optimization (EnOpt) for approximate gradient computation) for robust constrained optimization problems associated with polymer, Smart water, and solvent flooding. Here, the objective function is the expectation of the Net Present Value (NPV) function over given geological realizations. For a given set of well settings, the NPV function is defined based on the EOR simulation model, which follows from an appropriate extension of the black-oil model. The developed solution method is used to find the economic benefits and also the ranking of EOR methods for different oil reservoirs developed to mimic North Sea reservoirs. Performing the entire optimization routine in a transformed domain along with truncations has been a common practice for handling simple linear constraints in reservoir optimization. Aside from the fact that this method has a negative impact on the quality of gradient computation, it is complicated to use for non-linear constraints. The second contribution of this thesis proposes a technique based on the exterior penalty method for handling general linear and non-linear constraints in reservoir optimization problems to improve gradient computation quality by the EnOpt method for efficient and improved optimization algorithm. Because of the computationally expensive NPV function due to the costly reservoir simulation of EOR methods, the solution method for the underlying EOR optimization problem becomes inefficient, especially for large reservoir problems. To speedup the overall computation of the solution method, this thesis introduces a novel full order model (FOM)-based certified adaptive machine learning optimization procedures to locally approximate the expensive NPV function. A supervised feedforward deep neural network (DNN) algorithm is employed to locally create surrogate model. In the FOM-based optimization algorithm of this study, several FOM NPV function evaluations are required by the EnOpt method to approximate the gradient function at each (outer) iteration until convergence. To limit the number FOM-based evaluations, we consider building surrogate models locally to replace the FOM based NPV function at each outer iteration and proceed with an inner optimization routine until convergence. We adapt the surrogate model using some FOM-based criterion where necessary until convergence. The demonstration of methodology for polymer optimization problem on a benchmark model results in an improved optimum and found to be more efficient compared to using the full order model optimization procedures