Particle swarm grammatical evolution for energy demand estimation

[EN] Grammatical Swarm is a search and optimization algorithm that belongs to the more general Grammatical Evolution family, which works with a set of solutions called individuals or particles. It uses the Particle Swarm Optimization algorithm as the search engine in the evolution of solutions. In this paper, we present a Grammatical Swarm algorithm for total energy demand estimation in a country from macroeconomic variables. Each particle in the Grammatical Swarm encodes a different model for energy demand estimation, which will be decoded by a predefined grammar. The parameters of the model are also optimized by the proposed algorithm, in such a way that the model is adjusted to a training set of real energy demand data, selecting the more appropriate variables to appear in the model. We analyze the performance of the Grammatical Swarm evolution in two real problems of one-year ahead energy demand estimation in Spain and France. The proposal is compared with previous approaches with competitive results.Spanish Ministerial Commission of Science and Technology (MICYT), Grant/Award Number: TIN2017-85887-C2-2-P; Ministerio de Ciencia, Innovacion y Universidades, Grant/Award Number: PGC2018-095322-B-C22 and RTI2018-095180-B-I00; Comunidad de Madrid y Fondos Estructurales de la Union Europea, Grant/Award Number: S2018/TCS-4566 and Y2018/NMT-4668; GenObIA-CM, Grant/Award Number: S2017/BMD-3773; Ministerio de Economia, Industria y Competitividad, Grant/Award Number: MTM2017-89664-PMartínez-Rodríguez, D.; Colmenar, JM.; Hidalgo, JI.; Villanueva Micó, RJ.; Salcedo-Sanz, S. (2020). Particle swarm grammatical evolution for energy demand estimation. Energy Science & Engineering. 8(4):1068-1079. https://doi.org/10.1002/ese3.568S1068107984Safarzyńska, K., & van den Bergh, J. C. J. M. (2017). Integrated crisis-energy policy: Macro-evolutionary modelling of technology, finance and energy interactions. Technological Forecasting and Social Change, 114, 119-137. doi:10.1016/j.techfore.2016.07.033Li, F., Song, Z., & Liu, W. (2014). China’s energy consumption under the global economic crisis: Decomposition and sectoral analysis. Energy Policy, 64, 193-202. doi:10.1016/j.enpol.2013.09.014Salcedo-Sanz, S., Muñoz-Bulnes, J., Portilla-Figueras, J. A., & Del Ser, J. (2015). One-year-ahead energy demand estimation from macroeconomic variables using computational intelligence algorithms. Energy Conversion and Management, 99, 62-71. doi:10.1016/j.enconman.2015.03.109Sánchez-Oro, J., Duarte, A., & Salcedo-Sanz, S. (2016). Robust total energy demand estimation with a hybrid Variable Neighborhood Search – Extreme Learning Machine algorithm. Energy Conversion and Management, 123, 445-452. doi:10.1016/j.enconman.2016.06.050Ceylan, H., & Ozturk, H. K. (2004). Estimating energy demand of Turkey based on economic indicators using genetic algorithm approach. Energy Conversion and Management, 45(15-16), 2525-2537. doi:10.1016/j.enconman.2003.11.010Shaik, S., & Yeboah, O.-A. (2018). Does climate influence energy demand? A regional analysis. Applied Energy, 212, 691-703. doi:10.1016/j.apenergy.2017.11.109United Nations Climate Change Conference.The Paris Agreement. UNTC XXVII 7.d.Suganthi, L., & Samuel, A. A. (2012). Energy models for demand forecasting—A review. Renewable and Sustainable Energy Reviews, 16(2), 1223-1240. doi:10.1016/j.rser.2011.08.014Geem, Z. W., & Roper, W. E. (2009). Energy demand estimation of South Korea using artificial neural network. Energy Policy, 37(10), 4049-4054. doi:10.1016/j.enpol.2009.04.049Ünler, A. (2008). Improvement of energy demand forecasts using swarm intelligence: The case of Turkey with projections to 2025. Energy Policy, 36(6), 1937-1944. doi:10.1016/j.enpol.2008.02.018Kıran, M. S., Özceylan, E., Gündüz, M., & Paksoy, T. (2012). Swarm intelligence approaches to estimate electricity energy demand in Turkey. Knowledge-Based Systems, 36, 93-103. doi:10.1016/j.knosys.2012.06.009Kıran, M. S., Özceylan, E., Gündüz, M., & Paksoy, T. (2012). A novel hybrid approach based on Particle Swarm Optimization and Ant Colony Algorithm to forecast energy demand of Turkey. Energy Conversion and Management, 53(1), 75-83. doi:10.1016/j.enconman.2011.08.004Yu, S., Wei, Y.-M., & Wang, K. (2012). A PSO–GA optimal model to estimate primary energy demand of China. Energy Policy, 42, 329-340. doi:10.1016/j.enpol.2011.11.090Yu, S., Zhu, K., & Zhang, X. (2012). Energy demand projection of China using a path-coefficient analysis and PSO–GA approach. Energy Conversion and Management, 53(1), 142-153. doi:10.1016/j.enconman.2011.08.015Yu, S., & Zhu, K. (2012). A hybrid procedure for energy demand forecasting in China. Energy, 37(1), 396-404. doi:10.1016/j.energy.2011.11.015Geng, Z., Zeng, R., Han, Y., Zhong, Y., & Fu, H. (2019). Energy efficiency evaluation and energy saving based on DEA integrated affinity propagation clustering: Case study of complex petrochemical industries. Energy, 179, 863-875. doi:10.1016/j.energy.2019.05.042Han, Y., Long, C., Geng, Z., Zhu, Q., & Zhong, Y. (2019). A novel DEACM integrating affinity propagation for performance evaluation and energy optimization modeling: Application to complex petrochemical industries. Energy Conversion and Management, 183, 349-359. doi:10.1016/j.enconman.2018.12.120Han, Y., Wu, H., Jia, M., Geng, Z., & Zhong, Y. (2019). Production capacity analysis and energy optimization of complex petrochemical industries using novel extreme learning machine integrating affinity propagation. Energy Conversion and Management, 180, 240-249. doi:10.1016/j.enconman.2018.11.001Colmenar, J. M., Hidalgo, J. I., & Salcedo-Sanz, S. (2018). Automatic generation of models for energy demand estimation using Grammatical Evolution. Energy, 164, 183-193. doi:10.1016/j.energy.2018.08.199O’Neill, M., & Brabazon, A. (2006). Grammatical Swarm: The generation of programs by social programming. Natural Computing, 5(4), 443-462. doi:10.1007/s11047-006-9007-7O’Neill, M., & Ryan, C. (2001). Grammatical evolution. IEEE Transactions on Evolutionary Computation, 5(4), 349-358. doi:10.1109/4235.942529KennedyJ EberhartR.Particle swarm optimization. Vol.4. Proceedings of ICNN'95 - International Conference on Neural Networks.Perth WA;1995:1942‐1948.https://doi.org/10.1109/ICNN.1995.488968Tsoulos, I. G., Gavrilis, D., & Glavas, E. (s. f.). Neural network construction using grammatical evolution. Proceedings of the Fifth IEEE International Symposium on Signal Processing and Information Technology, 2005. doi:10.1109/isspit.2005.1577206Beni, G., & Wang, J. (1993). Swarm Intelligence in Cellular Robotic Systems. Robots and Biological Systems: Towards a New Bionics?, 703-712. doi:10.1007/978-3-642-58069-7_38Krause, J., Ruxton, G. D., & Krause, S. (2010). Swarm intelligence in animals and humans. Trends in Ecology & Evolution, 25(1), 28-34. doi:10.1016/j.tree.2009.06.016Marini, F., & Walczak, B. (2015). Particle swarm optimization (PSO). A tutorial. Chemometrics and Intelligent Laboratory Systems, 149, 153-165. doi:10.1016/j.chemolab.2015.08.020Ling, C. X. (1995). Overfitting and generalization in learning discrete patterns. Neurocomputing, 8(3), 341-347. doi:10.1016/0925-2312(95)00050-

Towards the Evolution of Multi-Layered Neural Networks: A Dynamic Structured Grammatical Evolution Approach

Current grammar-based NeuroEvolution approaches have several shortcomings. On the one hand, they do not allow the generation of Artificial Neural Networks (ANNs composed of more than one hidden-layer. On the other, there is no way to evolve networks with more than one output neuron. To properly evolve ANNs with more than one hidden-layer and multiple output nodes there is the need to know the number of neurons available in previous layers. In this paper we introduce Dynamic Structured Grammatical Evolution (DSGE): a new genotypic representation that overcomes the aforementioned limitations. By enabling the creation of dynamic rules that specify the connection possibilities of each neuron, the methodology enables the evolution of multi-layered ANNs with more than one output neuron. Results in different classification problems show that DSGE evolves effective single and multi-layered ANNs, with a varying number of output neurons

The Optimal combination: Grammatical Swarm, Particle Swarm Optimization and Neural Networks.

Social behaviour is mainly based on swarm colonies, in which each individual shares its knowledge about the environment with other individuals to get optimal solutions. Such co-operative model differs from competitive models in the way that individuals die and are born by combining information of alive ones. This paper presents the particle swarm optimization with differential evolution algorithm in order to train a neural network instead the classic back propagation algorithm. The performance of a neural network for particular problems is critically dependant on the choice of the processing elements, the net architecture and the learning algorithm. This work is focused in the development of methods for the evolutionary design of artificial neural networks. This paper focuses in optimizing the topology and structure of connectivity for these networks

Self-Organizing Architectural design based on Morphogenetic Programming.

In this paper, we present our research into self-organizing building algorithms. This idea of self-organization of animal/plants behaviour interests researchers to explore the mechanisms required for this emergent phenomena and try to apply them in other domains. We were able to implement a typical construction algorithm in a 3D simulation environment and reproduce the results of previous research in the area. LSystems, morphogenetic programming and wasp nest building are explained in order to understand self-organizing models. We proposed Grammatical swarm as a good tool to optimize building structures

Novelty grammar swarms

Tese de mestrado, Engenharia Informática (Sistemas de Informação), Universidade de Lisboa, Faculdade de Ciências, 2015Particle Swarm Optimization (PSO) é um dos métodos de optimização populacionais mais conhecido. Normalmente é aplicado na otimização funções de fitness, que indicam o quão perto o algoritmo está de atingir o objectivo da pesquisa, fazendo com que esta se foque em áreas de fitness mais elevado. Em problemas com muitos ótimos locais, regularmente a pesquisa fica presa em locais com fitness elevado mas que não são o verdadeiro objetivo. Com vista a solucionar este problema em certos domínios, nesta tese é introduzido o Novelty-driven Particle Swarm Optimization (NdPSO). Este algoritmo é inspirado na pesquisa pela novidade (novelty search), um método relativamente recente que guia a pesquisa de forma a encontrar instâncias significativamente diferentes das anteriores. Desta forma, o NdPSO ignora por completo o objetivo perseguindo apenas a novidade, isto torna-o menos susceptivel a ser enganado em problemas com muitos optimos locais. Uma vez que o novelty search mostrou potencial a resolver tarefas no âmbito da programação genética, em particular na evolução gramatical, neste projeto o NdPSO é usado como uma extensão do método de Grammatical Swarm que é uma combinação do PSO com a programação genética. A implementação do NdPSO é testada em três domínios diferentes, representativos daqueles para o qual este algoritmo poderá ser mais vantajoso que os algoritmos guiados pelo objectivo. Isto é, domínios enganadores nos quais seja relativamente intuitivo descrever um comportamento. Em cada um dos domínios testados, o NdPSO supera o aloritmo standard do PSO, uma das suas variantes mais conhecidas (Barebones PSO) e a pesquisa aleatória, mostrando ser uma ferramenta promissora para resolver problemas enganadores. Uma vez que esta é a primeira aplicação da pesquisa por novidade fora do paradigma evolucionário, neste projecto é também efectuado um estudo comparativo do novo algoritmo com a forma mais comum de usar a pesquisa pela novidade (na forma de algoritmo evolucionário).Particle Swarm Optimization (PSO) is a well-known population-based optimization algorithm. Most often it is applied to optimize fitness functions that specify the goal of reaching a desired objective or behavior. As a result, search focuses on higher-fitness areas. In problems with many local optima, search often becomes stuck, and thus can fail to find the intended objective. To remedy this problem in certain kinds of domains, this thesis introduces Novelty-driven Particle Swarm Optimization (NdPSO). Taking motivation from the novelty search algorithm in evolutionary computation, in this method search is driven only towards finding instances significantly different from those found before. In this way, NdPSO completely ignores the objective in its pursuit of novelty, making it less susceptible to deception and local optima. Because novelty search has previously shown potential for solving tasks in Genetic Programming, particularly, in Grammatical Evolution, this paper implements NdPSO as an extension of the Grammatical Swarm method which in effect is a combination of PSO and Genetic Programming.The resulting NdPSO implementation was tested in three different domains representative of those in which it might provide advantage over objective-driven PSO, in particular, those which are deceptive and in which a meaningful high-level description of novel behavior is easy to derive. In each of the tested domains NdPSO outperforms both objective-based PSO and random-search, demonstrating its promise as a tool for solving deceptive problems. Since this is the first application of the search for novelty outside the evolutionary paradigm an empirical comparative study of the new algorithm to a standard novelty search Evolutionary Algorithm is performed

Derivation of Context-free Stochastic L-Grammar Rules for Promoter Sequence Modeling Using Support Vector Machine

Formal grammars can used for describing complex repeatable structures such as DNA sequences. In this paper, we describe the structural composition of DNA sequences using a context-free stochastic L-grammar. L-grammars are a special class of parallel grammars that can model the growth of living organisms, e.g. plant development, and model the morphology of a variety of organisms. We believe that parallel grammars also can be used for modeling genetic mechanisms and sequences such as promoters. Promoters are short regulatory DNA sequences located upstream of a gene. Detection of promoters in DNA sequences is important for successful gene prediction. Promoters can be recognized by certain patterns that are conserved within a species, but there are many exceptions which makes the promoter recognition a complex problem. We replace the problem of promoter recognition by induction of context-free stochastic L-grammar rules, which are later used for the structural analysis of promoter sequences. L-grammar rules are derived automatically from the drosophila and vertebrate promoter datasets using a genetic programming technique and their fitness is evaluated using a Support Vector Machine (SVM) classifier. The artificial promoter sequences generated using the derived L- grammar rules are analyzed and compared with natural promoter sequences

An adaptive neuro-fuzzy propagation model for LoRaWAN

This article proposes an adaptive-network-based fuzzy inference system (ANFIS) model for accurate estimation of signal propagation using LoRaWAN. By using ANFIS, the basic knowledge of propagation is embedded into the proposed model. This reduces the training complexity of artificial neural network (ANN)-based models. Therefore, the size of the training dataset is reduced by 70% compared to an ANN model. The proposed model consists of an efficient clustering method to identify the optimum number of the fuzzy nodes to avoid overfitting, and a hybrid training algorithm to train and optimize the ANFIS parameters. Finally, the proposed model is benchmarked with extensive practical data, where superior accuracy is achieved compared to deterministic models, and better generalization is attained compared to ANN models. The proposed model outperforms the nondeterministic models in terms of accuracy, has the flexibility to account for new modeling parameters, is easier to use as it does not require a model for propagation environment, is resistant to data collection inaccuracies and uncertain environmental information, has excellent generalization capability, and features a knowledge-based implementation that alleviates the training process. This work will facilitate network planning and propagation prediction in complex scenarios