Boosting wavelet neural networks using evolutionary algorithms for short-term wind speed time series forecasting

This paper addresses nonlinear time series modelling and prediction problem using a type of wavelet neural networks. The basic building block of the neural network models is a ridge type function. The training of such a network is a nonlinear optimization problem. Evolutionary algorithms (EAs), including genetic algorithm (GA) and particle swarm optimization (PSO), together with a new gradient-free algorithm (called coordinate dictionary search optimization – CDSO), are used to train network models. An example for real speed wind data modelling and prediction is provided to show the performance of the proposed networks trained by these three optimization algorithms

Sequential Training of Neural Networks with Gradient Boosting

This paper presents a novel technique based on gradient boosting to train a shallow neural network (NN). Gradient boosting is an additive expansion algorithm in which a series of models are trained sequentially to approximate a given function. A neural network can also be seen as an additive model where the scalar product of the responses of the last hidden layer and its weights provide the final output of the network. Instead of training the network as a whole, the proposed algorithm trains the network sequentially in $T$ steps. First, the bias term of the network is initialized with a constant approximation that minimizes the average loss of the data. Then, at each step, a portion of the network, composed of $J$ neurons, is trained to approximate the pseudo-residuals on the training data computed from the previous iterations. Finally, the $T$ partial models and bias are integrated as a single NN with $T \times J$ neurons in the hidden layer. Extensive experiments in classification and regression tasks are carried out showing a competitive generalization performance with respect to neural networks trained with different standard solvers, such as Adam, L-BFGS and SGD. Furthermore, we show that the proposed method design permits to switch off a number of hidden units during test (the units that were last trained) without a significant reduction of its generalization ability. This permits the adaptation of the model to different classification speed requirements on the fly

A Comparative Analysis of XGBoost

XGBoost is a scalable ensemble technique based on gradient boosting that has demonstrated to be a reliable and efficient machine learning challenge solver. This work proposes a practical analysis of how this novel technique works in terms of training speed, generalization performance and parameter setup. In addition, a comprehensive comparison between XGBoost, random forests and gradient boosting has been performed using carefully tuned models as well as using the default settings. The results of this comparison may indicate that XGBoost is not necessarily the best choice under all circumstances. Finally an extensive analysis of XGBoost parametrization tuning process is carried out

Deep Convolutional Neural Network Ensembles using ECOC

Deep neural networks have enhanced the performance of decision making systems in many applications including image understanding, and further gains can be achieved by constructing ensembles. However, designing an ensemble of deep networks is often not very beneficial since the time needed to train the networks is very high or the performance gain obtained is not very significant. In this paper, we analyse error correcting output coding (ECOC) framework to be used as an ensemble technique for deep networks and propose different design strategies to address the accuracy-complexity trade-off. We carry out an extensive comparative study between the introduced ECOC designs and the state-of-the-art ensemble techniques such as ensemble averaging and gradient boosting decision trees. Furthermore, we propose a combinatory technique which is shown to achieve the highest classification performance amongst all.Comment: 13 pages double column IEEE transactions styl

An Overview of Carbon Footprint Mitigation Strategies. Machine Learning for Societal Improvement, Modernization, and Progress

Among the most pressing issues in the world today is the impact of globalization and energy consumption on the environment. Despite the growing regulatory framework to prevent ecological degradation, sustainability continues to be a problem. Machine learning can help with the transition toward a net-zero carbon society. Substantial work has been done in this direction. Changing electrical systems, transportation, buildings, industry, and land use are all necessary to reduce greenhouse gas emissions. Considering the carbon footprint aspect of sustainability, this chapter provides a detailed overview of how machine learning can be applied to forge a path to ecological sustainability in each of these areas. The chapter highlights how various machine learning algorithms are used to increase the use of renewable energy, efficient transportation, and waste management systems to reduce the carbon footprint. The authors summarize the findings from the current research literature and conclude by providing a few future directions

Linear and nonlinear Model Predictive Control (MPC) for regulating pedestrian flows with discrete speed instructions

Airports, shopping malls, stadiums, and large venues in general, can become congested and chaotic at peak times or in emergency situations. Linear Model Predictive Control (MPC) is an effective technology in generating dynamic speed or distance instructions for regulating pedestrian flows, and constitutes a promising interventional technique to improve safety and evacuation time during emergency egress operations. We compare linear and nonlinear MPC controllers and study the influence of using continuous vs. discrete control actions. We aim to evaluate the efficacy of simple instructions that pedestrians can easily follow during evacuations. Linear and Nonlinear AutoRegressive eXogenous models (ARX and NLARX) for prediction are identified from input?output data from strategically designed microscopic evacuation simulations. A microscopic simulation framework is used to design and validate different MPC controllers tuned and refined using the identified models. We evaluate the prediction models? performance and study how the controlled variable type, density, or crowd-pressure, influences the controllers? performance. As a relevant contribution, we show that MPC control with discrete instructions is ideally suited to design and deploy practical pedestrian flow control systems. We found that an adequate size of the set of speed instructions is critical to obtain a good balance between controllability and performance, and that density output control is preferred over crowd-pressure.Universidad de Alcal

A Life-Long Learning XAI Metaheuristic-based Type-2 Fuzzy System for Solar Radiation Modelling

Solar photovoltaic (PV) power generation is one of the most important sources for renewable energy. However, PV power generation is entirely dependent on the amount of downward solar radiation reaching the solar cells. This is determined by uncertain and uncontrollable meteorological factors such as temperature, humidity, wind speed, and direction, as well as other factors such as topographical characteristics. Good solar radiation prediction models can increase energy output while decreasing the operation costs of photovoltaic power generation. For example, in some provinces in China, PV stations are required to upload short-term online power forecast information to power dispatching agencies. Numerous AI, statistical, and numerical weather prediction models have been used in many real-world renewable energy applications, with a focus on modeling accuracy. However, there is a need for Explainable AI (XAI) models that could be easily understood, analyzed, and augmented by the stakeholders. In this paper, we present a compact, explainable, and lifelong learning metaheuristic-based Interval Type-2 Fuzzy Logic System (IT2FLS) for Solar Radiation Modeling. The generated model will be composed of a small number of short IF-Then rules that have been optimized via simulated annealing to produce models with high prediction accuracy. These models are updated through a life-long learning approach to maximize their accuracy and maintain interpretability. In the process of lifelong learning, the proposed method transferred the model's knowledge to new geographical locations with minimal forgetting. The proposed method achieved good prediction accuracy and outperformed on new geographical locations other transparent and black-box models by 13.2% as well as maintaining excellent generalization ability. The resulting models have been evaluated and accepted by experts, and thanks to the generated transparency, the experts were able to augment the models with their expertise, which increased the models' accuracy

Hydrostatic Performance Of Reinforced Concrete Pipe For Infiltration

Groundwater infiltration into underground sewer systems has long been a costly issue for municipalities. With reinforced concrete pipe (RCP) being a primary option for sewer systems, existing hydrostatic testing methods conducted by manufacturers to measure internal pipe pressure, as required by specifications, do not reflect in-situ external hydrostatic conditions. This thesis records the development of a novel testing method to evaluate the RCP joint performance for infiltration. The test is safe and easy to conduct by RCP producers at the factory. The test method mimics field conditions of possible RCP joint gap and joint offset. Over 100 tests were conducted, including 600 mm, 900 mm and 1200 mm RCP with conventional single offset self-lubricated gaskets. This study also evaluates the gasket performance for infiltration. Pipe joint performance curves were developed based on the test results. Comparison to laboratory load deformation tests on gaskets were conducted, indicating that predictions of the sealing potential derived using gasket geometry agreed well with results of infiltration tests. The study shows that the joint gap plays an important role in the sealing potential. The developed apparatus allows the observation of gasket movement under infiltration pressure against the gasket leading to failure. The performance curves also allow the prediction of an infiltration potential leading to a practical applicational procedure to guide RCP installation. A case study of deep RCP pipe subjected to groundwater pressure illustrated the usefulness of the performance curves to derive maximum allowable joint gaps, which contractors could rely on during RCP installation. The findings should allow deducing technical guidance on how water tightness of RCP can be achieved at installation below the prevailing groundwater level. Two oversampling methods: Synthetic Minority Over-sampling Technique (SMOTE) and Density-Based SMOTE were employed to address the unbalanced dataset. Accordingly, applying advanced machine learning techniques, the scale of variation in the test data can be analyzed and accurately predicted using tree-based supervised classification methods: random forest, extra trees and gradient boosting

Control and management of energy storage systems in microgrids

The rate of integration of the renewable energy sources in modern grids have significantly increased in the last decade. These intermittent, non-dispatchable renewable sources, though environment friendly tend to be grid unfriendly. This is precisely due to the issues pertaining to grid congestion, voltage regulation and stability of grids being reported as a result of the incorporation of renewable sources. In this scenario, the use of energy storage systems (ESS ) in electric grids is being widely proposed to overcome these issues. However, integrating energy storage systems alone will not compensate for the issue created by renewable generation. The control and management of the ESS should be done optimally so that their full capabilities are exploited to overcome the issues in the power grids and to ensure their lower cost of investment by prolonging ESS lifetime through minimising degradation. Motivated by this aspect this Ph.D work focusses on developing an efficient, optimal control and management strategy for ESS in a microgrid, especially hybrid ESS. The Ph.D work addresses this issue by proposing a hierarchical control scheme comprising of a lower power management and higher energy management stage with contributions in each stage. In the power management stage this work focusses on improving aspects of real time control of power converters interfacing ESS to grid and the microgrid system as whole. The work proposes control systems with improved dynamic behaviour for power converters based on the reset control framework. In the microgrid control the work presents a primary+secondary control scheme with improved voltage regulation performance under disturbances, using an observer. The real time power splitting strategies among hybrid ESS accounting for the ESS operating efficiencies and degradation mechanisms will also be addressed in the primary+secondary control of power management stage. The design criteria, stability and robustness analysis will be carried out, along with simulation or experimental verifications. In the higher level energy management stage, the contribution of this work involves application of an economic MPC framework for the management of ESS in microgrids. The work specifically addresses the problems of mitigating grid congestion from renewable power feed-in, minimising ESS degradation and maximising self consumption of generated renewable energy using the MPC based energy management system. A survey of the forecasting methods that can be used for MPC will be carried out and a neural network based forecasting unit for time series prediction will be developed. The practical issue of accounting for forecasting error in the decision making of MPC will be addressed and impact of the resulting conservative decision making on the system performance will be analysed. The improvement in performance with the proposed energy management scheme will be demonstrated and quantified.La integración de las fuentes de energía renovables en las redes modernas ha aumentado significativamente en la última década. Estas fuentes renovables, aunque muy convenientes para el medio ambiente son de naturaleza intermitente, y son no panificables, cosa que genera problemas en la red de distribución. Esto se debe precisamente a los problemas relacionados con la congestión de la red y la regulación del voltaje. En este escenario, el uso de sistemas de almacenamiento de energía (ESS) en redes eléctricas está siendo ampliamente propuesto para superar estos problemas. Sin embargo, la integración de sistemas de almacenamiento de energía por sí solos no compensará el problema creado por la generación renovable. El control y la gestión del ESS deben realizarse de manera óptima, de modo que se aprovechen al máximo sus capacidades para superar los problemas en las redes eléctricas, garantizar un coste de inversión razonable y prolongar la vida útil del ESS minimizando su degradación. Motivado por esta problemática, esta tesis doctoral se centra en desarrollar una estrategia de control y gestión eficiente para los ESS integrados en una microrred, especialmente cuando se trata de ESS de naturaleza. El trabajo de doctorado propone un esquema de control jerárquico compuesto por un control de bajo nivel y una parte de gestión de energía operando a más alto nivel. El trabajo realiza aportaciones en los dos campos. En el control de bajo nivel, este trabajo se centra en mejorar aspectos del control en tiempo real de los convertidores que interconectan el ESS con la red y el sistema de micro red en su conjunto. El trabajo propone sistemas de control con comportamiento dinámico mejorado para convertidores de potencia desarrollados en el marco del control de tipo reset. En el control de microrred, el trabajo presenta un esquema de control primario y uno secundario con un rendimiento de regulación de voltaje mejorado bajo perturbaciones, utilizando un observador. Además, el trabajo plantea estrategias de reparto del flujo de potencia entre los diferentes ESS. Durante el diseño de estos algoritmos de control se tienen en cuenta los mecanismos de degradación de los diferentes ESS. Los algoritmos diseñados se validarán mediante simulaciones y trabajos experimentales. En el apartado de gestión de energía, la contribución de este trabajo se centra en la aplicación del un control predictivo económico basado en modelo (EMPC) para la gestión de ESS en microrredes. El trabajo aborda específicamente los problemas de mitigar la congestión de la red a partir de la alimentación de energía renovable, minimizando la degradación de ESS y maximizando el autoconsumo de energía renovable generada. Se ha realizado una revisión de los métodos de predicción del consumo/generación que pueden usarse en el marco del EMPC y se ha desarrollado un mecanismo de predicción basado en el uso de las redes neuronales. Se ha abordado el análisis del efecto del error de predicción sobre el EMPC y el impacto que la toma de decisiones conservadoras produce en el rendimiento del sistema. La mejora en el rendimiento del esquema de gestión energética propuesto se ha cuantificado.La integració de les fonts d'energia renovables a les xarxes modernes ha augmentat significativament en l’última dècada. Aquestes fonts renovables, encara que molt convenients per al medi ambient són de naturalesa intermitent, i són no panificables, cosa que genera problemes a la xarxa de distribució. Això es deu precisament als problemes relacionats amb la congestió de la xarxa i la regulació de la tensió. En aquest escenari, l’ús de sistemes d'emmagatzematge d'energia (ESS) en xarxes elèctriques està sent àmpliament proposat per superar aquests problemes. No obstant això, la integració de sistemes d'emmagatzematge d'energia per si sols no compensarà el problema creat per la generació renovable. El control i la gestió de l'ESS s'han de fer de manera _optima, de manera que s'aprofitin al màxim les seves capacitats per superar els problemes en les xarxes elèctriques, garantir un cost d’inversió raonable i allargar la vida útil de l'ESS minimitzant la seva degradació. Motivat per aquesta problemàtica, aquesta tesi doctoral es centra a desenvolupar una estratègia de control i gestió eficient per als ESS integrats en una microxarxa, especialment quan es tracta d'ESS de natura híbrida. El treball de doctorat proposa un esquema de control jeràrquic compost per un control de baix nivell i una part de gestió d'energia operant a més alt nivell. El treball realitza aportacions en els dos camps. En el control de baix nivell, aquest treball es centra a millorar aspectes del control en temps real dels convertidors que interconnecten el ESS amb la xarxa i el sistema de microxarxa en el seu conjunt. El treball proposa sistemes de control amb comportament dinàmic millorat per a convertidors de potència desenvolupats en el marc del control de tipus reset. En el control de micro-xarxa, el treball presenta un esquema de control primari i un de secundari de regulació de voltatge millorat sota pertorbacions, utilitzant un observador. A més, el treball planteja estratègies de repartiment de el flux de potència entre els diferents ESS. Durant el disseny d'aquests algoritmes de control es tenen en compte els mecanismes de degradació dels diferents ESS. Els algoritmes dissenyats es validaran mitjanant simulacions i treballs experimentals. En l'apartat de gestió d'energia, la contribució d'aquest treball se centra en l’aplicació de l'un control predictiu econòmic basat en model (EMPC) per a la gestió d'ESS en microxarxes. El treball aborda específicament els problemes de mitigar la congestió de la xarxa a partir de l’alimentació d'energia renovable, minimitzant la degradació d'ESS i maximitzant l'autoconsum d'energia renovable generada. S'ha realitzat una revisió dels mètodes de predicció del consum/generació que poden usar-se en el marc de l'EMPC i s'ha desenvolupat un mecanisme de predicció basat en l’ús de les xarxes neuronals. S'ha abordat l’anàlisi de l'efecte de l'error de predicció sobre el EMPC i l'impacte que la presa de decisions conservadores produeix en el rendiment de el sistema. La millora en el rendiment de l'esquema de gestió energètica proposat s'ha quantificat