Time series prediction and forecasting using Deep learning Architectures

Nature brings time series data everyday and everywhere, for example, weather data, physiological signals and biomedical signals, financial and business recordings. Predicting the future observations of a collected sequence of historical observations is called time series forecasting. Forecasts are essential, considering the fact that they guide decisions in many areas of scientific, industrial and economic activity such as in meteorology, telecommunication, finance, sales and stock exchange rates. A massive amount of research has already been carried out by researchers over many years for the development of models to improve the time series forecasting accuracy. The major aim of time series modelling is to scrupulously examine the past observation of time series and to develop an appropriate model which elucidate the inherent behaviour and pattern existing in time series. The behaviour and pattern related to various time series may possess different conventions and infact requires specific countermeasures for modelling. Consequently, retaining the neural networks to predict a set of time series of mysterious domain remains particularly challenging. Time series forecasting remains an arduous problem despite the fact that there is substantial improvement in machine learning approaches. This usually happens due to some factors like, different time series may have different flattering behaviour. In real world time series data, the discriminative patterns residing in the time series are often distorted by random noise and affected by high-frequency perturbations. The major aim of this thesis is to contribute to the study and expansion of time series prediction and multistep ahead forecasting method based on deep learning algorithms. Time series forecasting using deep learning models is still in infancy as compared to other research areas for time series forecasting.Variety of time series data has been considered in this research. We explored several deep learning architectures on the sequential data, such as Deep Belief Networks (DBNs), Stacked AutoEncoders (SAEs), Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs). Moreover, we also proposed two different new methods based on muli-step ahead forecasting for time series data. The comparison with state of the art methods is also exhibited. The research work conducted in this thesis makes theoretical, methodological and empirical contributions to time series prediction and multi-step ahead forecasting by using Deep Learning Architectures

A new approach to seasonal energy consumption forecasting using temporal convolutional networks

There has been a significant increase in the attention paid to resource management in smart grids, and several energy forecasting models have been published in the literature. It is well known that energy forecasting plays a crucial role in several applications in smart grids, including demand-side management, optimum dispatch, and load shedding. A significant challenge in smart grid models is managing forecasts efficiently while ensuring the slightest feasible prediction error. A type of artificial neural networks such as recurrent neural networks, are frequently used to forecast time series data. However, due to certain limitations like vanishing gradients and lack of memory retention of recurrent neural networks, sequential data should be modeled using convolutional networks. The reason is that they have strong capabilities to solve complex problems better than recurrent neural networks. In this research, a temporal convolutional network is proposed to handle seasonal short-term energy forecasting. The proposed temporal convolutional network computes outputs in parallel, reducing the computation time compared to the recurrent neural networks. Further performance comparison with the traditional long short-term memory in terms of MAD and sMAPE has proved that the proposed model has outperformed the recurrent neural network

Alternative Sources of Energy Modeling, Automation, Optimal Planning and Operation

An economic development model analyzes the adoption of alternative strategy capable of leveraging the economy, based essentially on RES. The combination of wind turbine, PV installation with new technology battery energy storage, DSM network and RES forecasting algorithms maximizes RES integration in isolated islands. An innovative model of power system (PS) imbalances is presented, which aims to capture various features of the stochastic behavior of imbalances and to reduce in average reserve requirements and PS risk. Deep learning techniques for medium-term wind speed and solar irradiance forecasting are presented, using for first time a specific cloud index. Scalability-replicability of the FLEXITRANSTORE technology innovations integrates hardware-software solutions in all areas of the transmission system and the wholesale markets, promoting increased RES. A deep learning and GIS approach are combined for the optimal positioning of wave energy converters. An innovative methodology to hybridize battery-based energy storage using supercapacitors for smoother power profile, a new control scheme and battery degradation mechanism and their economic viability are presented. An innovative module-level photovoltaic (PV) architecture in parallel configuration is introduced maximizing power extraction under partial shading. A new method for detecting demagnetization faults in axial flux permanent magnet synchronous wind generators is presented. The stochastic operating temperature (OT) optimization integrated with Markov Chain simulation ascertains a more accurate OT for guiding the coal gasification practice

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Reservoir computing (RC), first applied to temporal signal processing, is a recurrent neural network in which neurons are randomly connected. Once initialized, the connection strengths remain unchanged. Such a simple structure turns RC into a non-linear dynamical system that maps low-dimensional inputs into a high-dimensional space. The model's rich dynamics, linear separability, and memory capacity then enable a simple linear readout to generate adequate responses for various applications. RC spans areas far beyond machine learning, since it has been shown that the complex dynamics can be realized in various physical hardware implementations and biological devices. This yields greater flexibility and shorter computation time. Moreover, the neuronal responses triggered by the model's dynamics shed light on understanding brain mechanisms that also exploit similar dynamical processes. While the literature on RC is vast and fragmented, here we conduct a unified review of RC's recent developments from machine learning to physics, biology, and neuroscience. We first review the early RC models, and then survey the state-of-the-art models and their applications. We further introduce studies on modeling the brain's mechanisms by RC. Finally, we offer new perspectives on RC development, including reservoir design, coding frameworks unification, physical RC implementations, and interaction between RC, cognitive neuroscience and evolution.Comment: 51 pages, 19 figures, IEEE Acces

Demand forecasting for fast-moving products in grocery retail

Demand forecasting is a critically important task in grocery retail. Accurate forecasts allow the retail companies to reduce their product spoilage, as well as maximize their profits. Fast-moving products, or products with a lot of sales and fast turnover, are particularly important to forecast accurately due to their high sales volumes. We investigate dynamic harmonic regression, Poisson GLM with elastic net, MLP and two-layer LSTM in fast-moving product demand forecasting against the naive seasonal forecasting baseline. We evaluate two modes of seasonality modelling in neural networks: Fourier series against seasonal decomposition. We specify the full procedure for comparing forecasting models in a collection of product-location sales time series, involving two-stage cross-validation, and careful hyperparameter selection. We use Halton sequences for neural network hyperparameter selection. We evaluate the model results in demand forecasting using hypothesis testing, bootstrapping, and rank comparison methods. The experimental results suggest that the dynamic harmonic regression produces superior results in comparison to Poisson GLM, MLP and two-layer LSTM models for demand forecasting in fast-moving products with long sales histories. We additionally show that deseasonalization results in better forecasts in comparison to Fourier seasonality modelling in neural networks

Photovoltaic forecasting with artificil neural networks

Tese de mestrado em Engenharia da Energia e do Ambiente, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2014São necessários esforços adicionais para promover a utilização de sistemas de produção de energia fotovoltaica conectados à rede como uma fonte fundamental de sistemas de energia elétrica, em níveis de penetrações mais elevados. Nesta tese é abordada a variabilidade da geração elétrica por sistemas fotovoltaicos e é desenvolvida com base na premissa de que o desempenho e a gestão de pequenas redes elétricas podem ser melhorados quando são utilizadas as informações de previsão de energia solar. É implementado um sistema de arquitetura de rede neuronal para o modelo auto-regressivo não-linear com variáveis exógenas (NARX) utilizando, não só, dados meteorológicos locais, mas também medições de sistemas fotovoltaicos circunjacentes. Diferentes configurações de entrada são otimizadas e comparadas para avaliar os efeitos no desempenho do modelo para previsão. A precisão das previsões revelou melhoria quando lhe são adicionadas informações de sistemas fotovoltaicos circunjacentes. Após ser selecionada a configuração de entrada da rede com o melhor desempenho, são testadas previsões com várias horas de antecedência e comparadas com o modelo da persistência, para verificar a precisão do modelo na previsão de diferentes horizontes temporais de curto prazo. O modelo NARX superou, claramente, o modelo de persistência, resultando num RMSE de 3,7% e de 4,5% aquando da antecipação das previsões de 5min e 2h30min, respetivamente.Additional efforts are required to promote the use of grid-connected photovoltaic (PV) systems as a fundamental source in electric power systems at the higher penetration levels. This thesis addresses the variability of PV electric generation and is built based on the premise that the performance and management of small electric networks can be improved when solar power forecast information is used. A neural network architecture system for the Nonlinear Autoregressive with eXogenous inputs (NARX) model is implemented using not only local meteorological data but also measurements of neighbouring PV systems. Input configurations are optimized and compared to assess the effects in the model forecasting performance. The added value of the information of the neighbouring PV systems has demonstrated to further improve the prediction accuracy. After selecting the input configuration with the best network performance, forecasts up to several hours in advance are tested to verify the model forecasting accuracy for different short-term time horizons and compared with the persistence model. The NARX model clearly outperformed the persistence model and yielded a 3.7% and a 4.5% RMSE for the anticipation of the 5min and 2h30 forecasts, respectively

Prediction of Splitting Tensile Strength of Self-Compacting Recycled Aggregate Concrete Using Novel Deep Learning Methods

[EN] The composition of self-compacting concrete (SCC) contains 60–70% coarse and fine aggregates, which are replaced by construction waste, such as recycled aggregates (RA). However, the complexity of its structure requires a time-consuming mixed design. Currently, many researchers are studying the prediction of concrete properties using soft computing techniques, which will eventually reduce environmental degradation and other material waste. There have been very limited and contradicting studies regarding prediction using different ANN algorithms. This paper aimed to predict the 28-day splitting tensile strength of SCC with RA using the artificial neural network technique by comparing the following algorithms: Levenberg–Marquardt (LM), Bayesian regularization (BR), and Scaled Conjugate Gradient Backpropagation (SCGB). There have been very limited and contradicting studies regarding prediction by using and comparing different ANN algorithms, so a total of 381 samples were collected from various published journals. The input variables were cement, admixture, water, fine and coarse aggregates, and superplasticizer; the data were randomly divided into three sets—training (60%), validation (10%), and testing (30%)—with 10 neurons in the hidden layer. The models were evaluated by the mean squared error (MSE) and correlation coefficient (R). The results indicated that all three models have optimal accuracy; still, BR gave the best performance (R = 0.91 and MSE = 0.2087) compared with LM and SCG. BR was the best model for predicting TS at 28 days for SCC with RA. The sensitivity analysis indicated that cement (30.07%) was the variable that contributed the most to the prediction of TS at 28 days for SCC with RA, and water (2.39%) contributed the least.S