Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction

This paper reports the results of the NN3 competition, which is a replication of the M3 competition with an extension of the competition towards neural network (NN) and computational intelligence (CI) methods, in order to assess what progress has been made in the 10 years since the M3 competition. Two masked subsets of the M3 monthly industry data, containing 111 and 11 empirical time series respectively, were chosen, controlling for multiple data conditions of time series length (short/long), data patterns (seasonal/non-seasonal) and forecasting horizons (short/medium/long). The relative forecasting accuracy was assessed using the metrics from the M3, together with later extensions of scaled measures, and non-parametric statistical tests. The NN3 competition attracted 59 submissions from NN, CI and statistics, making it the largest CI competition on time series data. Its main findings include: (a) only one NN outperformed the damped trend using the sMAPE, but more contenders outperformed the AutomatANN of the M3; (b) ensembles of CI approaches performed very well, better than combinations of statistical methods; (c) a novel, complex statistical method outperformed all statistical and Cl benchmarks; and (d) for the most difficult subset of short and seasonal series, a methodology employing echo state neural networks outperformed all others. The NN3 results highlight the ability of NN to handle complex data, including short and seasonal time series, beyond prior expectations, and thus identify multiple avenues for future research. (C) 2011 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved

Training Echo State Networks with Regularization through Dimensionality Reduction

In this paper we introduce a new framework to train an Echo State Network to predict real valued time-series. The method consists in projecting the output of the internal layer of the network on a space with lower dimensionality, before training the output layer to learn the target task. Notably, we enforce a regularization constraint that leads to better generalization capabilities. We evaluate the performances of our approach on several benchmark tests, using different techniques to train the readout of the network, achieving superior predictive performance when using the proposed framework. Finally, we provide an insight on the effectiveness of the implemented mechanics through a visualization of the trajectory in the phase space and relying on the methodologies of nonlinear time-series analysis. By applying our method on well known chaotic systems, we provide evidence that the lower dimensional embedding retains the dynamical properties of the underlying system better than the full-dimensional internal states of the network

DEEP RECURRENT NEURAL NETWORKS IN ENERGY DEMAND FORECASTING: A CASE STUDY OF KAZAKHSTAN'S ELECTRICAL CONSUMPTION

The critical transformation of the energy sector demands innovative approaches to ensure the reliability and efficiency of energy systems. In this pursuit, this study delved into the potential of Deep Recurrent Neural Networks (DRNNs) for forecasting energy demand, using a comprehensive dataset detailing Kazakhstan's electrical consumption over a span of two years. Traditional statistical models have historically played a role in energy demand prediction, but the growing intricacy of the energy landscape calls for more advanced solutions. The paper presented a comparison of the DRNN with other traditional and machine learning models and highlighted the superior performance of DRNNs, especially in capturing complex temporal relationships. The energy sector is confronting unprecedented challenges due to population growth and the integration of diverse energy sources, leading to increased demand and system strains. Accurate energy demand prediction is essential for system reliability. Traditional models, though widely used, often overlook intricate variables like weather patterns and temporal factors. Through rigorous methodology, encompassing exploratory data analysis, feature engineering, and hyperparameter optimization, an optimized DRNN model was developed. The results demonstrated the DRNN's exceptional capability in processing complex time-series data, as evidenced by its attainment of an R-squared value of 83.6%. Additionally, it achieved Mean Absolute Errors and Root Mean Squared Errors of less than 2%. However, there were noticeable deviations in some predictions, suggesting areas for refinement. This research underscores the significance of DRNNs in energy demand prediction, highlighting their advantages over traditional models while also noting the need for ongoing optimization. The findings underscore DRNN's promise as a robust forecasting tool, pivotal for the energy sector's future resilience and efficiency