Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction

This paper reports the results of the NN3 competition, which is a replication of the M3 competition with an extension of the competition towards neural network (NN) and computational intelligence (CI) methods, in order to assess what progress has been made in the 10 years since the M3 competition. Two masked subsets of the M3 monthly industry data, containing 111 and 11 empirical time series respectively, were chosen, controlling for multiple data conditions of time series length (short/long), data patterns (seasonal/non-seasonal) and forecasting horizons (short/medium/long). The relative forecasting accuracy was assessed using the metrics from the M3, together with later extensions of scaled measures, and non-parametric statistical tests. The NN3 competition attracted 59 submissions from NN, CI and statistics, making it the largest CI competition on time series data. Its main findings include: (a) only one NN outperformed the damped trend using the sMAPE, but more contenders outperformed the AutomatANN of the M3; (b) ensembles of CI approaches performed very well, better than combinations of statistical methods; (c) a novel, complex statistical method outperformed all statistical and Cl benchmarks; and (d) for the most difficult subset of short and seasonal series, a methodology employing echo state neural networks outperformed all others. The NN3 results highlight the ability of NN to handle complex data, including short and seasonal time series, beyond prior expectations, and thus identify multiple avenues for future research. (C) 2011 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved

A study on the ability of support vector regression and neural networks to forecast basic time series patterns

Recently, novel learning algorithms such as Support Vector Regression (SVR) and Neural Networks (NN) have received increasing attention in forecasting and time series prediction, offering attractive theoretical properties and successful applications in several real world problem domains. Commonly, time series are composed of the combination of regular and irregular patterns such as trends and cycles, seasonal variations, level shifts, outliers or pulses and structural breaks, among others. Conventional parametric statistical methods are capable of forecasting a particular combination of patterns through ex ante selection of an adequate model form and specific data preprocessing. Thus, the capability of semi-parametric methods from computational intelligence to predict basic time series patterns without model selection and preprocessing is of particular relevance in evaluating their contribution to forecasting. This paper proposes an empirical comparison between NN and SVR models using radial basis function (RBF) and linear kernel functions, by analyzing their predictive power on five artificial time series: stationary, additive seasonality, linear trend, linear trend with additive seasonality, and linear trend with multiplicative seasonality. Results obtained show that RBF SVR models have problems in extrapolating trends, while NN and linear SVR models without data preprocessing provide robust accuracy across all patterns and clearly outperform the commonly used RBF SVR on trended time series.IFIP International Conference on Artificial Intelligence in Theory and Practice - Neural NetsRed de Universidades con Carreras en Informática (RedUNCI

Neural networks as a learning paradigm for general normal form games

This paper addresses how neural networks learn to play one-shot normal form games through experience in an environment of randomly generated game payoffs and randomly selected opponents. This agent based computational approach allows the modeling of learning all strategic types of normal form games, irregardless of the number of pure and mixed strategy Nash equilibria that they exhibit. This is a more realistic model of learning than the oft used models in the game theory learning literature which are usually restricted either to repeated games against the same opponent (or games with different payoffs but belonging to the same strategic class). The neural network agents were found to approximate human behavior in experimental one-shot games very well as the Spearman correlation coefficients between their behavior and that of human subjects ranged from 0.49 to 0.8857 across numerous experimental studies. Also, they exhibited the endogenous emergence of heuristics that have been found effective in describing human behavior in one-shot games. The notion of bounded rationality is explored by varying the topologies of the neural networks, which indirectly affects their ability to act as universal approximators of any function. The neural networks' behavior was assessed across various dimensions such as convergence to Nash equilibria, equilibrium selection and adherence to principles of iterated dominance

Learning Hyperinflations

Emprical studies of hyperinflations reveal that the rational expectations hypothesis fails to hold. To address this issue, we study a model of hyperinflation and learning in an attempt to better understand the volatility in movements of expectations, money, and prices. The findings surprisingly imply that the dynamics under neural network learning appear to support the outcome achieved under least squares learning reported in the earlier literature. Relaxing the assumption that inflationary expectations are rational, however, is essential since it improves the fit of the model to actual data from episodes of severe hyperinflation. Simulations provide ample evidence that if equilibrium in the model exists, then the inflation rate converges to the low inflation rational expectations equilibrium. This suggests a classical result: a permanent increase in the government deficit raises the stationary inflation rate (Marcet and Sargent, 1989)

Learning Hyperinflations

Emprical studies of hyperinflations reveal that the rational expectations hypothesis fails to hold. To address this issue, we study a model of hyperinflation and learning in an attempt to better understand the volatility in movements of expectations, money, and prices. The findings surprisingly imply that the dynamics under neural network learning appear to support the outcome achieved under least squares learning reported in the earlier literature. Relaxing the assumption that inflationary expectations are rational, however, is essential since it improves the fit of the model to actual data from episodes of severe hyperinflation. Simulations provide ample evidence that if equilibrium in the model exists, then the inflation rate converges to the low inflation rational expectations equilibrium. This suggests a classical result: a permanent increase in the government deficit raises the stationary inflation rate (Marcet and Sargent, 1989)Hyperinflation, Learning, Rational Expectations Equlibria, Neural Networks

A Neural Network Approach to Border Gateway Protocol Peer Failure Detection and Prediction

The size and speed of computer networks continue to expand at a rapid pace, as do the corresponding errors, failures, and faults inherent within such extensive networks. This thesis introduces a novel approach to interface Border Gateway Protocol (BGP) computer networks with neural networks to learn the precursor connectivity patterns that emerge prior to a node failure. Details of the design and construction of a framework that utilizes neural networks to learn and monitor BGP connection states as a means of detecting and predicting BGP peer node failure are presented. Moreover, this framework is used to monitor a BGP network and a suite of tests are conducted to establish that this neural network approach as a viable strategy for predicting BGP peer node failure. For all performed experiments both of the proposed neural network architectures succeed in memorizing and utilizing the network connectivity patterns. Lastly, a discussion of this framework\u27s generic design is presented to acknowledge how other types of networks and alternate machine learning techniques can be accommodated with relative ease

Real-time Forecasting and Control for Oscillating Wave Energy Devices

Ocean wave energy represents a signicant resource of renewable energy and can make an important contribution to the development of a more sustainable solution in support of the contemporary society, which is becoming more and more energy intensive. A perspective is given on the benefits that wave energy can introduce, in terms of variability of the power supply, when combined with oshore wind. Despite its potential, however, the technology for the generation of electricity from ocean waves is not mature yet. In order to raise the economic performance of Wave energy converters (WECs), still far from being competitive, a large scope exists for the improvement of their capacity factor through more intelligent control systems. Most control solutions proposed in the literature, for the enhancement of the power absorption of WECs, are not implemented in practise because they require future knowledge of the wave elevation or wave excitation force. The non-causality of the unconstrained optimal conditions, termed complex-conjugate control, for the maximum wave energy absorption of WECs consisting of oscillating systems, is analysed. A link between fundamental properties of the radiation of the floating body and the prediction horizon required for an effective implementation of complex-conjugate control is identified. An extensive investigation of the problem of wave elevation and wave excitation force forecasting is then presented. The prediction is treated as a purely stochastic problem, where future values of the wave elevation or wave excitation force are estimated from past measurements at the device location only. The correlation of ocean waves, in fact, allows the achievement of accurate predictions for 1 or 2 wave periods into the future, with linear Autoregressive (AR) models. A relationship between predictability of the excitation force and excitation properties of the floating body is also identified. Finally, a controller for an oscillating wave energy device is developed. Based on the assumption that the excitation force is a narrow-banded harmonic process, the controller is effectively tuned through a single parameter of immediate physical meaning, for performance and motion constraint handling. The non-causality is removed by the parametrisation, the only input of the controller being an on-line estimate of the frequency and amplitude of the excitation force. Simulations in (synthetic and real) irregular waves demonstrate that the solution allows the achievement of levels of power capture that are very close to non-causal complex-conjugate control, in the unconstrained case, and Model predictive control (MPC), in the constrained case. In addition, the hierarchical structure of the proposed controller allows the treatment of the issue of robustness to model uncertainties in quite a straightforward and effective way

Generic Architecture for Predictive Computational Modelling with Application to Financial Data Analysis: Integration of Semantic Approach and Machine Learning

The PhD thesis introduces a Generic Architecture for Predictive Computational Modelling capable of automating analytical conclusions regarding quantitative data structured as a data frame. The model involves heterogeneous data mining based on a semantic approach, graph-based methods (ontology, knowledge graphs, graph databases) and advanced machine learning methods. The main focus of my research is data pre-processing aimed at a more efficient selection of input features to the computational model. Since the model I propose is generic, it can be applied for data mining of all quantitative datasets (containing two-dimensional, size-mutable, heterogeneous tabular data); however, it is best suitable for highly interconnected data. To adapt this generic model to a specific use case, an Ontology as the formal conceptual representation for the relevant domain knowledge is needed. I have determined to use financial/market data for my use cases. In the course of practical experiments, the effectiveness of the PCM model application for the UK companies’ financial risk analysis and the FTSE100 market index forecasting was evaluated. The tests confirmed that the PCM model has more accurate outcomes than stand-alone traditional machine learning methods. By critically evaluating this architecture, I proved its validity and suggested directions for future research