Long-term Time Series Prediction Using Wrappers For Variable Selection And Clustering For Data Partition

In an attempt to implement long-term time series prediction based on the recursive application of a one-step-ahead multilayer neural network predictor, we have considered the eleven short time series provided by the organizers of the Special Session NN3 Neural Network Forecasting Competition, and have proposed a joint application of a variable selection technique and a clustering procedure. The purpose was to define unbiased partition subsets and predictors with high generalization capability, based on a wrapper methodology. The proposed approach overcomes the performance of the predictor that considers all the lags in the regression vector. After obtaining the eleven long-term predictors, we conclude the paper presenting the eighteen multi-step predictions for each time series, as requested in the competition. ©2007 IEEE.30683073Puma-Villanueva W.J. & Von Zuben, F.J. Data partition and variable selection for time series prediction using wrappers. IEEE International Joint Conference on Neural Networks (IJCNN), Vancouver, July 16-21, 2006Box, G.E.P., Jenkins, G.M., Time Series Analysis: Forecasting, and Control. Holden Day, San Francisco, CA. 1976Guyon, I., Elisseeff, A., An introduction to variable and feature selection (2003) Journal of Machine Learning Research, 3, pp. 1157-1182Kohavi, R., John, G., Wrappers for Feature Subset Selection (1997) Artificial Intelligence, 97 (1-2), pp. 273-324Bonnlander, B.V., (1996) Nonparametric selection of input variables for connectionist learning, , PhD thesis, University of ColoradoCover, T.M., Thomas, J.A., (1991) Elements of Information Theory, , Wiley, New YorkFast, F.F., Binary Feature Selection with Conditional Mutual Information (2004) Journal of Machine Learning Research, 5, pp. 1531-1555Wang, G., Lochovsky, F.H., Feature selection with conditional mutual information maximin in text categorization (2004) Conference on Information and Knowledge Management, pp. 342-349Leray, P., Gallinari, P., Feature selection with neural networks (1999) Behaviormetrika (special issue on Analysis of Knowledge Representation in Neural Network Models), 26 (1), pp. 145-166Conway, A.J., Macpherson, K.P., Brown, J.C., Delayed time series predictions with neural networks (1998) Neurocomputing, 18 (1-3), pp. 81-89Nelson, M., Hill, T., Remus, T., O'Connor, M., Time series forecasting using NNs: Should the data be deseasonalized first (1999) Journal of Forecasting, 18, pp. 359-367Ripley, B., (1993) Statistical aspects of neural networks. In Chaos and Networks - Statistical and Probabilistic Aspects, pp. 40-123. , eds O. Barnorff-Nielsen, J. Jensen and W. Kendall, London: Chapman and HallSharda, R., Patil, R.B., Conectionist approach to time series prediction: An empirical test (1992) Journal of Intelligent Manufacturiong, 3, pp. 317-323Cherkassky, V., Mulier, F., (1998) Learning from data, concepts, theory and methods, , John Wiley & Sons, New YorkHornik, K., Stinchcombe, M., White, H., Multilayer feedforward networks are universal approximators (1989) Neural Networks, 2, pp. 359-366Foster, W.R., Collopy, F., Ungar, L.H., Neural network forecasting of short, noisly time series (1992) Comput. Chem. Engng, 16, pp. 293-297Lima, C.A.M., Puma-Villanueva, W.J., dos Santos, E.P., Von Zuben, F.J., Mixture of experts applied to financial time series prediction (2004) Proceedings of the XIII Brazilian Symposium on Neural Networks, , in Portuguese, paper no. 3708Refenes, A.N., Azema-Barac, M., Karousssos, S.A., Currency exchange rate forecasting by error backpropagation (1992) Proceedings of the Twenty-Fifth Annual Hawaii International Conference on System Sciences, 4, pp. 504-515Tang, Z., de Almeida, C., Fishwick, P.A., Time series forecasting using neural networks vs. Box-Jenkins methodology (1991) Simulation, 57 (5), pp. 303-310Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., The accuracy of extrapolation (time series) methods: Results of a forecasting competition (1982) Journal of Forecasting, 1, pp. 111-153Makridakis, S., Forecasting Accuracy and System Complexity (1995) RAIRO, 29 (3), pp. 259-283Hartigan, J., Wang, M., A K-means clustering algorithm (1979) Applied Statistics, 28, pp. 100-108Bishop, C.M., (1995) Neural Networks for Pattern Recognition, , Clarendon Press, OxfordTumer, K. and Ghosh, J. Theoretical foundations of linear and order statistics combiners for neural pattern classifiers, IEEE Transactions on Neural Networks, March 1995Cellucci, C.J.Albano, A. M.Rapp, P. E. Statistical validation of mutual information calculations: Comparison of alternative numerical algorithms. Physical Review E 71, pp.066208-1-14, 2005Hansen, L.K., Salamon, P., Neural network ensembles (1990) IEEE Transactions on Pattern Analysis and Machine Intelligence, 12 (10), pp. 993-1001Hashem, S., Schmeiser, B., Yih, Y., Optimal linear combinations of neural networks: An overview (1994) Proceedings of the 1994 IEEE International Conference on Neural Networks, , Orlando, F