Long-term Time Series Prediction Using Wrappers For Variable Selection And Clustering For Data Partition

In an attempt to implement long-term time series prediction based on the recursive application of a one-step-ahead multilayer neural network predictor, we have considered the eleven short time series provided by the organizers of the Special Session NN3 Neural Network Forecasting Competition, and have proposed a joint application of a variable selection technique and a clustering procedure. The purpose was to define unbiased partition subsets and predictors with high generalization capability, based on a wrapper methodology. The proposed approach overcomes the performance of the predictor that considers all the lags in the regression vector. After obtaining the eleven long-term predictors, we conclude the paper presenting the eighteen multi-step predictions for each time series, as requested in the competition. ©2007 IEEE.30683073Puma-Villanueva W.J. & Von Zuben, F.J. Data partition and variable selection for time series prediction using wrappers. IEEE International Joint Conference on Neural Networks (IJCNN), Vancouver, July 16-21, 2006Box, G.E.P., Jenkins, G.M., Time Series Analysis: Forecasting, and Control. Holden Day, San Francisco, CA. 1976Guyon, I., Elisseeff, A., An introduction to variable and feature selection (2003) Journal of Machine Learning Research, 3, pp. 1157-1182Kohavi, R., John, G., Wrappers for Feature Subset Selection (1997) Artificial Intelligence, 97 (1-2), pp. 273-324Bonnlander, B.V., (1996) Nonparametric selection of input variables for connectionist learning, , PhD thesis, University of ColoradoCover, T.M., Thomas, J.A., (1991) Elements of Information Theory, , Wiley, New YorkFast, F.F., Binary Feature Selection with Conditional Mutual Information (2004) Journal of Machine Learning Research, 5, pp. 1531-1555Wang, G., Lochovsky, F.H., Feature selection with conditional mutual information maximin in text categorization (2004) Conference on Information and Knowledge Management, pp. 342-349Leray, P., Gallinari, P., Feature selection with neural networks (1999) Behaviormetrika (special issue on Analysis of Knowledge Representation in Neural Network Models), 26 (1), pp. 145-166Conway, A.J., Macpherson, K.P., Brown, J.C., Delayed time series predictions with neural networks (1998) Neurocomputing, 18 (1-3), pp. 81-89Nelson, M., Hill, T., Remus, T., O'Connor, M., Time series forecasting using NNs: Should the data be deseasonalized first (1999) Journal of Forecasting, 18, pp. 359-367Ripley, B., (1993) Statistical aspects of neural networks. In Chaos and Networks - Statistical and Probabilistic Aspects, pp. 40-123. , eds O. Barnorff-Nielsen, J. Jensen and W. Kendall, London: Chapman and HallSharda, R., Patil, R.B., Conectionist approach to time series prediction: An empirical test (1992) Journal of Intelligent Manufacturiong, 3, pp. 317-323Cherkassky, V., Mulier, F., (1998) Learning from data, concepts, theory and methods, , John Wiley & Sons, New YorkHornik, K., Stinchcombe, M., White, H., Multilayer feedforward networks are universal approximators (1989) Neural Networks, 2, pp. 359-366Foster, W.R., Collopy, F., Ungar, L.H., Neural network forecasting of short, noisly time series (1992) Comput. Chem. Engng, 16, pp. 293-297Lima, C.A.M., Puma-Villanueva, W.J., dos Santos, E.P., Von Zuben, F.J., Mixture of experts applied to financial time series prediction (2004) Proceedings of the XIII Brazilian Symposium on Neural Networks, , in Portuguese, paper no. 3708Refenes, A.N., Azema-Barac, M., Karousssos, S.A., Currency exchange rate forecasting by error backpropagation (1992) Proceedings of the Twenty-Fifth Annual Hawaii International Conference on System Sciences, 4, pp. 504-515Tang, Z., de Almeida, C., Fishwick, P.A., Time series forecasting using neural networks vs. Box-Jenkins methodology (1991) Simulation, 57 (5), pp. 303-310Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., The accuracy of extrapolation (time series) methods: Results of a forecasting competition (1982) Journal of Forecasting, 1, pp. 111-153Makridakis, S., Forecasting Accuracy and System Complexity (1995) RAIRO, 29 (3), pp. 259-283Hartigan, J., Wang, M., A K-means clustering algorithm (1979) Applied Statistics, 28, pp. 100-108Bishop, C.M., (1995) Neural Networks for Pattern Recognition, , Clarendon Press, OxfordTumer, K. and Ghosh, J. Theoretical foundations of linear and order statistics combiners for neural pattern classifiers, IEEE Transactions on Neural Networks, March 1995Cellucci, C.J.Albano, A. M.Rapp, P. E. Statistical validation of mutual information calculations: Comparison of alternative numerical algorithms. Physical Review E 71, pp.066208-1-14, 2005Hansen, L.K., Salamon, P., Neural network ensembles (1990) IEEE Transactions on Pattern Analysis and Machine Intelligence, 12 (10), pp. 993-1001Hashem, S., Schmeiser, B., Yih, Y., Optimal linear combinations of neural networks: An overview (1994) Proceedings of the 1994 IEEE International Conference on Neural Networks, , Orlando, F

Phylogenetic Inheritance Of Genetic Variability Produced By Neutral Models Of Evolution.

The amount of genetic variability in species and populations has been mainly related to microevolutionary forces operating in natural populations and the influence of phylogenetic processes for the distribution of genetic variability has been neglected. To investigate how the current genetic variability distribution depends on the genetic variability of ancestral species, we simulated the evolution of heterozygosity on a pre-determined phylogeny under three neutral models of evolution: genetic drift, drift vs mutation and drift vs migration. The distribution of genetic variability resulting from the simulations was used to estimate the phylogenetic signal by the phylogenetic comparative method of autocorrelation. Phylogenetic signal in genetic variability was observed for each of the three models, and its intensity was generally higher and persisted longer when forces of drift, mutation and migration were reduced. The prediction of a phylogenetic signal in genetic variability has consequences for: population genetics, which must consider biological processes acting at the species level influencing the amount and distribution of genetic variability; the macroevolutionary theory, by giving a theoretical basis for species selection by suggesting a heritability of genetic variability between species, and the meta-analyses of genetic variability, which must deal with the non-independence of species. The patterns observed in phylogenetic signal produced by different models of evolution can be used further to compare with data obtained from molecular markers. This is the first study that analyzes the theoretical expectations for the existence of a phylogenetic signal in a population genetic trait.741327134

Mixture Of Heterogeneous Experts Applied To Time Series: A Comparative Study

Prediction models for time series generally include preprocessing followed by the synthesis of an input-output mapping. Neural network models have been adopted to perform both steps, by means of unsupervised and supervised learning, respectively. The flexibility and the generalization capability are the most relevant attributes in favor of connectionist approaches. However, even though time series prediction can be roughly interpreted as learning from data, high levels of performance will solely be achieved if some peculiarities of each time series are properly considered in the design, particularly the existence of trend and seasonality. Instead of directly adopting detrend and/or deseasonality treatments, this paper proposes a novel paradigm for supervised learning based on a mixture of heterogeneous experts. Some mixture models have already been proved to produce good performance as predictors, but the present approach will be devoted to a hybrid mixture composed of a set of distinct experts. The purpose is not only to further explore the "divide-and-conquer" principle, but also to compare the performance of mixture of heterogeneous experts with the standard mixture of experts approach, using ten distinct time series. The obtained results indicate that mixture of heterogeneous experts generally requires a more elaborate gating device and performs better in the case of more challenging time series. © 2005 IEEE.211601165Box, G.E.P., Jenkins, G.M., (1976) Time Series Analysis: Forecasting, and Control, , Holden Day, San Francisco, CABridle, J.S., Probabilistic interpretation of feedforward classification network outputs with relationships to statistical pattern recognition (1990) Neurocomputing: Algorithms. Architectures, and Applications, pp. 227-236. , F. Fogelman Soulié and J. Hérault (eds.), Springer Verlag, New YorkCheng, B., Titterington, M., Neural Networks: A review from a statistical perspective with discussion (1994) Statist. Sci., 9, pp. 2-54Conway, A.J., (1995) The Prediction and Analysis of Solar Terrestrial Time Series, , Ph.D. Thesis, University of GlasgowConway, A.J., Macpherson, K.P., Brown, J.C., Delayed time series predictions with neural networks (1998) Neurocomputing, 18 (1-3), pp. 81-89Cybenko, G., Approximation by superpositions of sigmoid function (1989) Mathematics of Control Signals and Systems, 2, pp. 303-314Foster, W.R., Collopy, F., Ungar, L.H., Neural network forecasting of short, noisly time series (1992) Comput. Chem. Engng, 16, pp. 293-297Franses, P.H., Draisma, G., Recognizing changing seasonal patterns using artificial neural networks (1997) Journal of Econometrics, 81, pp. 273-280Fritsch, J., (1996) Modular Neural Networks for Speech Recognition, , Master's thesis, Carnegie Mellon University & University of KarlsruheFritsch, J., Finke, M., Waibel, A., Context-dependent hybrid HME/HMM speech recognition using polyphone clustering decision trees (1997) Procs. of ICASSPFunahashi, K., On the approximate realization of continuous mappings by neural networks (1989) Neural Networks, 2, pp. 183-192Hansen, J.V., Nelson, R.D., Forecasting and recombining time-series components by using neural networks (2003) Journal of the Operational Research Society, 54 (3), pp. 307-317Hornik, K., Stinchcombe, M., White, H., Multilayer feedforward networks are universal approximators (1989) Neural Networks, 2, pp. 359-366Huerta, G., Jiang, W., Tanner, M.A., Time series modeling via hierarchical mixtures (2003) Statistica Sinica, 13, pp. 1097-1118Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, E.G., Adaptive mixture of local experts (1991) Neural Computation, 3 (1), pp. 79-87Jordan, M.I., Jacobs, A.R., Hierarchical mixtures of experts and EM algorithm (1994) Neural Computation, 6, pp. 181-214Kang, S., (1991) An Investigation of the Use of Feedforward Neural Networks for Forecasting, , Ph.D. Thesis, Kent StateKimura, A., Arizono, I., Ohta, H., An improvement of a back propagation algorithm by the extended kalman filter and demand forecasting by layered neural networks (1996) Int. J. of Systems Science, 27 (5), pp. 473-482Lima, C.A.M., Puma-Villanueva, W.J., Dos Santos, E.P., Von Zuben, F.J., Mixture of experts applied to financial time series prediction (2004) Proceedings of the XIII Brazilian Symposium on Neural Networks, , paper no. 3708Lima, C.A.M., Coelho, A.L.V., Von Zuben, F.J., Mixture of experts applied to nonlinear dynamic systems identification: A comparative study (2002) Proceedings of the VII Brazilian Symposium on Neural Networks, pp. 162-167. , Porto de Galinhas. Recife. November 11-14Makridakis, S., Anderson, A., Carbone, R., Fildes, R., Hibon, R., Lewandowski, R., Newton, J., Winkler, R., The accuracy of extrapolation time series methods: Results of a forecasting competition (1982) Journal of Forecasting, 1, pp. 111-153Marseguerra, M., Minoggio, S., Rossi, A., Zio, E., Neural networks prediction and fault diagnosis applied to stationary and non stationary ARMA modeled time series (1992) Progress in Nuclear Energy, 27 (1), pp. 25-36McLachlan, G.J., Basford, K.E., (1988) Mixture Models: Inference and Applications to Clustering, , Marcel DeckkerMoerland, P., Classification using localized mixture of experts (1999) Procs. of ICANN, 2, pp. 838-843Narendra, K.S., Parthasarathy, K., Identification and control of dynamical systems neural networks (1990) IEEE Transactions Neural Networks, 1 (1), pp. 4-27Nelson, M., Hill, T., Remus, T., O'Connor, M., Time series forecasting using NNs: Should the data be deseasonalized first (1999) Journal of Forecasting, 18, pp. 359-367Ramamurti, V., Ghosh, J., Structural adaptation in mixture of experts (1996) Procs. ICPR 96, pp. 704-708. , Track DRefenes, A.N., Azema-Barac, M., Karousssos, S.A., Currency exchange rate forecasting by error backpropagation (1992) Proceedings of the Twenty-fifth Annual Hawaii International Conference on System Sciences, 4, pp. 504-515Ripley, B., Statistical aspects of neural networks (1993) Chaos and Networks - Statistical and Probabilistic Aspects, pp. 40-123. , (eds O. Bamorff-Nielsen, J. Jensen and W. Kendall), London: Chapman and HallSharda, R., Patil, R.B., Conectionist approach to time series prediction: An empirical test (1992) Journal of Intelligent Manufacturiong, 3, pp. 317-323Tang, Z., De Almeida, C., Fishwick, P.A., Time series forecasting using neural networks vs. Box-Jenkins methodology (1991) Simulation, 57 (5), pp. 303-310Van Der Vaar, H.R., An example of the performance of time series methods with respect to a known model (1997) Time Series and Ecological Processes, Proceeding of a SIMS Conference, , Alta, Utha, sponsored by SIAM Institute for Mathematics and SocietyWaterhouse, S.R., Robinson, A.J., Classification using hierarchical mixtures of experts (1994) Procs. IEEE Workshop on Neural Networks for Signal Processing, pp. 177-186. , Long Beach CAWeigend, A.S., Mangeas, M., Srivastava, A.N., Nonlinear gated experts for time series: Discovering regimes and avoiding overfitting (1995) International Journal of Neural Systems, 6, pp. 373-399White, H., Gallant, A.R., (1992) There Exists a Neural Network That Does Not Make Avoidable Mistakes, , White, H. (ed.), Artificial Neural Networks: Approximations and Learning Theory, Oxfort: BlackwellWhite, H., Stinchcombe, M., (1992) Approximating and Learning Unknown Mapping Using Multilayer Feedforward Networks with Bounded Weights, , White, H. (ed.). Artificial Neural Networks: Approximations and Learning Theory, Oxfort: BlackwellXu, L., Jordan, M.I., Hinton, G.E., An alternative model for mixtures of experts (1995) Advances in Neural Information Processing Systems, 7, pp. 633-640. , G. Tesauro, D. S. Touretzky, and T. K. Leen, (eds), MIT Press, Cambridge MAZeevi, A., Meir, R., Adler, R., Time series prediction using mixtures of experts (1996) Proceedings of Advances in Neural Information Processing SystemsZhang, G.P., Qi, M., Neural network forecasting for seasonal and trend time series (2005) European Journal of Operation Research, 160, pp. 501-51