Time Series Prediction with Direct and Recurrent Neural Networks

Abstract

Presents a comparative study for prediction of time series of the Consumer Price Index-CPI using recurrent neural network (RNN).  For this, three models are designed for networks with recurrent and are given the changes in "backpropagation" to allow them to incorporate the models ARX (Auto-Regressive with external input) and NARX (Nonlinear Auto Regressive with external input). Furthermore, we present a third architecture, re-fed with the hidden layer, nicknamed ARXI, which is a special case of the Elman Network. Is carried out training for all networks and tests the ability to generalize them (identification stage), in order to select the best architectures of recurrent networks to prediction of the IPC. After this stage, it makes the models validation, by means of the test the extrapolation capacity of the networks, i.e., presented data were not used during the training phase and gets the responses that indicate the capacity to predict future CPI for various times (validation phase). We conclude that NARX networks are those with best performance and that the hybrid system proposed by [5] constitutes an excellent tool when you want to get minimal networks that make a series of perdition satisfactorily.

Keywords

• Recurrent Neural Networks,Time Series Prediction,NARX network

References

1. Lam M., Neural network techniques for financial performance prediction: integrating fundamental and technical analysis. Decision Support Systems 37, 2004; 567-581.
2. Yao J.T., Towards a better forecasting model for economic indices. In: Proceedings of the 6th Joint Conference on Information Science, 2002; 299-303.
3. José M. Menezes Jr. & Guilherme A. Barreto A New Look at Nonlinear Time Series Prediction with NARX Recurrent Neural Network, IX Brazilian Neural Networks Symposium (SBRN'2006), (2006), Ribeirão Preto-SP.
4. S. Crone (2005) Stepwise Selection of Artificial Neural Network Models for Time Series Prediction, Journal of Intelligent Systems, Vol. 14, No. 2-3, 2005, pp. 99-122.
5. De Campos L.M.L, R. M., A Biologically inspired methodology for neural networks design In: IEEE Conference on Cybernetics and Intelligent Systems. IEEE, pp. 619-624, 2004.
6. Cant-Paz, E. and C. Kamath, An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems, IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, pages 915-927, 2007.
7. N. Feng, G. Ning, and X. Zheng, A Framework for simulating axon guidance, In: Proceedings of Neuro computing, 2005, pp.70-84.
8. De Campos, Lídio Mauro Lima ; Roisenberg, Mauro ; Oliveira, Roberto Célio Limão de . (Accept) Automatic Design of Neural Networks with L-Systems and Genetic Algorithms - A Biologically Inspired Methodology. IJCNN 2011 - IEEE International Joint Conference on Neural Networks – San Jose, California, USA, 2011.

9. M.Tomita. Dynamic construction of Finite automata from examples using hill-climbing. In: Proceedings of the Fourth Annual Conference of the Cognitive Science Society. Ann Arbor, MI, pp. 105-108, 1982.
10. Aguirre, L.A., Introdução à Identificação de Sistemas, Editora UFMG, terceira edição, 2007.
11. PARK, D. C. et. al. Electrical load forecasting using an artificial neural network. Artificial Neural Network: forecasting time series. E.U.A., IEEE Press, p. 43-49, 1994.
12. LIN, T. et al. Learning long-term dependencies in NARX recurrent neural networks. IEEE Transactions on Neural Networks, v. 7, n. 6, p. 1424_1438, 1996.