Comparison of Model Selection Criteria for Models Including Trend and Seasonal Components in Econometric Time Series

Abstract

The paper aims to compare commonly used model selection criteria in time series modeling, such as Adjusted R2, log-likelihood, Akaike Information Criterion (AIC), Schwarz Information Criterion (SIC), Hannan-Quinn (HQ) Information Criterion, and Mean Squared Error (MSE). In this context, for an additive time series, data was produced in different sample sizes from n=60 to n=500 from (17) different stationary stochastic processes, including constant, trend, seasonal and irregular components. Each production was repeated 10000 times and the criteria were calculated. For very large sample sizes, the HQ information criterion provides the best results for all types of time series models. It was observed that log-likelihood performed poorly in almost all models. It has been found that "Adjusted R2" is the best option for models with sample sizes less than 120, and "AIC" criterion is the best option for choosing the right model as the sample size increased.

Keywords

• Stationary Stochastic Process
• Model Selection Criteria
• Data Generating Process
• Monte Carlo Simulation

Ekonometrik Zaman Serilerinde Trend ve Mevsimsel Bileşenleri İçeren Modeller İçin Model Seçim Kriterlerinin Karşılaştırılması

Abstract

Çalışmada zaman serileri modellemelerinde sıklıkla karşılaştığımız Düzeltilmiş R2, log-olabilirlik, Akaike Bilgi Kriteri (AIC), Schwarz Bilgi Kriteri (SIC), Hannan-Quinn (HQ) Bilgi Kriteri ve Ortalama Hata Kareler (MSE) gibi model seçim kritelerinin karşılaştırılması amaçlanmaktadır. Bu bağlamda, toplamsal bir zaman serisi için, sabit, trend, mevsimsel ve düzensiz bileşeni içeren (17) farklı durağan stokastik süreçten n=60’dan n=500’e kadar farklı örneklem büyüklüklerinde veri üretilmiştir. Her bir üretim 10000 kez tekrarlanarak kriterler hesaplatılmıştır. Çok büyük örneklem uzunluklarında HQ bilgi kriteri her türlü zaman serisi için en iyi sonuçları vermektedir. Log-olabilirlik ise hemen hemen modellerin tamamında kötü bir performas sergilediği gözlenmiştir. Örneklem büyüklüğü 120'den küçük modeller için "Düzeltilmiş R2"nin, daha büyük örneklem büyüklüklerine gidildikçe “AIC” kriterinin doğru modeli seçmek için en iyi seçenek olduğu bulunmuştur.

Keywords

• Durağan Stokastik Süreç
• Model Seçim Kriterleri
• Veri Üretim Süreci
• Monte Carlo Simülasyonu

References

1. Acquah, H. G. (2019). Comparison of Akaike information criterion (AIC) and Bayesian information criterion (BIC) in selection of an asymmetric price relationship, International Journal of Agricultural Economics and Extension ISSN 2329-9797 Vol. 7 (1), pp. 001-006, www.internationalscholarsjournals.org.
2. Akaike, H. (1974). A new look at the statistical model identification, IEEE Transactions on Automatic Control, 19 (6). pp: 716–23.
3. Akaike, H. (1981). Likelihood of a model and information criteria. Journal of Econometrics, 16:3–14.
4. Bloomfield, P. (2000). Fourier analysis of time series an introduction, Probability and Statistics Applied Probability and Statistics Section, John Wiley &Sons, Inc., Canada and USA.
5. Buteikis, A. (2020). Pratical econometrics and data, Faculty of Mathematics and Informatics, Institute of Applied Mathematics, Vilnius University, Lithuania, http://web. vu.lt/mif/a.buteikis/.
6. Caporale, G. M. and Gil-Alana, L. (2007). Testing for deterministic and stochastic cycles in macroeconomic time series. Empirica 34, 155–169 (2007). https://doi. org/10.1007/s10663-007-9033-4
7. Clayton, M. K., Geisser, S. and Jennings, D. (1986). A comparison of several model selection procedures Bayesian inference and decision techniques: Essays in Honor of Bruno De Finetti, Studies in Bayesian Econometrics and Statistics, Vol 6, Chapter 27, (Ed: P.K. Goel and A.Zellner), Elseiver Science Publishers B.V.
8. Dete, C. H., Lokonon, B. E., Gneyou, K. E., Senou, M., Glèlè Kakaï, R., (2025). Relative Performance of Model Selection Criteria for Cox Proportional Hazards Regression Based on Kullback’s Symmetric Divergence, Journal of Probability and Statistics, 2025, 3808705, 16 pages,. https://doi.org/10.1155/jpas/3808705.

9. Ding, J., Tarokh, V. and Yang, Y. (2018). Model selection techniques-an overwiew, IEEE Signal Processing Magazine, 35(6), DOI:10.1109/ MSP.2018.2867638Corpus ID: 53035396.
10. Doğan, İ. and Doğan, N. (2020). Model performans kriterlerinin kronolojisine ve metodolojik yönlerine genel bir bakış: bir gözden geçirme, Türkiye Klinikleri Biyoistatistik Dergisi, 12(1):114-25.
11. Ebbler, D. H. (1975), On the probability of correct model selecting using the maximum r-square choıce criterion, International Economic Review, 16-2, pp.516-520.
12. Franses, P. H., Hylleberg, S. and Lee, H. S. (1995). Spurious deterministic seasoanality, Economics Letters 48 (1995) 249-256.
13. Kitagawa, G. and Gerch, W. (2012), Smoothness Priors Analysis of Time Series, Lecture Notes in Statistics 116, Springer-Verlag New York.
14. Gil-Alana, L.A. (2005). Testing of stochastic trends, seasonal and cylical components in macroeconomic time series, The Korean Communications in Statistics, 12(1), pp. 101-115.
15. Giordano, F., Niglio, M. And Storti, G. (2000).A simulation study for the evaluation of the seasonal adjustment and forecasting performances of the TESS System, Statistica Applicata,12(3).
16. Hacker, R. S. and Hatemi-J, A. (2022), “Model selection in time series analysis: using information criteria as an alternative to hypothesis testing”, Journal of Economic Studies, Vol. 49 No. 6, pp. 1055-1075. https://doi. org/10.1108/JES-09-2020-0469
17. Hannan, E. J. and Quinn, B. G. (1979).The Determination of the Order of an Autoregression, Journal of the Royal Statistical Society. Series B (Methodological). JSTOR, 190–95.
18. Hylleberg, S. (2006). Seasonal adjustment, Working Paper 2006(4), Deparment of Economics, University of AARHUS, Denmark.
19. Harvey, A. C. and Peters, S. (1990). Estimation Procedures for structural time series. Journal of Forecasting, 9(89), 89-108.
20. Hurvich, C. M. and Tsai, C. L. (1989). Regression and time series model selection in smal samples. Biometrica, 76(2), 297-307.
21. Ihaka, R. (2005). Time series analysis lecture notes for 475.726, Statistics Department University of Auckland, New Zeland.Kadane, J. B. and Lazar, N. A. (2004). Methods and Criteria for model selection, Journal of the American Statistical Association, 99(465), pp. 279-290.
22. Kitawaga, G. (2010).Introduction to time series modelling, Momographs on Statistics and Applied Probability 114, A Chapman & Hall Book, Taylor&Francis Group, Japan.
23. Kuha, J. (2004). AIC and BIC: Comparisons of assumptions and performance. Sociological Methods & Research, 33(2), 188–229. https://doi. org/10.1177/0049124103262065
24. Li, Y., Sushanta, K. M., Wang, N., Yu, J. and Zeng, T. (2025). Deviance Information Criterion for Bayesian model selection: Theoretical justification and applications, Journal of Econometrics, 2025, 105978, ISSN 0304-4076, https://doi.org/10.1016/j.jeconom.2025.105978.
25. McQuarrie, A. D. and Tsai, C. L. (1998) Regression and time series model selection,World Sciencetific, 1998
26. Mavreski, R., Milanov, P.,Traykov, M. and Pencheva, N. (2018). Performance comparison of model selection criteria by generated experimental data, ITM Web of Conferences 16, 022006 AMCSE 2017, https://doi. org/10.1051/itmconf/20181602006.
27. Mills, J. A. and Prasad, K. (1992) A comparison of model selection criteria, Econometric Reviews, 11:2, 201- 234, DOI: 10.1080/07474939208800232.
28. Riansut, W. (2023). A study of the effectiveness of model selection criteria for multiple regression model. Rajamangala University of Technology Srivijaya Research Journal, 15(1), 198–212. Retrieved from https://li01.tci-thaijo.org/index.php/rmutsvrj/article/ view/249278.
29. Rodríguez, E. G., Villalobos, H., Muñoz, V. M. G.and Rodríguez, A. R. (2015). Computational method for extracting and modeling periodicities in time series, Open Journal of Statistics,5(6), 604-617, doi: 10.4236/ ojs.2015.56062.
30. Roy, B., & Lesaffre, E. (2025). Focused information criteria for model selection – a Bayesian perspective. Journal of Applied Statistics, 1–19. https://doi.org/10.1080/02664763.2025.2514152.
31. Schwarz, Gideon, and others. 1978. “Estimating the Dimension of a Model.” The Annals of Statistics 6 (2). Institute of Mathematical Statistics: 461–64.
32. Ucal, M. Ş. (2006). Ekonometrik model seçim kriterleri üzerine kısa bir inceleme, Cumhuriyet Üniversitesi İktisadi ve İdari Bilimler Dergisi, 7(2), 41-56.
33. Wasserman, L. (2000).Bayesian model selection and model averaging. Journal of Mathematical Psychology, 44(1), 92–107. https://doi.org/10.1006/ jmps.1999.1278
34. Zhang, J., Ding, J. and Yang, Y. (2022). Is a classification procedure good enough?—A goodness-of-fit assessment tool for classification learning. Journal of the American Statistical Association, 1–11. https:// www.tandfonline.com/doi/abs/10.1080/01621459.2021.1979010
35. Zhang, J., Yang, Y. And Ding, J. (2023). Information criteria for model selection, WIREs Computantional Statistics, Advanced Review, Wiley, DOI: 10.1002/wics.1607.