Financial Time Series Prediction Using Machine Learning

Öz

Making up an important working area of finance applications, financial time series forecasting, with the advancements in Machine Learning (ML) methods in recent years, has become a topic that finance and academic circles attach more importance to. The aim of this study is to present a comparative review of ML methods in financial time series future value. In the study, the last 5-year closing data of two developed and emerging stock market indices and two high-volume stocks of the Istanbul stock market were used. Support Vector Regression (SVR), which is often used in index forecasting and found successful, and Random Forest (RF) and eXtreme Gradient Boosting (XGB) methods which are rarely used ensemble machine learning methods in time series forecasting in literature, are preferred. As a result of the study, when MAE, MAPE and RMSE criterions are taken into consideration, SVR was confirmed to be the best forecasting method.

Anahtar Kelimeler

• financial time series
• machine learning
• support vector machine
• random forest
• xgboost

Makine Öğrenmesi ile Finansal Zaman Serisi Tahminleme

Öz

Finans uygulamalarının önemli bir çalışma alanını oluşturan finansal zaman serisi tahminlemesi son yıllarda makine öğrenmesi (Machine Learning, ML) yöntemlerinin gelişimi ile finans ve akademi çevrelerinin daha fazla önem atfettiği bir konu olmuştur. Bu çalışmanın amacı, finansal zaman serisi gelecek değerinin tahmininde ML yöntemlerinin karşılaştırmalı olarak bir incelemesini sunmaktır. Çalışmada gelişmiş ve gelişmekte olan iki borsa endeksi ve İstanbul borsasının yüksek hacimli iki hisse senedinin son 5 yıllık kapanış verileri kullanılmıştır. Endeks tahmininde sıklıkla kullanılmış ve başarılı bulunan Destek Vektör Regresyonu (Suport Vector Regression, SVR) ve literatürde zaman serisi tahmininde izine az rastladığımız topluluk (ensemble) makine öğrenmesi yöntemleri olan Rassal Orman (Random Forest, RF) ve Extrem Gradyan Arttırma (eXtreme Gradient Boosting, XGB) yöntemleri tercih edilmiştir. Çalışmanın bulgularına göre, MAE, MAPE ve RMSE kriterleri göz önünde bulundurulduğunda en iyi tahmin yöntemi SVR olarak tespit edilmiştir.

Anahtar Kelimeler

• finansal zaman serisi
• makine öğrenmesi
• destek vektör makinesi
• rassal orman
• xgboost

Kaynakça

1. Abraham, A., Nath, B., & Mahanti, P. K. (2001, May). Hybrid İntelligent Systems for Stock Market Analysis. In International Conference on Computational Science (pp. 337-345), Springer, Berlin, Heidelberg.
2. Ahmad, M.W., Reynolds, J., Rezgui, Y. (2018). Predicti& Modelling for Solar Thermal Energy Systems: A Comparison of Support Vector Regression, Random Forest, Extra Trees And Regression Trees, Journal of Cleaner Production, 203, 810-821.
3. Akita, R., Yoshihara, A., Matsubara, T., & Uehara, K. (2016, June). Deep Learning for Stock Prediction Using Numerical and Textual İnformation. In 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS) (pp. 1-6). IEEE.
4. Arslankaya, S., & Toprak, Ş. (2021). Makine Öğrenmesi ve Derin Öğrenme Algoritmalarını Kullanarak Hisse Senedi Fiyat Tahmini. International Journal of Engineering Research and Development, 13(1), 178-192.
5. Ashfaq, N., Nawaz, Z., & Ilyas, M. (2021). A Comparative Study of Different Machine Learning Regressors for Stock Market Prediction. Arxiv Preprint Arxiv:2104.07469. Breiman, L. (2001). Random Forests. Machine Learning, 45(1), 5-32.
6. Cao, J., & Wang, J. (2019). Stock Price Forecasting Model Based on Modified Convolution Neural Network and Financial Time Series Analysis. International Journal of Communication Systems, 32(12), e3987.
7. Cao, J., Li, Z., & Li, J. (2019). Financial Time Series Forecasting Model Based On CEEMDAN And LSTM. Physica A: Statistical Mechanics and Its Applications, 519, 127-139.
8. Cao, L. J., & Tay, F. E. H. (2003). Support Vector Machine with Adaptive Parameters in Financial Time Series Forecasting. IEEE Transactions on Neural Networks, 14, 1506– 1518. Doi:10.1109/TNN.2003.820556.

9. Cao, L., & Tay, F. E. (2001). Financial Forecasting Using Support Vector Machines. Neural Computing and Applications, 10(2), 184-192.
10. Chen SM (1996) Forecasting Enrollments Based On Fuzzy Time-Series. Fuzzy Sets Syst 81:311–319.
11. Chen, H., Xiao, K., Sun, J., & Wu, S. (2017). A Double-Layer Neural Network Framework for High-Frequency Forecasting. ACM Transactions on Management Information Systems (TMIS), 7(4), 1-17.
12. Chen, T., & Guestrin, C. (2016, August). Xgboost: A Scalable Tree Boosting System. In Proceedings of The 22nd Acm Sigkdd İnternational Conference on Knowledge Discovery and Data Mining (pp. 785-794).
13. Chen, Y. S., Cheng, C. H., & Tsai, W. L. (2014). Modeling Fitting-Function-Based Fuzzy Time Series Patterns for Evolving Stock İndex Forecasting. Applied İntelligence, 41(2), 327-347.
14. Cherkassky, V., Ma, Y. (2004). Practical Selection of SVM Parameters and Noise Estimation for SVM Regression, Neural Networks 17, 113–126.
15. Choudhry, R., & Garg, K. (2008). A Hybrid Machine Learning System for Stock Market Forecasting. International Journal of Computer and Information Engineering, 2(3), 689-692.
16. Crone, S., Nikolopoulos, K.: Results of The NN3 Neural Network Forecasting Competition. The 27th International Symposium on Forecasting, Program, pp. 129 (2007).
17. Demirel, U., Çam H., & Ünlü R., (2021). Predicting Stock Prices Using Machine Learning Methods and Deep Learning Algorithms: The Sample of The Istanbul Stock Exchange. Gazi University Journal of Science, 34(1), 63-82.
18. Deviren, B., Kocakaplan, Y., Keskin, M., Balcılar, M., Özdemir, Z. A., & Ersoy, E. (2014). Analysis of Bubbles and Crashes In The TRY/USD, TRY/EUR, TRY/JPY and TRY/CHF Exchange Rate Within The Scope of Econophysics. Physica A: Statistical Mechanics and Its Applications, 410, 414-420.
19. Di Matteo, Tiziana. 2007. Multi-Scaling In Finance. Quantitative Finance 7: 21–36.
20. Egeli, B., Ozturan, M., & Badur, B. (2003). Stock Market Prediction Using Artificial Neural Networks. Decision Support Systems, 22, 171-185.
21. Enke, D., & Thawornwong, S. (2005). The Use of Data Mining and Neural Networks for Forecasting Stock Market Returns. Expert Systems with Applications, 29(4), 927-940.
22. Fischer, T., & Krauss, C. (2018). Deep Learning with Long Short-Term Memory Networks for Financial Market Predictions. European Journal of Operational Research, 270(2), 654-669.
23. Fu, J., Lum, K. S., Nguyen, M. N., & Shi, J. (2007, June). Stock Prediction Using Fcmac-Byy. In International Symposium on Neural Networks (pp. 346-351). Springer, Berlin, Heidelberg.
24. Gerlein, E. A., Mcginnity, M., Belatreche, A., & Coleman, S. (2016). Evaluating Machine Learning Classification for Financial Trading: An Empirical Approach. Expert Systems with Applications, 54, 193-207.
25. Gunn, S.R. (1998). Support Vector Machines for Classification and Regression. ISIS Technical Report (Available At: Http://Users.Ecs.Soton.Ac.Uk/Srg/Publications/Pdf/SVM.Pdf).
26. Hamzaçebi, C., Akay, D., & Kutay, F. (2009). Comparison of Direct and Iterative Artificial Neural Network Forecast Approaches In Multi-Periodic Time Series Forecasting. Expert Systems with Applications, 36(2), 3839-3844.
27. Hansen, J. V., Mcdonald, J. B., & Nelson, R. D. (1999). Time Series Prediction with Genetic‐Algorithm Designed Neural Networks: An Empirical Comparison with Modern Statistical Models. Computational Intelligence, 15(3), 171-184.
28. He K, Yu L, Lai KK. Crude Oil Price Analysis and Forecasting Using Wavelet Decomposed Ensemble Model. Energy 2012;46(1):564e74.
29. Henrique, B. M., Sobreiro, V. A., & Kimura, H. (2018). Stock Price Prediction Using Support Vector Regression on Daily And up to The Minute Prices. The Journal of Finance and Data Science, 4(3), 183-201.
30. Hsu, M. W., Lessmann, S., Sung, M. C., Ma, T., & Johnson, J. E. (2016). Bridging the Divide In Financial Market Forecasting: Machine Learners & Financial Economists. Expert Systems With Applications, 61, 215-234.
31. Hu, M. Y., Zhang, G., Jiang, C. X., & Patuwo, B. E. (1999). A Cross‐Validation Analysis of Neural Network out‐of‐Sample Performance In Exchange Rate Forecasting. Decision Sciences, 30(1), 197-216.
32. Huang, W., Nakamori, Y., & Wang, S. Y. (2005). Forecasting Stock Market Movement Direction With Support Vector Machine. Computers & Operations Research, 32(10), 2513-2522.
33. Karasu, S., Altan, A., Bekiros, S., & Ahmad, W. (2020). A New Forecasting Model With Wrapper-Based Feature Selection Approach Using Multi-Objective Optimization Technique For Chaotic Crude Oil Time Series. Energy, 212, 118750.
34. Kazem, A., Sharifi, E., Hussain, F. K., Saberi, M., & Hussain, O. K. (2013). Support Vector Regression with Chaos-Based Firefly Algorithm for Stock Market Price Forecasting. Applied Soft Computing, 13(2), 947-958.
35. Kim, K. J. (2003). Financial Time Series Forecasting Using Support Vector Machines. Neurocomputing, 55(1-2), 307-319.
36. Kim, K. J., & Han, I. (2000). Genetic Algorithms Approach to Feature Discretization In Artificial Neural Networks for The Prediction of Stock Price Index. Expert Systems with Applications, 19(2), 125-132.
37. Kumar, D., Meghwani, S. S., & Thakur, M. (2016). Proximal Support Vector Machine Based Hybrid Prediction Models for Trend Forecasting In Financial Markets. Journal of Computational Science, 17, 1-13.
38. Kumar, R., Kumar, P., & Kumar, Y. (2021, January). Analysis of Financial Time Series Forecasting Using Deep Learning Model. In 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence) (pp. 877-881), IEEE.
39. Kuremoto, T., Kimura, S., Kobayashi, K., & Obayashi, M. (2014). Time Series Forecasting Using A Deep Belief Network with Restricted Boltzmann Machines. Neurocomputing, 137, 47-56.
40. Lu, C. J., Lee, T. S., & Chiu, C. C. (2009). Financial Time Series Forecasting Using İndependent Component Analysis and Support Vector Regression. Decision Support Systems, 47(2), 115-125.
41. Moghaddam, A. H., Moghaddam, M. H., & Esfandyari, M. (2016). Stock Market Index Prediction Using Artificial Neural Network. Journal of Economics, Finance and Administrative Science, 21(41), 89-93.
42. Nava, N., Di Matteo, T., & Aste, T. (2018). Financial Time Series Forecasting Using Empirical Mode Decomposition and Support Vector Regression. Risks, 6(1), 7.
43. Nava, Noemi, Tiziana Di Matteo, And Tomaso Aste. 2016(a). Time-Dependent Scaling Patterns in High Frequency Financial Data. The European Physical Journal Special Topics 225: 1997–2016.
44. Niu, T., Wang, J., Lu, H., Yang, W., & Du, P. (2020). Developing A Deep Learning Framework with Two-Stage Feature Selection for Multivariate Financial Time Series Forecasting. Expert Systems with Applications, 148, 113237.
45. Nonejad N. Prediction Aggregate Stock Market Volatility Using Financial and Macroeconomic Predictors: Which Models Forecast Best, When and Why?. J Empir Financ. 2017;42:131‐154.
46. Pai, P. F., & Lin, C. S. (2005). A Hybrid ARIMA and Support Vector Machines Model In Stock Price Forecasting. Omega, 33(6), 497-505.
47. Patel, J., Shah, S., Thakkar, P., & Kotecha, K. (2015). Predicting Stock Market Index Using Fusion of Machine Learning Techniques. Expert Systems with Applications, 42(4), 2162-2172.
48. Rasel, R. I., Sultana, N., & Meesad, P. (2015). An Efficient Modelling Approach for Forecasting Financial Time Series Data Using Support Vector Regression and Windowing Operators. International Journal of Computational Intelligence Studies, 4(2), 134-150.
49. Ser-Huang Poon, Forecasting Volatility In Financial Markets: A Review, J. Econ. Lit. 41 (2) (2003) 478–539.
50. Sezer, O. B., Gudelek, M. U., & Ozbayoglu, A. M. (2020). Financial Time Series Forecasting with Deep Learning: A Systematic Literature Review: 2005–2019. Applied Soft Computing, 90, 106181.
51. Smola, A.J., Scholkopf, B., 2004. A Tutorial on Support Vector Regression. Stat. Comput. 14, 199–222.
52. Tay, F. E., & Cao, L. (2001). Application of Support Vector Machines In Financial Time Series Forecasting. Omega, 29(4), 309-317.
53. Tsai, C. F., & Wang, S. P. (2009, March). Stock Price Forecasting by Hybrid Machine Learning Techniques. In Proceedings of The İnternational Multiconference of Engineers and Computer Scientists (Vol. 1, No. 755, P. 60).
54. V.N. Vapnik, (2000). The Nature Of Statistical Learning Theory, Springer, New York.
55. Vapnik, V., Cortes, C. (1995). Support Vector Networks. Machine Learning. 20 (3), 273–297.
56. Vapnik, V.N. (1999). An Overview of Statistical Learning Theory, IEEE Transactions on Neural Networks 10 988–999.
57. Vijh, M., Chandola, D., Tikkiwal, V. A., & Kumar, A. (2020). Stock Closing Price Prediction Using Machine Learning Techniques. Procedia Computer Science, 167, 599-606.
58. Wang,L., Zhou, X., Zhu, X., Dong, Z., Guo, W. (2016). Estimation of Biomass In Wheat Using Random Forest Regression Algorithm and Remote Sensing Data, The Crop Journal, 4(3),212-219.
59. Yakut, Y., Yakut, E., & Yavuz, S. (2014). Yapay Sinir Ağları ve Destek Vektör Makineleri Yöntemleriyle Borsa Endeksi Tahmini. Süleyman Demirel Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 19(1), 139-157.
60. Yan, D., Zhou, Q., Wang, J., & Zhang, N. (2017). Bayesian Regularisation Neural Network Based on Artificial İntelligence Optimisation. International Journal of Production Research, 55(8), 2266-2287.
61. Yetis, Y., Kaplan, H., & Jamshidi, M. (2014, August). Stock Market Prediction by Using Artificial Neural Network. In 2014 World Automation Congress (WAC) (pp. 718-722). IEEE.
62. Yu HK (2005) Weighted Fuzzy Time-Series Models for TAIEX Forecasting. Physica A 34, 609–624.
63. Yu L, Wang S, Lai KK. Forecasting Crude Oil Price with an EMD-Based Neural Network Ensemble Learning Paradigm. Energy Econ 2008;30(5):2623e35.
64. Yu, L., Chen, H., Wang, S., & Lai, K. K. (2009). Evolving Least Squares Support Vector Machines For Stock Market Trend Mining. IEEE Transactions On Evolutionary Computation, 38, 802–815. Doi:10.1109/TEVC.2008.928176.
65. Yu, S., Tian, L., Liu, Y., & Guo, Y. (2021, July). LSTM-XGBoost Application of The Model To The Prediction of Stock Price. In International Conference on Artificial Intelligence and Security (pp. 86-98). Springer, Cham.
66. Zhang, N., Lin, A., & Shang, P. (2017). Multidimensional K-Nearest Neighbor Model Based on EEMD for Financial Time Series Forecasting. Physica A: Statistical Mechanics and its Applications, 477, 161-173.