LSTM ve GRU Modellerinde Açıklanabilir Yapay Zekâ Yaklaşımı: İktisadi Veriler Üzerine Bir Uygulama

Abstract

Çalışmanın temel amacı, mali politikaların istihdam üzerindeki etkisini açıklamalı yapay zekâ metotlarıyla araştırmaktır. Araştırmada, Ocak 2005 ile Ekim 2023 arasındaki döneme ait aylık istihdam, bütçe tahsilatı, bütçe giderleri ve bütçe tahsilatının bütçe giderlerine oranlarına ilişkin veriler kullanılmıştır. Analizde, benzer performans değerlerine sahip olduğu ortaya çıkan LSTM ve GRU modelleri bazında uygulanan açıklamalı yapay zekâ algoritmalarının bulgularının birbirine yakın olması, önemli bir sonuç olarak değerlendirilebilir. Modellerin performans değerleri birbirine yakın olduğu için her iki model bazında XAI yöntemleri kullanılmıştır. Bunun için Dalex paketinin olanaklarından yararlanılmıştır. LSTM ve GRU mimarilerinin yapısı gereği bağımsız değişkenlerin üç boyutlu olması gerekmektedir. Dalex paketinin girdileri ise iki boyutlu veri şeklindedir. Makalenin analiz kısmında paketin bu eksiklikleri yazar tarafından geliştirilen kodlar ile giderilmiştir.

Keywords

• LSTM
• GRU
• XAI

Explainable Artificial Intelligence Approach in LSTM and GRU Models: An Application on Economic Data

Abstract

The main objective of the study is to investigate the impact of fiscal policies on employment within the framework of economic theories by using explainable artificial intelligence methods. The study uses monthly data on employment, budget revenues, budget expenditures, and the ratio of budget revenues to budget expenditures for the period between January 2005 and October 2023. In the analysis, it can be considered an important result that the findings of the explainable artificial intelligence algorithms applied on the basis of LSTM and GRU models, which are found to have similar performance values, are close to each other. Since the performance values of the models are close to each other, XAI methods were used on the basis of both models. For this, the facilities of the Dalex package were used. Due to the structure of the LSTM and GRU architectures, the independent variables are required to be three-dimensional. The inputs of the Dalex package are in the form of two-dimensional data. In the analysis part of the article, these deficiencies of the package were overcome with the codes developed by the author.

Keywords

• LSTM
• GRU
• XAI

References

1. Adadi A, Berrada M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI), in IEEE Access, vol. 6, pp. 52138-52160, doi: 10.1109/ACCESS.2018.2870052.
2. Akköse, O. (2020, 22 Aralık). Uzun-Kısa Vadeli Bellek (LSTM). 10.02.2024 tarihinde https://medium.com/deep-learning-turkiye/uzun-k%C4%B1sa-vadeli-bellek-lstm-b018c07174a3#:~:text=Unutma%20Kap%C4%B1s%C4%B1(Forget%20Gate)%3A,State%20ile%20ta%C5%9F%C4%B1nmaya%20devam%20eder adresinden edinilmiştir.
3. Angelov, P.P.; Soares, E.A.; Jiang, R.; Arnold, N.I.; Atkinson, P.M.(2021). Explainable artificial intelligence: An analytical review. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 11(5)
4. Baniecki, H., Kretowicz, W., Piatyszek, P., Wiśniewski, J., & Biecek, P. (2021). dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python. Journal of Machine Learning Research, 22(214), 1–7. Retrieved from http://jmlr.org/papers/v22/20-1473.html
5. Bnm, Manju (2022, 8 Haziran), Non-Parametric Time Series Forecasting. 10. 02 2024 tarihinde https://manjubnm.medium.com/amazon-forecastsnon-parametric-time-series-forecasting-6ac8217acbd adresinden edinilmiştir
6. Battaglini, M. ve Coate, S. (2011). Fıscal Polıcy And Unemployment (Working Paper No 17562). Natıonal Bureau Of Economıc Research (Nber). http://www.nber.org/papers/w17562 (14.02.2024)
7. Biecek, P.& Burzykowskı, T. (2021). Explanatory Model Analysıs, Explore, Explain, and Examine Predictive Models. CRC Press
8. Celik, Ş.(2015). Detection of Causal Relationship among Apricot Production, Dollar Exchange Rate and Gold Prices Using Co-integration Analysis and VECM: The Case of Turkey, International Journal of Science and Research (IJSR) , 5(7), 2056-2059., Doi: 10.21275/v5i7 Full Text Online (PDF) https://www.ijsr.net/archive/v5i7/ART2016677.pdf

9. Das, A., & Rad, P. (2020). Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey. ArXiv, abs/2006.11371.
10. Dickey, D. A. & Fuller, W. A. (1979). Distribution of the Estimators for Autoregressive Time Series with a Unit Root. Journal of the American Statistical Association, 74, 427-431. https://doi.org/10.1080/01621459.1979.10482531.
11. Duman, H.(2023). Cevresel Koşulların Sıgırlarda Sut Verimi Üzerine Etkisinin Makiıne Ögrenme modelleri ile Araştırılması, Yayımlanmamış Doktora Tezi, Iğdır Üniversitesi, Iğdır.
12. Ertop, K. (2006). Makro İktisat. Nihat Sayar Eğitim Vakfı Yayınları
13. Freeborough,W.; van Zyl, T. (2022). Investigating Explainability Methods in Recurrent Neural Network Architectures for Financial Time Series Data. Appl. Sci, 12(3), 1427. https://doi.org/10.3390/app12031427
14. Giordano, R., Momigliano, S., Neri, S. & Per, R. (2008). The Effects Of Fiscal Policy In Italy: Evidence From A Var Model [Working paper 656]. Banca D’Italia. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1105745
15. Gholami, H., Mohammadifar, A., Golzari, S. Song, Y., Pradhan, B. (2023). Interpretability of simple RNN and GRU deep learning models used to map land susceptibility to gully erosion, Science of the Total Environment, 904, 166960 https://doi.org/10.1016/j.scitotenv.2023.166960
16. Gonzalo, J. 1994. Five Alternative Methods of Estimating Long-Run Equilibrium Relationships. Journal of Econometrics, 60, 203–233. https://doi.org/10.1016/0304-4076(94)90044-2.
17. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., Elhadad, N. (2015). Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, 10–13 August 2015; pp. 1721–1730.
18. Gunning D, Stefik M, Choi J, Miller T, Stumpf S, Yang G-Z. (2019). XAI-explainable artificial intelligence. Science Robotics, 4(37).
19. Johansen, S. and Juselius, K. (1990). Maximum Likelihood Estimation and Inference on Cointegration—With Applications to the Demand for Money. Oxford Bulletin of Economics and Statistics, 52, 169-210.
20. Hsieh, T.-Y., Wang, S., Sun, Y., & Honavar, V. (2021). Explainable multivariate time series classification: A deep neural network which learns to attend to important variables as well as informative time intervals. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining (pp. 607–615). Association for Computing Machinery. https://doi.org/10.1145/3437963.3441815
21. Hochreiter, S. & Schmidhuber, J. (1997). Long short-term memory. Neural Comput., 9 (8), 1735–1780
22. Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: Principles and Practice (2nd ed.). OTexts. https://otexts.com/fpp2/
23. Kızrak, A. (2019). Açıklanabilir Yapay Zekâ Nedir ve İhtiyaç Mıdır? 14.02.2024 tarihinde https://ayyucekizrak.medium.com/a%C3%A7%C4%B1klanabilir-yapay-zeka-nedir-ve-i%CC%87htiya%C3%A7-m%C4%B1d%C4%B1r-65adef9b086 adresinden edinilmiştir.
24. Machlev, R., Heistrene, L., Perl, M., Levy, K.Y., Belikov, J., Mannor, S., Levron, Y. (2022). Explainable Artificial Intelligence (XAI) techniques for energy and power systems: Review, challenges and opportunities, Energy and AI, 9, https://doi.org/10.1016/j.egyai.2022.100169.
25. Mateus, B. C. Mendes, M., Farinha, J. T., Assis, R. & Cardoso, A. M. (2021). Comparing Lstm And Gru Models To Predict The Condition Of A Pulp Paper Press, Energies, 14, 6958. https://doi.org/10.3390/en14216958
26. Li, X. and Yang, T. (2021). Forecast of the Employment Situation of College Graduates Based on the LSTM Neural Network. Hindawi Computational Intelligence and Neuroscience, 1-11, https://doi.org/10.1155/2021/5787355
27. Mulaudzi, R. Ajoodha, R. (2020). An Exploration of Machine Learning Models to Forecast the Unemployment Rate of South Africa: A Univariate Approach. 2nd International Multidisciplinary Information Technology and Engineering Conference (IMITEC), 25-27 Nov. Kimberley, South Africa ISBN: 978-1-7281-9520-9
28. Nkoro, E., Uko, A. K.(2016), Autoregressive Distributed Lag (ARDL) cointegration technique: application and interpretation. Journal of Statistical and Econometric Methods, 5(4), 63-91
29. Ono, T. (2019). Growth, Unemployment, And Fıscal Polıcy: A Polıtıcal Economy Analysıs, Macroeconomic Dynamics, 23(8), 3099-3139
30. Olah, C. (2022). Understanding, L.S.T.M.: Networks. http:// colah. github. io/ posts/ 2015- 08- Under stand ingLS TMs/.
31. Özel U. (2020). Açıklanabilir Yapay Zekâ (Explainable AI). 12.02.2024 tarihinde https://www.umutozel.com/explainable-ai adresinden edinilmiştir.
32. Parikh, R.B., Teeple, S.i Navathe, A.S.(2019). Addressing bias in artificial intelligence in health care. JAMA, 22(24):2377-2378, doi: 10.1001/jama.2019.18058
33. Ravanelli, M. Brakel, P. Omologo, M. and Bengio, Y. (2018). Light Gated Recurrent Units for Speech Recognition. IEEE Journal Of Emergıng Topıcs In Computatıonal Intellıgence, 2( 2), 92-101
34. Sel, A. (2020). Python Uygulamalı İstatistik ve Veri Bilimi. Akademisyen Kitap Evi yaynları
35. TC Hazine ve Maliye Bakanlığı Muhasebat Genel Müdürlüğü 2023, https://muhasebat.hmb.gov.tr/iller-itibariyle-merkezi-yonetim-butce-istatistikleri-2004-2019
36. Topal, M. H. (2017). Türkiye’de Kamu Yatırımlarının İstihdam Üzerindeki Etkisi: Bölgesel Bir Analiz (2004-2016). Küresel İktisat ve İşletme Çalışmaları Dergisi, 6(12), 186-204
37. Türkiye İstatistik Kurumu (2022), Nüfusun İşgücü Durumu, Temel İşgücü Göstergeleri (15+ yaş) Tablosu, https://data.tuik.gov.tr/Bulten/Index?p=Isgucu-Istatistikleri-Kasim-2022-49384
38. Kundu, S., Singhania, R. (2020). Forecasting the United States Unemployment Rate by Using Recurrent Neural Networks with Google Trends Data International Journal of Trade, Economics and Finance, 11 (6),
39. Wang, M. & Ying, F. (2023). Point and interval prediction for significant wave height based on LSTM-GRU and KDE, Ocean Engineering, 289, 1-17
40. Wang, K., Ma, C., Qiao, Y., Lu, X. Hao, W., Dong, S. (2021). A hybrid deep learning model with 1DCNN-LSTM-Attention networks for short-term traffic flow prediction, Physica A: Statistical Mechanics and its Applications, 583, 2021, ISSN 0378-4371, https://doi.org/10.1016/j.physa.2021.126293.
41. Yeomans, J., Thwaites, S., Robertson, W.S., Booth, D., Ng, B. & Thewlis, D. (2019). Simulating Time-Series Data for Improved Deep Neural Network Performance. IEEE, 7, 131248–131255.
42. Yurtsever, M. (2023). Unemployment rate forecasting: LSTM‑GRU hybrid approach, Journal for Labour Market Research, 57(18). doi.org/10.1186/s12651-023-00345-8
43. Yao, Z., Wang, Z., Wang, D., Wu, J. & Chen, L. (2023). An ensemble CNN-LSTM and GRU adaptive weighting model based improved sparrow search algorithm for predicting runoff using historical meteorological and runoff data as input. Journal of Hydrology, 625, 129977 7
44. Zhu, J.H., Pei, J.H., Zhao, Y. (2019). Research on convolution kernel initialization method in convolutional neural network (CNN) training, Signal Process. 35 (4) (2019) 641–648.