TY  - JOUR
T1  - Gated recurrent unit network-based fuzzy time series forecasting model
TT  - Kapılı Tekrarlayan Hücreler Tabanlı Bulanık Zaman Serileri Tahminleme Modeli
AU  - Arslan, Serdar
PY  - 2023
DA  - June
DO  - 10.35414/akufemubid.1175297
JF  - Afyon Kocatepe Üniversitesi Fen Ve Mühendislik Bilimleri Dergisi
PB  - Afyon Kocatepe Üniversitesi
WT  - DergiPark
SN  - 2149-3367
SP  - 677
EP  - 692
VL  - 23
IS  - 3
LA  - en
AB  - Time series forecasting has lots of  applications in various industries such as weather, business, retail and energy consumption forecasting. Accurate prediction in these applications is very important and also difficult task because of complexity and uncertainty of time series.  Nowadays, using deep learning methods is a popular approach in time series forecasting and shows better performance than classical methods. However, in the literature, there are few studies which use deep learning methods in fuzzy time series (FTS) forecasting. In this study, we propose a novel FTS forecasting model which is based upon hybridization of Recurrent Neural Networks with FTS  to deal with complexity and also uncertanity of these series. The proposed model utilizes Gated Recurrent Unit (GRU) to make prediction by using combination of membership values and also past value from original time series data as model input and produce real forecast value. Moreover, the proposed model can handle first order fuzzy relations as well as high order ones.  In experiments, we have compared our model results with those of state-of-art methods by using two real world datasets; The Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) and Nikkei Stock Average. The results indicate that our model outperforms or performs similar to other methods. The proposed model is also validated by using Covid-19 active case dataset and shows better performance than Long Short-term Memory (LSTM) networks.
KW  - Gated Recurrent Unit
KW  - Time Series Forecasting
KW  - Fuzzy Time Series
KW  - Deep Learning
N2  - Zaman serisi tahminleme hava durumu, iş dünyası, satış verileri ve enerji tüketimi tahminleme gibi bir çok alanda uygulama alanına sahiptir. Bu alanlarda tahminleme yaparken kesin sonuçlar elde etmek çok önemlidir ama aynı zamanda zaman serilerinin karmaşık ve de belirsizlik içeren veriler olması nedeniyle çok zordur. Günümüzde, derin öğrenme metotları bu alanda klasik metotlara göre daha iyi sonuçlar vermektedir. Fakat literatürde bulanık zaman serileri tahminleme konusunda çok az çalışma vardır. Bu çalışmada, zaman serilerindeki karmaşıklığın ve belirsizliğin doğurduğu problemleri yok etmek için Yinelemeli sinir Ağları ile bulanık time serilerini bir arada kullanan bir model ortaya konmuştur. Bu çalışmada, Kapılı Tekrarlayan Hücreler kullanarak geçmiş veriler ile bulanık verilerin üyelik değerleri birleştirilerek tahminleme değeri hesaplanmıştır. Ayrıca, bu çalışmadaki model ilk seviye bulanık ilişkileri ele alabildiği gibi, çoklu seviye bulanık ilişkileri de kapsamaktadır. Testlerde literatürde var olan çalışmalar ilgili model ile iki açık veri seti ile karşılaştırılmış olup bahsi geçen modelin daha iyi veya benzer sonuçlar verdiği gözlemlenmiştir. Ayrıca model Covid-19 verileri kullanılarak da test edilmiş ve Uzun-Kısa Süreli Bellek modellerinden daha iyi sonuç vermiştir.
CR  - Aladag, Cagdas H., Basaran, M. A., Egrioglu, E., Yolcu, U., & Uslu, V. R. (2009). Forecasting in high order fuzzy times series by using neural networks to define fuzzy relations. Expert Systems with Applications, 36(3 PART 1), 4228–4231. https://doi.org/10.1016/j.eswa.2008.04.001
CR  - Aladag, Cagdas H, Basaran, M. A., Egrioglu, E., Yolcu, U., & Uslu, V. R. (2009). Forecasting in high order fuzzy times series by using neural networks to define fuzzy relations. Expert Systems with Applications, 36(3, Part 1), 4228–4231. https://doi.org/https://doi.org/10.1016/j.eswa.2008.04.001
CR  - Aladag, Cagdas Hakan. (2013). Using multiplicative neuron model to establish fuzzy logic relationships. Expert Systems with Applications, 40(3), 850–853. https://doi.org/10.1016/j.eswa.2012.05.039
CR  - Aladag, Cagdas Hakan, Yolcu, U., & Egrioglu, E. (2010a). A high order fuzzy time series forecasting model based on adaptive expectation and artificial neural networks. Mathematics and Computers in Simulation, 81(4), 875–882. https://doi.org/https://doi.org/10.1016/j.matcom.2010.09.011
CR  - Aladag, Cagdas Hakan, Yolcu, U., & Egrioglu, E. (2010b). A high order fuzzy time series forecasting model based on adaptive expectation and artificial neural networks. Mathematics and Computers in Simulation, 81(4), 875–882. https://doi.org/https://doi.org/10.1016/j.matcom.2010.09.011
CR  - Aladag, Cagdas Hakan, Yolcu, U., Egrioglu, E., & Dalar, A. Z. (2012). A new time invariant fuzzy time series forecasting method based on particle swarm optimization. Applied Soft Computing, 12(10), 3291–3299. https://doi.org/https://doi.org/10.1016/j.asoc.2012.05.002
CR  - Bahrepour, M., Akbarzadeh-T., M. R., Yaghoobi, M., & Naghibi-S., M. B. (2011). An adaptive ordered fuzzy time series with application to FOREX. Expert Systems with Applications, 38(1), 475–485. https://doi.org/10.1016/j.eswa.2010.06.087
CR  - Bas, E., Egrioglu, E., Aladag, C. H., & Yolcu, U. (2015). Fuzzy-time-series network used to forecast linear and nonlinear time series. Applied Intelligence, 43(2), 343–355. https://doi.org/10.1007/s10489-015-0647-0
CR  - Bas, E., Grosan, C., Egrioglu, E., & Yolcu, U. (2018). High order fuzzy time series method based on pi-sigma neural network. Engineering Applications of Artificial Intelligence, 72, 350–356. https://doi.org/https://doi.org/10.1016/j.engappai.2018.04.017
CR  - Becerra-Rico, J., Aceves-Fernández, M. A., Esquivel-Escalante, K., & Pedraza-Ortega, J. C. (2020). Airborne particle pollution predictive model using Gated Recurrent Unit (GRU) deep neural networks. Earth Science Informatics, 13(3), 821–834. https://doi.org/10.1007/s12145-020-00462-9
CR  - Bose, M., & Mali, K. (2019). Designing fuzzy time series forecasting models: A survey. International Journal of Approximate Reasoning, 111, 78–99. https://doi.org/10.1016/j.ijar.2019.05.002
CR  - Bulut, E. (2014). Modeling seasonality using the fuzzy integrated logical forecasting (FILF) approach. Expert Systems with Applications, 41(4, Part 2), 1806–1812. https://doi.org/https://doi.org/10.1016/j.eswa.2013.08.079
CR  - Cagcag Yolcu, O., & Lam, H. K. (2017). A combined robust fuzzy time series method for prediction of time series. Neurocomputing, 247, 87–101. https://doi.org/10.1016/j.neucom.2017.03.037
CR  - Cai, Q., Zhang, D., Zheng, W., & Leung, S. C. H. (2015). A new fuzzy time series forecasting model combined with ant colony optimization and auto-regression. Knowledge-Based Systems, 74, 61–68. https://doi.org/https://doi.org/10.1016/j.knosys.2014.11.003
CR  - Castillo, O., Alanis, A., Garcia, M., & Arias, H. (2007). An intuitionistic fuzzy system for time series analysis in plant monitoring and diagnosis. Applied Soft Computing, 7(4), 1227–1233. https://doi.org/https://doi.org/10.1016/j.asoc.2006.01.010
CR  - Chang, J.-R., Wei, L.-Y., & Cheng, C.-H. (2011). A hybrid ANFIS model based on AR and volatility for TAIEX forecasting. Applied Soft Computing, 11(1), 1388–1395. https://doi.org/https://doi.org/10.1016/j.asoc.2010.04.010
CR  - Chen, M.-Y., & Chen, B.-T. (2014). Online fuzzy time series analysis based on entropy discretization and a Fast Fourier Transform. Applied Soft Computing, 14, 156–166. https://doi.org/https://doi.org/10.1016/j.asoc.2013.07.024
CR  - Chen, S.-M., & Kao, P.-Y. (2013). TAIEX forecasting based on fuzzy time series, particle swarm optimization techniques and support vector machines. Information Sciences, 247, 62–71. https://doi.org/https://doi.org/10.1016/j.ins.2013.06.005
CR  - Chen, S.-M., & Tanuwijaya, K. (2011). Fuzzy forecasting based on high-order fuzzy logical relationships and automatic clustering techniques. Expert Systems with Applications, 38(12), 15425–15437. https://doi.org/https://doi.org/10.1016/j.eswa.2011.06.019
CR  - Chen, S. M. (1996). Forecasting enrollments based on fuzzy time series. Fuzzy Sets and Systems, 81(3), 311–319. https://doi.org/10.1016/0165-0114(95)00220-0
CR  - Cho, K., Merrienboer, B. van, Gülçehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. BT  - Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, (pp. 1724–1734). https://doi.org/10.3115/v1/d14-1179
CR  - Cui, Z., & Wang, Y. (2017). Deep Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction.
CR  - De Gooijer, J. G., & Hyndman, R. J. (2006). 25 years of time series forecasting. International Journal of Forecasting, 22(3), 443–473. https://doi.org/https://doi.org/10.1016/j.ijforecast.2006.01.001
CR  - Dutta, A., Kumar, S., & Basu, M. (2020). A Gated Recurrent Unit Approach to Bitcoin Price Prediction. In Journal of Risk and Financial Management  (Vol. 13, Issue 2). https://doi.org/10.3390/jrfm13020023
CR  - Efendi, R., Ismail, Z., & Deris, M. M. (2015). A new linguistic out-sample approach of fuzzy time series for daily forecasting of Malaysian electricity load demand. Applied Soft Computing, 28, 422–430. https://doi.org/https://doi.org/10.1016/j.asoc.2014.11.043
CR  - Egrioglu, E., Aladag, C. H., & Yolcu, U. (2013). Fuzzy time series forecasting with a novel hybrid approach combining fuzzy c-means and neural networks. Expert Systems with Applications, 40(3), 854–857. https://doi.org/https://doi.org/10.1016/j.eswa.2012.05.040
CR  - Egrioglu, E., Aladag, C. H., Yolcu, U., Uslu, V. R., & Basaran, M. A. (2009). A new approach based on artificial neural networks for high order multivariate fuzzy time series. Expert Systems with Applications, 36(7), 10589–10594. https://doi.org/https://doi.org/10.1016/j.eswa.2009.02.057
CR  - George E. P. Box, G. M. J. G. C. R. G. M. L. (2015). Time Series Analysis: Forecasting and Control. John Wiley & Sons.
CR  - Graves, A., Mohamed, A., & Hinton, G. E. (2013). Speech Recognition with Deep Recurrent Neural Networks. CoRR, abs/1303.5. http://arxiv.org/abs/1303.5778
CR  - Hájek, P., & Olej, V. (2017). Intuitionistic neuro-fuzzy network with evolutionary adaptation. Evolving Systems, 8(1), 35–47. https://doi.org/10.1007/s12530-016-9157-5
CR  - Huarng, K. (2001a). Effective lengths of intervals to improve forecasting in fuzzy time series. Fuzzy Sets and Systems, 123(3), 387–394. https://doi.org/https://doi.org/10.1016/S0165-0114(00)00057-9
CR  - Huarng, K. (2001b). Heuristic models of fuzzy time series for forecasting. Fuzzy Sets and Systems, 123(3), 369–386. https://doi.org/https://doi.org/10.1016/S0165-0114(00)00093-2
CR  - Huarng, K., & Yu, T. H. K. (2006). The application of neural networks to forecast fuzzy time series. Physica A: Statistical Mechanics and Its Applications, 363(2), 481–491. https://doi.org/10.1016/j.physa.2005.08.014
CR  - Jang, J.-. R. (1993). ANFIS: adaptive-network-based fuzzy inference system. IEEE Transactions on Systems, Man, and Cybernetics, 23(3), 665–685. https://doi.org/10.1109/21.256541
CR  - Jozefowicz, R., Zaremba, W., & Sutskever, I. (2015). An empirical exploration of Recurrent Network architectures. 32nd International Conference on Machine Learning, ICML 2015, 3, 2332–2340.
CR  - Kocak, C. (2017). ARMA(p,q) type high order fuzzy time series forecast method based on fuzzy logic relations. Applied Soft Computing, 58, 92–103. https://doi.org/https://doi.org/10.1016/j.asoc.2017.04.021
CR  - Kocak, C., Egrioglu, E., & Bas, E. (2021). A new deep intuitionistic fuzzy time series forecasting method based on long short-term memory. Journal of Supercomputing, 77(6), 6178–6196. https://doi.org/10.1007/s11227-020-03503-8
CR  - Lee, L.-W., Wang, L.-H., & Chen, S.-M. (2008). Temperature prediction and TAIFEX forecasting based on high-order fuzzy logical relationships and genetic simulated annealing techniques. Expert Systems with Applications, 34(1), 328–336. https://doi.org/https://doi.org/10.1016/j.eswa.2006.09.007
CR  - Li, S.-T., Cheng, Y.-C., & Lin, S.-Y. (2008). A FCM-based deterministic forecasting model for fuzzy time series. Computers & Mathematics with Applications, 56(12), 3052–3063. https://doi.org/https://doi.org/10.1016/j.camwa.2008.07.033
CR  - Lin, T., Horne, B. G., & Giles, C. L. (1998). How embedded memory in recurrent neural network architectures helps learning long-term temporal dependencies (Vol. 11, Issue 5, pp. 861–868). https://www.scopus.com/inward/record.uri?eid=2-s2.0-0032123428&doi=10.1016%2FS0893-6080%2898%2900018-5&partnerID=40&md5=ab7519875b4103882070c5a56a6c7a1e
CR  - Lindemann, B., Müller, T., Vietz, H., Jazdi, N., & Weyrich, M. (2021). A survey on long short-term memory networks for time series prediction. Procedia CIRP, 99, 650–655. https://doi.org/https://doi.org/10.1016/j.procir.2021.03.088
CR  - Martín Abadi, Ashish Agarwal, Paul Barham, E. B., Zhifeng Chen, Craig Citro, Greg S. Corrado, A. D., Jeffrey Dean, Matthieu 
Devin, Sanjay Ghemawat, I. G., Andrew Harp, Geoffrey Irving, Michael Isard, Rafal Jozefowicz, Y. J., Lukasz Kaiser, Manjunath 
Kudlur, Josh Levenberg, Dan Mané, M. S., Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, J. S., Benoit Steiner, Ilya 
Sutskever, Kunal Talwar, P. T., Vincent Vanhoucke, Vijay Vasudevan, F. V., Oriol Vinyals, Pete Warden, Martin Wattenberg, M. W., & Yuan Yu,  and X. Z. (n.d.). TensorFlow:Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/
CR  - Nie, J. (1997). Nonlinear time-series forecasting: A fuzzy-neural approach. Neurocomputing, 16(1), 63–76. https://doi.org/https://doi.org/10.1016/S0925-2312(97)00019-2
CR  - Novák, V. (1995). Towards Formalized Integrated Theory of Fuzzy Logic BT  - Fuzzy Logic and its Applications to Engineering, Information Sciences, and Intelligent Systems (Z. Bien & K. C. Min (eds.); pp. 353–363). Springer Netherlands. https://doi.org/10.1007/978-94-009-0125-4_35
CR  - Panigrahi, S., & Behera, H. S. (2020). A study on leading machine learning techniques for high order fuzzy time series forecasting. Engineering Applications of Artificial Intelligence, 87, 103245. https://doi.org/10.1016/j.engappai.2019.103245
CR  - Sadaei, H J, Enayatifar, R., Guimarães, F. G., Mahmud, M., & Alzamil, Z. A. (2016). Combining ARFIMA models and fuzzy time series for the forecast of long memory time series (Vol. 175, pp. 782–796). https://www.scopus.com/inward/record.uri?eid=2-s2.0-84963768497&doi=10.1016%2Fj.neucom.2015.10.079&partnerID=40&md5=17238b9bbacc56abf01b11050b134966
CR  - Sadaei, Hossein Javedani, de Lima e Silva, P. C., Guimarães, F. G., & Lee, M. H. (2019). Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series. Energy, 175, 365–377. https://doi.org/10.1016/j.energy.2019.03.081
CR  - Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673–2681. https://doi.org/10.1109/78.650093
CR  - Shen, G., Tan, Q., Zhang, H., Zeng, P., & Xu, J. (2018a). Deep Learning with Gated Recurrent Unit Networks for Financial Sequence Predictions. Procedia Computer Science, 131, 895–903. https://doi.org/https://doi.org/10.1016/j.procs.2018.04.298
CR  - Shen, G., Tan, Q., Zhang, H., Zeng, P., & Xu, J. (2018b). Deep Learning with Gated Recurrent Unit Networks for Financial Sequence Predictions. Procedia Computer Science, 131, 895–903. https://doi.org/https://doi.org/10.1016/j.procs.2018.04.298
CR  - Siami-Namini, S., Tavakoli, N., & Namin, A. S. (2019). The Performance of LSTM and BiLSTM in Forecasting Time Series. 2019 IEEE International Conference on Big Data (Big Data), 3285–3292. https://doi.org/10.1109/BigData47090.2019.9005997
Singh, P. (2017). A brief review of modeling approaches based on fuzzy time series. International Journal of Machine Learning and Cybernetics, 8(2), 397–420. https://doi.org/10.1007/s13042-015-0332-y
CR  - Singh, P., & Borah, B. (2013). High-order fuzzy-neuro expert system for time series forecasting. Knowledge-Based Systems, 46, 12–21. https://doi.org/10.1016/j.knosys.2013.01.030
CR  - Singh, P., & Borah, B. (2014). An effective neural network and fuzzy time series-based hybridized model to handle forecasting problems of two factors. Knowledge and Information Systems, 38(3), 669–690. https://doi.org/10.1007/s10115-012-0603-9
CR  - Song, Q., & Chissom, B. S. (1993). Fuzzy time series and its models. Fuzzy Sets and Systems, 54(3), 269–277. https://doi.org/https://doi.org/10.1016/0165-0114(93)90372-O
CR  - Stefanakos, C. (2016). Fuzzy time series forecasting of nonstationary wind and wave data. Ocean Engineering, 121, 1–12. https://doi.org/https://doi.org/10.1016/j.oceaneng.2016.05.018
CR  - Sullivan, J., & Woodall, W. H. (1994). A comparison of fuzzy forecasting and Markov modeling. Fuzzy Sets and Systems, 64(3), 279–293. https://doi.org/https://doi.org/10.1016/0165-0114(94)90152-X
CR  - Tan, Q., Ye, M., Yang, B., Liu, S., Ma, A. J., Yip, T. C.-F., Wong, G. L.-H., & Yuen, P. (2020). DATA-GRU: Dual-Attention Time-Aware Gated Recurrent Unit for Irregular Multivariate Time Series. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01), 930–937. https://doi.org/10.1609/aaai.v34i01.5440
CR  - Torbat, S., Khashei, M., & Bijari, M. (2018). A hybrid probabilistic fuzzy ARIMA model for consumption forecasting in commodity markets. Economic Analysis and Policy, 58, 22–31. https://doi.org/https://doi.org/10.1016/j.eap.2017.12.003
CR  - Tran, N., Nguyen, T., Nguyen, B. M., & Nguyen, G. (2018). A Multivariate Fuzzy Time Series Resource Forecast Model for Clouds using LSTM and Data Correlation Analysis. Procedia Computer Science, 126, 636–645. https://doi.org/10.1016/j.procS.2018.07.298
CR  - Tsaur, R.-C., O Yang, J.-C., & Wang, H.-F. (2005). Fuzzy relation analysis in fuzzy time series model. Computers & Mathematics with Applications, 49(4), 539–548. https://doi.org/https://doi.org/10.1016/j.camwa.2004.07.014
CR  - Tseng, F.-M., Tzeng, G.-H., Yu, H.-C., & Yuan, B. J. C. (2001). Fuzzy ARIMA model for forecasting the foreign exchange market. Fuzzy Sets and Systems, 118(1), 9–19. https://doi.org/https://doi.org/10.1016/S0165-0114(98)00286-3
CR  - Wang, Y., Liao, W., & Chang, Y. (2018). Gated Recurrent Unit Network-Based Short-Term Photovoltaic Forecasting. In Energies  (Vol. 11, Issue 8). https://doi.org/10.3390/en11082163
CR  - Wong, H.-L., Tu, Y.-H., & Wang, C.-C. (2010). Application of fuzzy time series models for forecasting the amount of Taiwan export. Expert Systems with Applications, 37(2), 1465–1470. https://doi.org/https://doi.org/10.1016/j.eswa.2009.06.106
CR  - Yang, S., Yu, X., Zhou, Y., & Yu, X. (2020). LSTM and GRU Neural Network Performance Comparison Study: Taking Yelp Review Dataset as an Example. Proceedings - 2020 International Workshop on Electronic Communication and Artificial Intelligence, IWECAI 2020, 98–101. https://doi.org/10.1109/IWECAI50956.2020.00027
CR  - Yu, T. H. K., & Huarng, K. H. (2010). A neural network-based fuzzy time series model to improve forecasting. Expert Systems with Applications, 37(4), 3366–3372. https://doi.org/10.1016/j.eswa.2009.10.013
CR  - Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338–353. https://doi.org/https://doi.org/10.1016/S0019-9958(65)90241-X
UR  - https://doi.org/10.35414/akufemubid.1175297
L1  - https://dergipark.org.tr/tr/download/article-file/2650845
ER  -