Long short-term memory network based deep transfer learning approach for sales forecasting

Öz

Firms that operate in the production and service sectors take various decisions for the future under uncertainty in order to combat the increasing competitive conditions. One of these critical decisions is sales forecasting. With the spread of digital technologies, the use of deep learning approaches in sales forecasting is increasing. Although deep learning gives successful results, it needs long training time with large amounts of data. As a solution to this situation, transfer learning (TL), which provides information transfer between problems, is used. Transfer learning provides the training of the source data and its transfer to the target data. In this study, a deep transfer learning approach is proposed to transfer the information obtained from the sales forecasting models of different products to future forecasting models. Sales data are considered as univariate time series. The edit distance with real penalty (ERP) between the target and source data is used as a measure of transferability in the selection of source data. A long short-term memory (LSTM) network has been trained, which enables modeling of temporal dependencies with the selected source data. ERP-LSTM-TL forecasting model is created for the target data by transferring parameters from the pre-trained LSTM network. In the experimental studies with sales datasets from various industries, ERP-LSTM-TL improved the forecasting accuracy and training time compared to the LSTM trained with target data. The performance of the proposed approach was compared with the performances of the classical forecasting and machine learning methods. ERP-LSTM-TL yielded statistically better results than the compared methods.

Anahtar Kelimeler

• Long short-term memory (LSTM)
• sales forecasting
• transfer learning
• source selection
• edit distance with real penalty

Satış tahmini için uzun kısa-süreli bellek ağı tabanlı derin transfer öğrenme yaklaşımı

Öz

Üretim ve hizmet sektörlerinde faaliyet gösteren firmalar, artan rekabet koşulları ile mücadele edebilmek için belirsizlik altında geleceğe yönelik çeşitli kararlar alırlar. Bu kritik kararlardan biri satış tahminidir. Dijital teknolojilerin yaygınlaşması ile derin öğrenme yaklaşımlarının satış tahmininde kullanımı artmaktadır. Derin öğrenme, başarılı sonuçlar vermesine rağmen büyük miktarda veri ile uzun eğitim sürelerine ihtiyaç duymaktadır. Bu duruma çözüm olarak problemler arası bilgi aktarımını sağlayan transfer öğrenme (TL) kullanılmaktadır. Transfer öğrenme, kaynak veriler ile modelin eğitimini ve hedef veriye aktarımını sağlamaktadır. Bu çalışmada, farklı ürünlerin satış tahmini modellerinden elde edilen bilginin gelecekteki tahmin modellerine aktarımını sağlamak üzere derin transfer öğrenme yaklaşımı önerilmiştir. Satış verisi tek değişkenli zaman serisi olarak ele alınmıştır. Kaynak veri seçiminde aktarılabilirlik ölçütü olarak hedef ve kaynak veri arasındaki gerçek cezalı düzenleme uzaklığı (ERP) kullanılmıştır. Seçilen kaynak veri ile zamansal bağımlılıkların modellenmesini sağlayan uzun kısa vadeli hafıza (LSTM) ağı eğitilmiştir. Ön eğitilen LSTM ağında parametre transferi yapılarak hedef veri için ERP-LSTM-TL tahmin modeli oluşturulmuştur. Çeşitli sektörlere ait satış veri kümelerinde yapılan deneysel çalışmalarda ERP-LSTM-TL, hedef veri ile eğitilen LSTM’e göre tahmin doğruluğunda ve eğitim süresinde iyileşme sağlamıştır. Önerilen yaklaşımın performansı klasik tahmin ve makine öğrenmesi yöntemlerinin performansları ile karşılaştırılmıştır. ERP-LSTM-TL karşılaştırılan yöntemlere göre istatistiksel olarak daha iyi sonuç vermiştir.

Anahtar Kelimeler

• Uzun kısa vadeli hafıza (LSTM)
• satış tahmini
• transfer öğrenme
• kaynak seçimi
• gerçek cezalı düzenleme uzaklığı

Kaynakça

1. Chopra, S., Meindl, P., Kalra, D. V., Supply chain management: Strategy, planning, and operation (Vol. 232), Boston, MA: Pearson, 2013.
2. Kraus, M., Feuerriegel, S., Oztekin, A., Deep learning in business analytics and operations research: Models, applications and managerial implications, European Journal of Operational Research, 281(3), 628-641, 2020.
3. Pan, S. J., Yang, Q., A survey on transfer learning, IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345-1359, 2009.
4. Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., ... , He, Q., A comprehensive survey on transfer learning, Proceedings of the IEEE, 109(1), 43-76, 2020.
5. Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., Liu, C., A survey on deep transfer learning, In International Conference on Artificial Neural Networks, Springer, Cham, 270-279, 2018.
6. Niu, S., Liu, Y., Wang, J., Song, H., A decade survey of transfer learning (2010–2020), IEEE Transactions on Artificial Intelligence, 1(2), 151-166, 2020.
7. Weber, M., Auch, M., Doblander, C., Mandl, P., Jacobsen, H. A., Transfer Learning with Time Series Data: A Systematic Mapping Study, IEEE Access, 165409-165432, 2021.
8. Meiseles, A., Rokach, L., Source model selection for deep learning in the time series domain, IEEE Access, 8, 6190-6200, 2020.

9. Fawaz, H. I., Forestier, G., Weber, J., Idoumghar, L., Muller, P. A., Transfer learning for time series classification, In 2018 IEEE International Conference on Big Data (Big Data), IEEE, 1367-1376, 2018.
10. Ye, R., Dai, Q., Implementing transfer learning across different datasets for time series forecasting, Pattern Recognition, 109, 107617, 2021.
11. Karb, T., Kühl, N., Hirt, R., Glivici-Cotruta, V., A network-based transfer learning approach to improve sales forecasting of new products, In Proceedings of the 28th European Conference on Information Systems (ECIS), An Online AIS Conference, Marrakech-Morocco, 15-17 Haziran, 2020.
12. Loureiro, A. L., Miguéis, V. L., da Silva, L. F., Exploring the use of deep neural networks for sales forecasting in fashion retail, Decision Support Systems, 114, 81-93, 2018.
13. Yuan, F. C., Lee, C. H., Intelligent sales volume forecasting using Google search engine data, Soft Computing, 24(3), 2033-2047, 2020.
14. Kaya, S. K., Yıldırım, Ö., A prediction model for automobile sales in Turkey using deep neural networks, Endüstri Mühendisliği, 31(1), 57-74, 2020.
15. Priyadarshi, R., Panigrahi, A., Routroy, S., Garg, G. K., Demand forecasting at retail stage for selected vegetables: a performance analysis, Journal of Modelling in Management, 2019.
16. Helmini, S., Jihan, N., Jayasinghe, M., Perera, S., Sales forecasting using multivariate long short term memory network models, PeerJ PrePrints, 7, e27712v1, 2019.
17. Punia, S., Singh, S. P., Madaan, J. K., A cross-temporal hierarchical framework and deep learning for supply chain forecasting, Computers & Industrial Engineering, 149, 106796, 2020.
18. Kolková, A., Navrátil, M., Demand forecasting in python: Deep learning model based on LSTM architecture versus statistical models, Acta Polytechnica Hungarica, 18(8), 2021.
19. Huber, J., Stuckenschmidt, H., Intraday shelf replenishment decision support for perishable goods, International Journal of Production Economics, 231, 107828, 2021.
20. He, Q. Q., Wu, C., Si, Y. W., LSTM with Particle swarm optimization for sales forecasting, Electronic Commerce Research and Applications, 101118, 2022.
21. Wang, J., Liu, G. Q., Liu, L., A selection of advanced technologies for demand forecasting in the retail industry, In 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), Suzhou-China, 317-320, 15-18 Mart, 2019.
22. Aci, M., Doğansoy, G.A., Demand forecasting for e-retail sector using machine learning and deep learning methods, Journal of the Faculty of Engineering and Architecture of Gazi University, 37:3, 1325-1339, 2022.
23. Thenmozhi, K., Reddy, U. S., Crop pest classification based on deep convolutional neural network and transfer learning, Computers and Electronics in Agriculture, 164, 104906, 2019.
24. Sargano, A. B., Wang, X., Angelov, P., Habib, Z., Human action recognition using transfer learning with deep representations, In 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, 463-469, 2017.
25. Ruder, S., Peters, M. E., Swayamdipta, S., Wolf, T., Transfer learning in natural language processing, In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials, 15-18, 2019.
26. Zhao, K., Wang, C., Sales forecast in e-commerce using convolutional neural network, arXiv preprint arXiv:1708.07946, 2017.
27. Pan, H., Zhou, H., Study on convolutional neural network and its application in data mining and sales forecasting for E-commerce, Electronic Commerce Research, 20(2), 297-320, 2020.
28. Hirt, R., Srivastava, A., Berg, C., Kühl, N., Sequential transfer machine learning in networks: Measuring the impact of data and neural net similarity on transferability, In Hawaii International Conference on Systems Sciences (HICSS-54), 7078-7087, January 5-8, 2021.
29. He, Q. Q., Pang, P. C. I., Si, Y. W., Transfer learning for financial time series forecasting, In Pacific Rim International Conference on Artificial Intelligence, 24-36, Springer, Cham, 2019.
30. Hochreiter, S., Schmidhuber, J., LSTM can solve hard long time lag problems, Advances in neural information processing systems, 473-479, 1997.
31. Bengio, Y., Simard, P., Frasconi, P., Learning long-term dependencies with gradient descent is difficult, IEEE Transactions on Neural Networks, 5(2), 157-166, 1994.
32. Abbasimehr, H., Shabani, M., Yousefi, M., An optimized model using LSTM network for demand forecasting, Computers & Industrial Engineering, 143, 106435, 2020.
33. Abanda, A., Mori, U., Lozano, J. A., A review on distance based time series classification, Data Mining and Knowledge Discovery, 33(2), 378-412, 2019.
34. Chen, L., Ng, R., On the marriage of lp-norms and edit distance, In Proceedings of the Thirtieth international Conference on Very Large Data Bases, 30, 792-803, 2004.
35. Levenshtein, V. I., Binary codes capable of correcting deletions, insertions, and reversals, In Soviet Physics Doklady, 10 (8), 707-710, 1966.
36. https://www.kaggle.com/datasets. Erişim tarihi Ekim 5, 2020.
37. Puspita, P. E., İnkaya, T., Akansel, M., Clustering-based sales forecasting in a forklift distributor, International Journal of Engineering Research and Development, 11 (1), 25-40, 2019.
38. Kingma, D. P., Ba, J., Adam: A method for stochastic optimization, arXiv preprint, arXiv:1412.6980, 2014.
39. Demšar, J., Statistical comparisons of classifiers over multiple data sets, The Journal of Machine Learning Research, 7, 1-30, 2006.
40. Iman, R. L., Davenport J. M., Approximations of the critical region of the Friedman statistic, Communications in Statistics-Theory and Methods, 9(6), 571-595, 1980.