Comparison of automated machine learning (AutoML) libraries in time series forecasting

Öz

Companies must make forecasts for the future to take necessary precautions, as well as to guard or expand their position and remain competitive. The development of data technologies has made it easier to reach meaningful data. Analyzing these data with methods such as artificial intelligence, machine learning, and deep learning makes it possible to obtain highly accurate results in future forecasts. However, the presence of numerous methods in the literature poses several challenges for researchers, including selecting the most suitable method and determining the appropriate techniques for model and hyper-parameter selection. Moreover, comparing different values in the model and making hyper-parameter selections can be tedious and time-consuming. Therefore, this study aims to use the Automated Machine Learning (AutoML) method, which is an advanced version of machine learning. AutoML automates machine learning models, allowing the use and development of machine learning algorithms without requiring expertise in this field. The study carried out forecasts using 6 different AutoML libraries on univariate time series datasets, and forecasting successes were compared over various performance metrics. According to the results obtained on the data set used, it was observed that the Auto_ARIMA library had the highest forecasting success rate among the selected libraries.

Anahtar Kelimeler

• AutoML
• AutoML libraries
• Time series forecasting

Zaman serisi tahminlemede otomatikleştirilmiş makine öğrenmesi (AutoML) kütüphanelerinin karşılaştırılması

Öz

Firmaların bulunduğu konumu koruması veya geliştirebilmesi, ileride meydana gelebilecek durumlara karşı önceden tedbir alabilmesi ve diğer firmalar ile rekabet edebilmesi açısından geleceğe yönelik tahminleme gerçekleştirmesi gereklidir. Gelişen veri teknolojileri ile anlamlı veriye ulaşabilmek kolaylaşmıştır ve yapay zekâ, makine öğrenmesi, derin öğrenme gibi yöntemlerle birlikte bu verilerin analiz edilmesiyle geleceğe yönelik tahminlemede oldukça başarılı sonuçlar elde edilebilmektedir. Ancak literatürde birçok farklı yöntemin bulunması, araştırmacıların hangi yöntemi kullanacağı, model ve hiper-parametre seçimi için hangi teknikleri kullanacağı gibi birçok problem oluşturabilmektedir. Model ve hiper-parametre seçimde farklı değerlerin karşılaştırılması zahmetli ve uzun süreli olabilir. Bu doğrultuda gerçekleştirilen çalışmanın amacı, makine öğrenmesinin gelişmiş bir versiyonu olan otomatikleştirilmiş makine öğrenmesi (AutoML) yöntemini kullanmaktır. AutoML, makine öğrenmesi modellerini otomatikleştirerek bu alanda uzmanlık gerektirmeden makine öğrenmesi algoritmalarının kullanımına ve geliştirilmesine olanak tanır. Çalışmada, tek değişkenli bir zaman serisi verisi üzerinde 6 farklı AutoML kütüphanesi ile tahminleme çalışması gerçekleştirilmiştir ve tahminleme başarıları çeşitli performans metrikleri üzerinden karşılaştırılmıştır. Kullanılan veri seti üzerinde elde edilen sonuçlara göre seçilen kütüphanelerden tahminleme başarısı en yüksek olanın Auto_ARIMA kütüphanesi olduğu gözlenmiştir.

Anahtar Kelimeler

• AutoML
• AutoML kütüphaneleri
• Zaman serisi tahminleme

Teşekkür

Bu çalışmanın 1. Yazarı TÜBİTAK 2211-A Yurt İçi Doktora Burs Programı tarafından desteklenmektedir. Ancak yayın ile ilgili tüm sorumluluk yayının sahibine aittir. Yayının içeriğinin bilimsel anlamda TÜBİTAK tarafından onaylandığı anlamına gelmez.

Kaynakça

1. 1. Alsharef A., Aggarwal K., Sonia, Kumar M., Mishra A., Review of ML and AutoML solutions to forecast time-series data, Arch Computat Methods Eng, 29 (7), 5297-311, 2022.
2. 2. Petropoulos F., Spiliotis E., The wisdom of the data: getting the most out of univariate time series forecasting, Forecasting, 3 (3), 478-97, 2021.
3. 3. Masini R.P., Medeiros M.C., Mendes E.F., Machine learning advances for time series forecasting, Journal of Economic Surveys, 37 (1), 76-111, 2023.
4. 4. Tealab A., Time series forecasting using artificial neural networks methodologies: a systematic review, Future Computing and Informatics Journal, 3 (2), 334-40, 2018.
5. 5. Torres J.F., Hadjout D., Sebaa A., Martínez-Álvarez F., Troncoso A., Deep learning for time series forecasting: a survey, Big Data, 9 (1), 3-21, 2021.
6. 6. Mohr F., Wever M., Hüllermeier E., ML-Plan: automated machine learning via hierarchical planning, Mach Learn, 107 (8), 1495-515, 2018.
7. 7. Karmaker S.K., Hassan M., Smith M.J., Xu L., Zhai C., Veeramachaneni K., AutoML to date and beyond: challenges and opportunities, Association for Computing Machinery, 54 (8), 175:1-175:36, 2021.
8. 8. Yao Q., Wang M., Chen Y., Dai W., Li Y.F., Tu W.W., Yang Q., Yu Y., Taking human out of learning applications: a survey on automated machine learning, arXiv preprint arXiv:1810.13306, 2018.

9. 9. Hutter F., Kotthoff L., Vanschoren J., Automated machine learning: methods, systems, challenges, Springer Nature, 219, 2019.
10. 10. Erickson N., Mueller J., Shirkov A., Zhang H., Larroy P., Li M., Smola A., Autogluon-tabular: robust and accurate automl for structured data, arXiv preprint arXiv:2003.06505, 2020.
11. 11. Prescient & Strategic Intelligence Private Limited. AutoML Market. https://www.reportlinker.com/p06191010/ AutoML-Market.html. Yayın tarihi Kasım, 2021. Erişim tarihi Şubat 7, 2023.
12. 12. Ahlgren F., Mondejar M.E., Thern M., Predicting dynamic fuel oil consumption on ships with automated machine learning, Energy Procedia, 158, 6126-6131, 2019.
13. 13. Zhang Q., Hu W., Liu Z., Tan J., TBM performance prediction with Bayesian optimization and automated machine learning, Tunnelling and Underground Space Technology, 103, 2020.
14. 14. Zeineddine H., Braendle U., Farah A., Enhancing prediction of student success: automated machine learning approach, Computers & Electrical Engineering, 89, 2021.
15. 15. Zhang C., Ye Z., Water pipe failure prediction using AutoML, Facilities, 39 (1/2), 36-49, 2021.
16. 16. Bender J., Trat M., Ovtcharova J., Benchmarking AutoML-supported lead time prediction, Procedia Computer Science, 200, 482-94, 2022.
17. 17. Duan S., Zhang X., AutoML-based drought forecast with meteorological variables, arXiv preprint arXiv:2207.07012, 2022.
18. 18. Gomathi S., Kohli R., Soni M., Dhiman G., Nair R., Pattern analysis: predicting COVID-19 pandemic in India using AutoML, World Journal of Engineering, 19 (1), 21-28, 2022.
19. 19. Muniz Do Nascimento W., Gomes-Jr L., Enabling low-cost automatic water leakage detection: a semi-supervised, AutoML-based approach, Urban Water Journal, 1-11, 2022.
20. 20. Bahri M., Salutari F., Putina A., Sozio M., AutoML: state of the art with a focus on anomaly detection, challenges, and research directions, International Journal of Data Science and Analytics, 14 (2), 113-126, 2022.
21. 21. Wu D., Guan Q., Fan Z., Deng H., Wu T., AutoML with parallel genetic algorithm for fast hyperparameters optimization in efficient IoT time series prediction, IEEE Transactions on Industrial Informatics, 1-10, 2022.
22. 22. Truong A., Walters A., Goodsitt J., Hines K., Bruss C.B., Farivar R., Towards automated machine learning: evaluation and comparison of AutoML approaches and tools, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), 1471-9, 2019.
23. 23. Waring J., Lindvall C., Umeton R., Automated machine learning: review of the state-of-the-art and opportunities for healthcare, Artificial Intelligence in Medicine, 104, 101822, 2020.
24. 24. Koc K., Gurgun A.P., Scenario-based automated data preprocessing to predict severity of construction accidents, Automation in Construction, 140, 104351, 2022.
25. 25. Bilal M., Ali G., Iqbal M.W., Anwar M., Malik M.S.A., Kadir R.A., Auto-Prep: efficient and automated data preprocessing pipeline, IEEE Access, 10, 107764-84, 2022.
26. 26. Bonidia R.P., Santos A.P.A., de Almeida B.L.S., Stadler P.F., da Rocha U.N., Sanches D.S., de Carvalho, A.C., BioAutoML: automated feature engineering and metalearning to predict noncoding RNAs in bacteria, Briefings in Bioinformatics, 23 (4), bbac218, 2022.
27. 27. He X., Zhao K., Chu X., AutoML: a survey of the state-of-the-art, Knowledge-Based Systems, 212, 106622, 2021.
28. 28. Adamczyk J., Malawski F., Comparison of manual and automated feature engineering for daily activity classification in mental disorder diagnosis, Computing & Informatics, 40 (4), 850-79, 2021.
29. 29. Elshawi R., Maher M., Sakr S., Automated machine learning: state-of-the-art and open challenges, arXiv preprint arXiv:1906.02287v2, 2019.
30. 30. Yu T., Zhu H., Hyper-parameter optimization: a review of algorithms and applications, arXiv preprint arXiv:2003.05689v1, 2020.
31. 31. Akinci T.C., Topsakal O., Wernerbach A., Machine learning-based wind speed time series analysis, 2022 Global Energy Conference (GEC), 391-4, 2022.
32. 32. Wadi S.A., Almasarweh M., Alsaraireh A.A., Predicting closed price time series data using ARIMA model, Modern Applied Science, 12 (11), 181, 2018.
33. 33. Choudhary A., Kumar S., Sharma M., Sharma K.P., A framework for data prediction and forecasting in WSN with auto ARIMA, Wireless Personal Communications, 123 (3), 2245-59, 2022.
34. 34. Siami-Namini S., Tavakoli N., Siami Namin A., A comparison of ARIMA and LSTM in forecasting time series, 17th IEEE International Conference on Machine Learning and Applications (ICMLA), 1394-1401, 2018.
35. 35. Chan W.N., Time series data mining: comparative study of ARIMA and prophet methods for forecasting closing prices of myanmar stock exchange, Journal of Computer Applications and Research, 1 (1), 2020.
36. 36. Adiga R., Forecasting the spread of COVID-19 with prophet model using Belgium dataset, International Journal of Advances in Soft Computing and Intelligent Systems (IJASCIS), 1 (1), 36-41, 2022.
37. 37. Al-Qazzaz R.A., Yousif S.A., High performance time series models using auto autoregressive integrated moving average, Indonesian Journal of Electrical Engineering and Computer Science (IJEECS), 27 (1), 422-430, 2022.
38. 38. Gupta S., Sharma D., Prediction of COVID-19 spread in world using pandemic dataset with application of auto ARIMA and SIR models, International Journal of Critical Infrastructures 18 (2), 148-58, 2022.
39. 39. Anyscale. Fast AutoML with FLAML + Ray Tune. https://www.anyscale.com/blog/fast-automl-with-flaml-ray-tune. Yayın tarihi Ağustos 24, 2021. Erişim tarihi Ocak 14, 2023.
40. 40. Wang C., Wu Q., Liu X., Quintanilla L., Automated machine learning & tuning with FLAML, In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 4828-4829, 2022.
41. 41. Patil P.S., Kappuram K., Rumao R., Bari P., Development of AMES: automated ML expert system, International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COM-IT-CON), 208-13, 2022.
42. 42. Derevitskii I.V., Mramorov N.D., Usoltsev S.D., Kovalchuk S.V., Hybrid bayesian network-based modeling: COVID-19-pneumonia case, Journal of Personalized Medicine, 12 (8), 1325, 2022.
43. 43. Hoell N., A survey of open source automation tools for data science predictions, arXiv preprint arXiv:2208.11792v1, 2022.
44. 44. Yenidoğan I., Çayir A., Kozan O., Dağ T., Arslan Ç., Bitcoin forecasting using ARIMA and PROPHET, 3rd International Conference on Computer Science and Engineering (UBMK), 621-4, 2018.
45. 45. Taylor S.J., Letham B., Forecasting at scale, The American Statistician, 72 (1), 37-45, 2018.
46. 46. Alsharef A., Sonia, Kumar K., Iwendi C., Time series data modeling using advanced machine learning and AutoML, Sustainability, 14 (22), 15292, 2022.
47. 47. Gandhi P. 7 libraries that help in time-series problems. https://towardsdatascience.com/7-libraries-that-help-in-time-series-problems-d59473e48ddd. Yayın tarihi Haziran 28, 2021. Erişim tarihi Ocak 24, 2023.
48. 48. Board of Governors of the Federal Reserve System (US), Industrial Production: Utilities: Electric and Gas Utilities (NAICS=2211,2), retrieved from FRED. https://fred.stlouisfed.org/series/IPG2211A2N. Güncellenme tarihi Şubat 15, 2023. Erişim tarihi Kasım 24, 2022.
49. 49. Cifuentes J., Marulanda G., Bello A., Reneses J., Air temperature forecasting using machine learning techniques: a review, Energies, 13 (16), 4215, 2020.
50. 50. González-Sopeña J.M., Pakrashi V., Ghosh B., An overview of performance evaluation metrics for short-term statistical wind power forecasting, Renewable and Sustainable Energy Reviews, 138, 110515, 2021.
51. 51. Witt S.F., Witt C.A., Modeling and forecasting demand in tourism, Londra: Academic Press., 1992.
52. 52. Lewis C.D., Industrial and business forecasting methods, Londra: Butterworths Publishing, 1982.