Performance Benchmarking of Classical Statistic, Machine Learning, and Deep Learning Time Series Models in Forecasting Measles Cases

Abstract

In the globalizing world, the reduction in travel time has facilitated the spread of infectious diseases, particularly those transmitted through contact and respiratory secretions. Measles, a highly contagious disease easily transmitted via respiratory droplets, continues to be a significant public health threat. Despite being largely preventable through vaccination, the measles virus remains endemic in regions with low vaccination rates, impacting public health in countries receiving migrants from these areas. Accurate computer-aided forecasting of measles outbreaks can assist policymakers in making informed decisions to prevent the spread of the disease. This study compares the performance of different time series models, including classical statistical methods, machine learning, and deep learning techniques, in forecasting the number of measles cases. For performance evaluation, a comparative analysis was conducted on datasets from Benin, Cameroon, and Nigeria. The forecasting performance of the models—ARIMA, HW, LSTM, Greykite, Prophet, and XGBoost—was assessed using RMSE, MAPE, MAE, and MSLE evaluation metrics. The models were trained on the first 147 months of data from each dataset, with their forecasting performance evaluated over the subsequent 12 months. The study results reveal that the XGBoost model achieved the lowest MSLE in predicting measles cases for Benin (0.08) and Nigeria (0.69), while the LSTM model performed best for Cameroon with an MSLE of 0.67. Using the developed computer-aided system, the next six months of measles cases were forecasted for these countries. To our best knowledge, this study is one of the first to benchmark different time series models, using diverse datasets in forecasting measles cases. The findings suggest that artificial intelligence-based prediction systems can play a crucial role in preventing the spread of infectious diseases like measles and in developing effective health policies.

Keywords

• Measles
• Time Series Analysis
• Machine Learning
• Deep Learning
• Benchmarking.

Ethical Statement

The study is complied with research and publication ethics.

References

1. D.E. Griffin "Measles Vaccine," Viral Immunology, 31(2), 86-95, 2018. https://doi.org/10.1089/vim.2017.0143.
2. A. A. Minta, M. Ferrari, S. Antoni, A. Portnoy, A. Sbarra, B. Lambert, C. Hatcher, C. H. Hsu, L.L. Ho, C. Steulet, M. Gacic-Dobo, P.A. Rota, M.N. Mulders, A.S. Bose, W.P. Caro, P. O’Connor, N.S. Crowcroft "Progress Toward Measles Elimination — Worldwide, 2000–2022," US Department of Health and Human Services | Centers for Disease Control and Prevention | Morbidity and Mortality Weekly Report (MMWR), 72(46), 1262-1268, 2023. http://dx.doi.org/10.15585/mmwr.mm7246a3.
3. R. A. Bednarczyk, W. A. Orenstein, S. B. Omer "Estimating the Number of Measles-Susceptible Children and Adolescents in the United States Using Data From the National Immunization Survey–Teen (NIS-Teen)," American Journal of Epidemiology, 184(2), 148-156, 2016. https://doi.org/10.1093/aje/kwv320.
4. World Health Organization, "Measles vaccines: WHO position paper, April 2017 – Recommendations," Vaccine, 37(2), 219-222, 2019. https://doi.org/10.1016/j.vaccine.2017.07.066.
5. World Health Organization, “Measles”, https://www.who.int/news-room/fact-sheets/detail/measles.
6. A.D. Mathis, K. Raines, N.B. Masters, T.D. Filardo, G. Kim, S.N. Crooke, B. Bankamp, P.A. Rota, D.E. Sugerman "Measles — United States, January 1, 2020–March 28, 2024," Centers for Disease Control and Prevention Morbidity and Mortality Weekly Report, 73(14), 295-300, 2024. https://doi.org/10.15585%2Fmmwr.mm7314a1.
7. A. Hussain, S. Ali, M. Ahmed, S. Hussain " The Anti-vaccination Movement: A Regression in Modern Medicine," Cureus, 10(7), 2018. https://doi.org/10.7759%2Fcureus.2919.
8. E. I. Iseri, K. Uyar, E. U. Ilhan "Forecasting Measles in the European Union Using the Adaptive Neuro-Fuzzy Inference System," Cyprus Journal Of Medical Sciences, 4, 34-37, 2019. http://dx.doi.org/10.5152/cjms.2019.611.

9. K. Uyar, U. Ilhan, E. I. Iseri, A. Ilhan "Forecasting Measles Cases in Ethiopia using Neuro-Fuzzy Systems," International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), 2019. https://doi.org/10.1109/ISMSIT.2019.8932882.
10. S. Mehrmolaei, M.R. Keyvanpour "Time series forecasting using improved ARIMA," IEEE 2016 Artificial Intelligence and Robotics (IRANOPEN), 92-97, 2016. https://doi.org/10.1109/RIOS.2016.7529496.
11. H.S. Siddalingaiah, A. Chaudhuri, D. Chandrakala "Measles occurrence, vaccination coverages and malnutrition in India: correlations, trends, and projections by time series analysis," International Journal of Community Medicine and Public Health, 5(1), 86-94, 2018. https://doi.org/10.18203/2394-6040.ijcmph20175532.
12. A. Akinbobola, A. S. Hamisu "Predicting Measles Occurrence Using Some Weather Variables in Kano, North western Nigeria," American Journal of Public Health Research, 6(4), 195-202, 2018. http://dx.doi.org/10.12691/ajphr-6-4-4.
13. C.E. Okorie, C.C. Ilomuanya, O.A. Bamigbala "Statistical Analysis On Impact Of Measles Among Children In Obudu Local Government Area Of Cross River State," Journal Health And Technology, 2(2), 2023. https://doi.org/10.47820/jht.v2i2.36.
14. S. Sharmin, I. Rayhan "Modelling of Infectious Diseases for Providing Signalof Epidemics: A Measles Case Study in Bangladesh," Journal of Health, Population and Nutrition, 29(6), 567-573, 2011. https://doi.org/10.3329/jhpn.v29i6.9893.
15. L. Samaras, M.A. Sicilia, E.G. Barriocanal "Predicting epidemics using search engine data: a comparative study on measles in the largest countries of Europe," BMC Public Health, 21(100), 157-168, 2021. https://doi.org/10.1186/s12889-020-10106-8.
16. H. Talirongan, M.Y. Orong, F.J.B. Talirongan "Alleviating Vulnerabilities of the Possible Outbreaks of Measles: A Data Trend Analysis and Prediction of Possible Cases," Mediterranean Journal of Basic and Applied Sciences (MJBAS), 4(4), 129-135, 2020. http://doi.org/10.46382/MJBAS.2020.4405.
17. Y. Alimohamadi, M. Sepandi "Forty-seven year trend of measles in Iran: An interrupted time series analysis," Health Science Reports, 6(2), 2023. https://doi.org/10.1002/hsr2.1139.
18. E. Yang, D. Yan, Q. Xu, Z. Wang, S. Liu "Nonlinear combination forecasting of measles incidence in Shenyang based on General Regression Neural Network," IEEE 2020 Chinese Control And Decision Conference (CCDC), 3015-3020, 2020. https://doi.org/10.1109/CCDC49329.2020.9164712.
19. R. Komitova, A. Kevorkyan, O. Boykinova, S. Krumova, M. Atanasova, R. Raycheva, Y. Stoilova, A. Kunchev "Difficulties in achieving and maintaining the goal of measles elimination in Bulgaria," Revue d'Épidémiologie et de Santé Publique, 67(3), 155-162, 2019. https://doi.org/10.1016/j.respe.2019.01.120.
20. R.T. Alegado, G.M. Tumibay "Forecasting Measles Immunization Coverage Using ARIMA Model," Journal of Computer and Communications, 7(10), 157-168, 2019. https://doi.org/10.4236/jcc.2019.710015.
21. T.C. Maradze, S.P. NYONI, T. NYONI "Modelling and Forecasting Immunization against Measles Disease in Nigeria Using Artificial Neural Networks (ANN)," International Research Journal of Innovations in Engineering and Technology (IRJIET), 5(3), 571-575, 2021. https://doi.org/10.47001/IRJIET/2021.503097.
22. K.A. Gyasi-Agyei, W.O. Denteh, A. Gyasi-Agyei "Analysis and Modeling of Prevalence of Measles in the Ashanti Region of Ghana," British Journal of Mathematics & Computer Science, 7(10), 209-225, 2013. https://doi.org/10.4236/jcc.2019.710015.
23. World Health Organization, “Distribution of measles cases by country and by month”. https://immunizationdata.who.int/global?topic=&location=,[Accessed at April 16, 2024].
24. P. Cihan "Forecasting of Monkeypox Cases in the World Using the ARIMA Model," European Journal of Science and Technology, 46, 37-45, 2023. https://doi.org/10.31590/ejosat.1190981.
25. P. Cihan "Forecasting fully vaccinated people against COVID-19 and examining future vaccination rate for herd immunity in the US, Asia, Europe, Africa, South America, and the World," Applied Soft Computing, 111, 2021. https://doi.org/10.1016%2Fj.asoc.2021.107708.
26. P. Cihan "ARIMA-Based Forecasting of Total COVID-19 Cases in the USA and India," Signal Processing and Communications Applications Conference (SIU), June 2021. https://doi.org/10.1109/SIU53274.2021.9477773.
27. P. Cihan "The machine learning approach for predicting the number of intensivecare, intubated patients and death: The COVID-19 pandemic in Turkey," Sigma Journal of Engineering and Natural Sciences, 40(1), 85-94 2022. https://dx.doi.org/10.14744/sigma.2022.00007.
28. P. Cihan "Comparative Performance Analysis of Deep Learning, Classical, and Hybrid Time Series Models in Ecological Footprint Forecasting," Applied Sciences, 14(4), 2024. https://doi.org/10.3390/app14041479.
29. P. Cihan "Impact of the COVID-19 lockdowns on electricity and natural gas consumption in the different industrial zones and forecasting consumption amounts: Turkey case study," International Journal of Electrical Power and Energy Systems, 134, 2022. https://doi.org/10.1016%2Fj.ijepes.2021.107369.
30. C.P.D. Veiga, C.R.P.D. Veiga, A. Catapan, U. Tortato, W.V.D. Silva "Demand forecasting in food retail: a comparison between the HoltWinters and ARIMA models," Wseas Transactions on Business and Economics, 11, 608-614, 2014. https://wseas.com/journals/bae/2014/a085707-276.pdf.
31. F. Shahid, A. Zameer, M. Muneeb " Predictions for COVID-19 with deep learning models of LSTM, GRU and Bi-LSTM," Chaos, Solitons & Fractals, 140, 2020. https://doi.org/10.1016/j.chaos.2020.110212.
32. R. Hosseini, A. Chen, K. Yang, S. Patra, Y. Su, S.E.A. Orjany, S. Tang, P. Ahammad "Greykite: Deploying Flexible Forecasting at Scale at LinkedIn," Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 3007-3017, 2022. https://doi.org/10.1145/3534678.3539165.
33. G. Battineni, N. Chintalapudi, F. Amenta "Forecasting of COVID-19 epidemic size in four high hitting nations (USA, Brazil, India and Russia) by Fb-Prophet machine learning model," Applied Computing and Informatics, 2020. https://doi.org/10.1108/ACI-09-2020-0059.
34. Z. Fang, S. Yang, C. Lv, S. An, W. Wu "Application of a data-driven XGBoost model for the prediction of COVID-19 in the USA: a time-series study," BMJ Open, 12(7), 2022. https://doi.org/10.1136/bmjopen-2021-056685.
35. P. Cihan "Fuzzy Rule-Based System for Predicting Daily Case in COVID-19 Outbreak," 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), November 2020. https://doi.org/10.1109/ISMSIT50672.2020.9254714.
36. O. Istaiteh, T. Owais, N. Al-Madi, S. Abu-Soud "Machine Learning Approaches for COVID-19 Forecasting," 2020 International Conference on Intelligent Data Science Technologies and Applications (IDSTA), November 2020. https://doi.org/10.1109/IDSTA50958.2020.9264101.