Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies

Mahmut Lütfullah Özbilen; Emre Eğriboz; Ruşen Halepmollası; İsmail Bilgen; Mehmet Haklıdır

Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies

Abstract

A global health emergency has been declared by WHO at the beginning of 2020 based on the increasing number of cases in the COVID-19 epidemic. Governments around the world have taken unprecedented measures. However, there is no guarantee that the measures taken are best to mitigate the effect of pandemic. We investigate the impact of government policies regarding interventions on deaths related to the COVID-19 and mitigation of the economic decline. In a simulation environment, we use Reinforcement Learning (RL) to explore the optimal policies to prevent COVID-19 outbreak. We use a specific simulator called PandemicSimulator which has detailed abilities to simulate spread of disease and people interactions at different locations. The simulator is utilized to train RL agents to take mitigation policies with minimum economic damage of the pandemic without exceeding the hospital capacity. We use Deep Q Networks to train the RL agent. We compare the performance of the our agent’s policy with the policy applied by the United Kingdom in terms of critical patients, deaths and economic damage. Results show that policies improved by the RL agent can help decision makers in the pandemic mitigation policies.

Keywords

References

1. Aleta, A., Martin-Corral, D., y Piontti, A. P., Ajelli, M., Litvinova, M., Chinazzi, M., Dean, N. E., Halloran, M. E., Longini Jr, I. M., Merler, S., et al. (2020). Modelling the impact of testing, contact tracing and house- hold quarantine on second waves of covid-19. Nature Human Behaviour, 4 (9), 964–971.
2. Awasthi, R., Guliani, K. K., Bhatt, A., Gill, M. S., Nagori, A., Kumaraguru, P., & Sethi, T. (2020). Vacsim: Learning effective strategies for covid19 vaccine distribution using reinforcement learning. arXiv preprint arXiv:2009.06602.
3. Arregui, S., Aleta, A., Sanz, J., & Moreno, Y. (2018). Projecting social contact matrices to different demographic structures. PLoS computational biology, 14 (12), e1006638.
4. Anderson, R. M., Heesterbeek, H., Klinkenberg, D., & Hollingsworth, T.D. (2020) How will country-based mitigation measures influence thecourse of the covid-19 epidemic? The lancet, 395 (10228), 931–934. Bednarski, B. P., Singh, A. D., & Jones, W. M. (2020). On collaborative re- inforcement learning to optimize the redistribution of critical medical supplies throughout the covid-19 pandemic. Journal of the American Medical Informatics Association.
5. Dervisoglu, H., Bilgen, I, Halepmollasi, R., Can, B., Haklidir, M., (2021) Unfairness of Deep Learning Methods Arising Gender Bias in Covid-19 Diagnosis of Medical Images. Artificial Intelligence Theory and Application, 2, (Special Issue), 81-94.
6. Elgin, C., Basbug, G., & Yalaman, A. (2020). Economic policy responses to a pandemic: Developing the covid-19 economic stimulus index. Covid Economics, 1 (3), 40–53.M.L. Ozbilen et al. Artificial Intelligence Theory and Applications: 3 (2021) 29-38
7. Gottesman, O., Johansson, F., Komorowski, M., Faisal, A., Sontag, D., Doshi- Velez, F., & Celi, L. A. (2019). Guidelines for reinforcement learning in healthcare. Nature medicine, 25 (1), 16–18.
8. Hu, C., Lovejoy, W. S., & Shafer, S. L. (1994). Comparison of some control strategies for threecompartment pk/pd models. Journal of Pharma- cokinetics and Biopharmaceutics, 22 (6), 525–550.

9. Jaiswal, A. K., Tiwari, P., Kumar, S., Gupta, D., Khanna, A., & Rodrigues,J. J. (2019). Identifying pneumonia in chest x-rays: A deep learning approach. Measurement, 145, 511–518.
10. Kompella*, V., Capobianco*, R., Jong, S., Browne, J., Fox, S., Meyers, L., Wurman, P., & Stone, P. (2020). Reinforcement learning for optimization of covid- 19 mitigation policies.
11. Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2018). Hybrid retrieval-generation reinforced agent for medical image report generation. arXiv preprint arXiv:1805.08298.
12. Liu, C. (2020). A microscopic epidemic model and pandemic prediction using multiagent reinforcement learning. arXiv preprint arXiv:2004.12959.
13. Martin-Calvo, D., Aleta, A., Pentland, A., Moreno, Y., & Moro, E. (2020) Effectiveness of social distancing strategies for protecting a community from a pandemic with a data driven contact network based on census and real-world mobility data. Complex. Dig.
14. Max Roser, E. O.-O., Hannah Ritchie, & Hasell, J. (2020). Coronavirus pan- demic (covid-19) [https://ourworldindata.org/coronavirus]. Our World in Data. 15. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I.,Wierstra, D., & Riedmiller, M. A. (2013). Playing atari with deep reinforcement learning. CoRR, abs/1312.5602. http://arxiv.org/abs/1312.5602
16. Najar, O., (2021) Hypothetical Framework For Early Detection of Covid19 From Symptomatic Information by Using Deep Learning. Artificial Intelligence Theory and Application, 2, (Special Issue), 115-121.
17. Organization, W. H. et al. (2020). Coronavirus disease 2019 (covid-19): Situation report, 82. Ozbilen, M., Egriboz, E., Halepmollasi, R., Bilgen, I, Haklidir, M., (2021) A Deep Reinforcement Learning Approach to Explore Optimal Policies for Covid-19 Pandemic Mitigation: Preliminary Analysis. Artificial Intelligence Theory and Application, 2, (Special Issue).
18. Schaefer, A. J., Bailey, M. D., Shechter, S. M., & Roberts, M. S. (2005). Modeling medical treatment using markov decision processes. Operations research and health care (pp. 593–612). Springer.
19. Taubenberger, J. K., & Morens, D. M. (2006). 1918 influenza: The mother of all pandemics. Revista Biomedica, 17 (1), 69–79.
20. Yu, C., Liu, J., & Nemati, S. (2019). Reinforcement learning in healthcare: A survey. arXiv preprint arXiv:1908.08796.

Details

Primary Language

English

Subjects

Clinical Sciences, Engineering

Journal Section

Short Report

Authors

Mahmut Lütfullah Özbilen ^* This is me
Türkiye

Emre Eğriboz This is me
Türkiye

Ruşen Halepmollası This is me
Türkiye

İsmail Bilgen This is me
Türkiye

Mehmet Haklıdır This is me
Türkiye

Publication Date

September 30, 2021

Submission Date

July 14, 2021

Acceptance Date

September 25, 2021

Published in Issue

Year 2021 Volume: 1 Number: 2

IZ

https://izlik.org/JA55UW93BZ

Cite

RIS / Bibtex

APA

Özbilen, M. L., Eğriboz, E., Halepmollası, R., Bilgen, İ., & Haklıdır, M. (2021). Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies. Artificial Intelligence Theory and Applications, 1(2), 29-38. https://izlik.org/JA55UW93BZ

AMA

1.Özbilen ML, Eğriboz E, Halepmollası R, Bilgen İ, Haklıdır M. Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies. AITA. 2021;1(2):29-38. https://izlik.org/JA55UW93BZ

Chicago

Özbilen, Mahmut Lütfullah, Emre Eğriboz, Ruşen Halepmollası, İsmail Bilgen, and Mehmet Haklıdır. 2021. “Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies”. Artificial Intelligence Theory and Applications 1 (2): 29-38. https://izlik.org/JA55UW93BZ.

EndNote

Özbilen ML, Eğriboz E, Halepmollası R, Bilgen İ, Haklıdır M (September 1, 2021) Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies. Artificial Intelligence Theory and Applications 1 2 29–38.

IEEE

[1]M. L. Özbilen, E. Eğriboz, R. Halepmollası, İ. Bilgen, and M. Haklıdır, “Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies”, AITA, vol. 1, no. 2, pp. 29–38, Sept. 2021, [Online]. Available: https://izlik.org/JA55UW93BZ

ISNAD

Özbilen, Mahmut Lütfullah - Eğriboz, Emre - Halepmollası, Ruşen - Bilgen, İsmail - Haklıdır, Mehmet. “Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies”. Artificial Intelligence Theory and Applications 1/2 (September 1, 2021): 29-38. https://izlik.org/JA55UW93BZ.

JAMA

1.Özbilen ML, Eğriboz E, Halepmollası R, Bilgen İ, Haklıdır M. Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies. AITA. 2021;1:29–38.

MLA

Özbilen, Mahmut Lütfullah, et al. “Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies”. Artificial Intelligence Theory and Applications, vol. 1, no. 2, Sept. 2021, pp. 29-38, https://izlik.org/JA55UW93BZ.

Vancouver

1.Mahmut Lütfullah Özbilen, Emre Eğriboz, Ruşen Halepmollası, İsmail Bilgen, Mehmet Haklıdır. Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies. AITA [Internet]. 2021 Sep. 1;1(2):29-38. Available from: https://izlik.org/JA55UW93BZ