Deep Reinforcement Learning for Simulation-Based Determination of COVID-19 Pandemic Mitigation Policies

Year 2021, Volume: 1 Issue: 2, 29 - 38, 30.09.2021

Mahmut Lütfullah Özbilen Emre Eğriboz Ruşen Halepmollası İsmail Bilgen Mehmet Haklıdır

Abstract

A global health emergency has been declared by WHO at the beginning of 2020 based on the increasing number of cases in the COVID-19 epidemic. Governments around the world have taken unprecedented measures. However, there is no guarantee that the measures taken are best to mitigate the effect of pandemic. We investigate the impact of government policies regarding interventions on deaths related to the COVID-19 and mitigation of the economic decline. In a simulation environment, we use Reinforcement Learning (RL) to explore the optimal policies to prevent COVID-19 outbreak. We use a specific simulator called PandemicSimulator which has detailed abilities to simulate spread of disease and people interactions at different locations. The simulator is utilized to train RL agents to take mitigation policies with minimum economic damage of the pandemic without exceeding the hospital capacity. We use Deep Q Networks to train the RL agent. We compare the performance of the our agent’s policy with the policy applied by the United Kingdom in terms of critical patients, deaths and economic damage. Results show that policies improved by the RL agent can help decision makers in the pandemic mitigation policies.

Keywords

reinforcement learning, deep learning, COVID-19, pandemic

References

• 1. Aleta, A., Martin-Corral, D., y Piontti, A. P., Ajelli, M., Litvinova, M., Chinazzi, M., Dean, N. E., Halloran, M. E., Longini Jr, I. M., Merler, S., et al. (2020). Modelling the impact of testing, contact tracing and house- hold quarantine on second waves of covid-19. Nature Human Behaviour, 4 (9), 964–971.
• 2. Awasthi, R., Guliani, K. K., Bhatt, A., Gill, M. S., Nagori, A., Kumaraguru, P., & Sethi, T. (2020). Vacsim: Learning effective strategies for covid19 vaccine distribution using reinforcement learning. arXiv preprint arXiv:2009.06602.
• 3. Arregui, S., Aleta, A., Sanz, J., & Moreno, Y. (2018). Projecting social contact matrices to different demographic structures. PLoS computational biology, 14 (12), e1006638.
• 4. Anderson, R. M., Heesterbeek, H., Klinkenberg, D., & Hollingsworth, T.D. (2020) How will country-based mitigation measures influence thecourse of the covid-19 epidemic? The lancet, 395 (10228), 931–934. Bednarski, B. P., Singh, A. D., & Jones, W. M. (2020). On collaborative re- inforcement learning to optimize the redistribution of critical medical supplies throughout the covid-19 pandemic. Journal of the American Medical Informatics Association.
• 5. Dervisoglu, H., Bilgen, I, Halepmollasi, R., Can, B., Haklidir, M., (2021) Unfairness of Deep Learning Methods Arising Gender Bias in Covid-19 Diagnosis of Medical Images. Artificial Intelligence Theory and Application, 2, (Special Issue), 81-94.
• 6. Elgin, C., Basbug, G., & Yalaman, A. (2020). Economic policy responses to a pandemic: Developing the covid-19 economic stimulus index. Covid Economics, 1 (3), 40–53.M.L. Ozbilen et al. Artificial Intelligence Theory and Applications: 3 (2021) 29-38
• 7. Gottesman, O., Johansson, F., Komorowski, M., Faisal, A., Sontag, D., Doshi- Velez, F., & Celi, L. A. (2019). Guidelines for reinforcement learning in healthcare. Nature medicine, 25 (1), 16–18.
• 8. Hu, C., Lovejoy, W. S., & Shafer, S. L. (1994). Comparison of some control strategies for threecompartment pk/pd models. Journal of Pharma- cokinetics and Biopharmaceutics, 22 (6), 525–550.
• 9. Jaiswal, A. K., Tiwari, P., Kumar, S., Gupta, D., Khanna, A., & Rodrigues,J. J. (2019). Identifying pneumonia in chest x-rays: A deep learning approach. Measurement, 145, 511–518.
• 10. Kompella*, V., Capobianco*, R., Jong, S., Browne, J., Fox, S., Meyers, L., Wurman, P., & Stone, P. (2020). Reinforcement learning for optimization of covid- 19 mitigation policies.
• 11. Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2018). Hybrid retrieval-generation reinforced agent for medical image report generation. arXiv preprint arXiv:1805.08298.
• 12. Liu, C. (2020). A microscopic epidemic model and pandemic prediction using multiagent reinforcement learning. arXiv preprint arXiv:2004.12959.
• 13. Martin-Calvo, D., Aleta, A., Pentland, A., Moreno, Y., & Moro, E. (2020) Effectiveness of social distancing strategies for protecting a community from a pandemic with a data driven contact network based on census and real-world mobility data. Complex. Dig.
• 14. Max Roser, E. O.-O., Hannah Ritchie, & Hasell, J. (2020). Coronavirus pan- demic (covid-19) [https://ourworldindata.org/coronavirus]. Our World in Data. 15. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I.,Wierstra, D., & Riedmiller, M. A. (2013). Playing atari with deep reinforcement learning. CoRR, abs/1312.5602. http://arxiv.org/abs/1312.5602
• 16. Najar, O., (2021) Hypothetical Framework For Early Detection of Covid19 From Symptomatic Information by Using Deep Learning. Artificial Intelligence Theory and Application, 2, (Special Issue), 115-121.
• 17. Organization, W. H. et al. (2020). Coronavirus disease 2019 (covid-19): Situation report, 82. Ozbilen, M., Egriboz, E., Halepmollasi, R., Bilgen, I, Haklidir, M., (2021) A Deep Reinforcement Learning Approach to Explore Optimal Policies for Covid-19 Pandemic Mitigation: Preliminary Analysis. Artificial Intelligence Theory and Application, 2, (Special Issue).
• 18. Schaefer, A. J., Bailey, M. D., Shechter, S. M., & Roberts, M. S. (2005). Modeling medical treatment using markov decision processes. Operations research and health care (pp. 593–612). Springer.
• 19. Taubenberger, J. K., & Morens, D. M. (2006). 1918 influenza: The mother of all pandemics. Revista Biomedica, 17 (1), 69–79.
• 20. Yu, C., Liu, J., & Nemati, S. (2019). Reinforcement learning in healthcare: A survey. arXiv preprint arXiv:1908.08796.