A Vision Transformer-based Approach for Automatic COVID-19 Diagnosis on Chest X-ray Images

Abstract

The new type of coronavirus disease (COVID-19), which has emerged in recent years, has become a serious disease that threatens health worldwide. COVID-19, which can be transmitted very quickly and with serious increases in death, has paved the way for many concerns. With the spread of the epidemic to a universal dimension, many studies have been carried out for the early diagnosis of this disease. With early diagnosis, both fatal cases are prevented and the planning of the epidemic can be easier. The fact that X-ışını images are much more advantageous than other imaging techniques in terms of time and applicability, and also that they are economical, has led to the focus of early diagnosis-based applications and methods on these images. Deep learning approaches have had a great impact in the diagnosis of COVID-19, as in the diagnosis of many diseases. In this study, we propose a diagnostic system based on the transformer method, which is the most up-to-date and much more popular architecture than previous techniques of deep learning such as CNN-based approaches. This method includes an approach based on vision transformer models and a more effective diagnosis of COVID-19 disease on a new dataset, the COVID-QU-Ex dataset. In experimental studies, it has been observed that vision transformer models are more successful than CNN models. In addition, the ViT-L16 model showed a much higher performance compared to similar studies in the literature, providing test accuracy and F1-score of over 96%.

Keywords

• COVID-19
• COVID-19 detection
• Vision transformers
• Lung disease detection
• Deep learning

Göğüs Röntgeni Görüntülerinden Otomatik COVID-19 Teşhisi için Görü Transformatörüne Dayalı Bir Yaklaşım

Abstract

Son yıllarda ortaya çıkan yeni tip Koronavirüs hastalığı (COVID-19), dünya çapında sağlığı tehdit eden ciddi bir hastalık olmuştur. COVID-19 çok hızlı bir şekilde bulaşabilen ve ciddi ölüm artışları ile birçok endişeye zemin hazırlamıştır. Salgının evrensel boyuta taşınmasıyla bu hastalığın erken teşhisine yönelik birçok çalışma yapılmıştır. Erken teşhis ile hem ölümcül vakaların önüne geçilmiş olunmakta hem de salgının planlanması daha kolay olabilmektedir. X-ışını görüntülerinin zaman ve uygulanabilirlik açısından diğer görüntüleme tekniklerine nazaran çok daha avantajlı olması ve ayrıca ekonomik olması erken teşhis bazlı uygulama ve yöntemlerin bu görüntülerin üzerine yoğunlaşmasına neden olmuştur. Derin öğrenme yaklaşımları birçok hastalık teşhisinde olduğu gibi COVID-19 teşhisinde de çok büyük bir etki oluşturmuştur. Bu çalışmada, derin öğrenmenin CNN tabanlı yaklaşımları gibi daha önceki tekniklerinden ziyade en güncel ve çok daha popüler bir mimarisi olan transformatör yöntemine dayalı bir teşhis sistemi önerdik. Bu sistem, görü transformatör modelleri temelli bir yaklaşım ve yeni bir veri seti olan COVID-QU-Ex üzerinde COVID-19 hastalığının daha efektif bir teşhisini içermektedir. Deneysel çalışmalarda, görü transformatör modellerinin CNN modellerinden daha başarılı olduğu gözlemlenmiştir. Ayrıca, ViT-L16 modeli %96’nın üzerinde test doğruluğu ve F1-skoru sunarak, literatürde benzer çalışmalara kıyasla çok daha yüksek bir başarım göstermiştir.

Keywords

• COVID-19
• COVID-19 tespiti
• Görü transformatörler
• Akciğer hastalık tespiti
• Derin Öğrenme

References

1. Abdul Gafoor, S., Sampathila, N., Madhushankara, M., & Swathi, K. S. (2022). Deep learning model for detection of COVID-19 utilizing the chest X-ışını images. Cogent Engineering, 9(1). https://doi.org/10.1080/23311916.2022.2079221
2. ADEM, K., & KILIÇARSLAN, S. (2021). COVID-19 Diagnosis Prediction in Emergency Care Patients using the Convolutional Neural Network. Afyon Kocatepe University Journal of Sciences and Engineering, 21, 300–309. https://doi.org/10.35414/akufemubid.788898
3. Alici-Karaca, D., Akay, B., Yay, A., Suna, P., Nalbantoglu, O. U., Karaboga, D., … Baran, M. (2022). A new lightweight convolutional neural network for radiation-induced liver disease classification. Biomedical Signal Processing and Control, 73. https://doi.org/10.1016/j.bspc.2021.103463
4. ARI, D., & ALAGÖZ, B. B. (2021). A Review of Genetic Programming Popular Techniques, Fundamental Aspects, Software Tools and Applications. Sakarya University Journal of Science. https://doi.org/10.16984/saufenbilder.793333
5. Bayat, S., & Işık, G. (2022). Recognition of Aras Bird Species From Their Voices With Deep Learning Methods. Journal of the Institute of Science and Technology, 12(3): 1250 - 1263.
6. Bhattacharyya, A., Bhaik, D., Kumar, S., Thakur, P., Sharma, R., & Pachori, R. B. (2022). A deep learning based approach for automatic detection of COVID-19 cases using chest X-ışını images. Biomedical Signal Processing and Control, 71. https://doi.org/10.1016/j.bspc.2021.103182
7. Burukanli, M., Çibuk, M., & Budak, Ü. (2021). Saldırı Tespiti için Makine Öğrenme Yöntemlerinin Karşılaştırmalı Analizi Comparative Analysis of Machine Learning Methods for Intrusion Detection. In BEU Journal of Science (Vol. 10).
8. Bülbül, M. A., & Öztürk, C. (2022). Optimization, modeling and implementation of plant water consumption control using genetic algorithm and artificial neural network in a hybrid structure. Arabian Journal for Science and Engineering, 47(2), 2329-2343.

9. Ciotti, M., Ciccozzi, M., Terrinoni, A., Jiang, W. C., Wang, C. bin, & Bernardini, S. (2020). The COVID-19 pandemic. Critical Reviews in Clinical Laboratory Sciences, pp. 365–388. Taylor and Francis Ltd. https://doi.org/10.1080/10408363.2020.1783198
10. Cleverley, J., Piper, J., & Jones, M. M. (2020, July 16). The role of chest radiography in confirming covid-19 pneumonia. The BMJ, Vol. 370. BMJ Publishing Group. https://doi.org/10.1136/bmj.m2426
11. Deb, S. D., Jha, R. K., Jha, K., & Tripathi, P. S. (2022). A multi model ensemble based deep convolution neural network structure for detection of COVID19. Biomedical Signal Processing and Control, 71. https://doi.org/10.1016/j.bspc.2021.103126
12. Dhiman, G., Chang, V., Kant Singh, K., & Shankar, A. (2022). ADOPT: automatic deep learning and optimization-based approach for detection of novel coronavirus COVID-19 disease using X-ışını images. Journal of Biomolecular Structure and Dynamics, 40(13), 5836–5847. https://doi.org/10.1080/07391102.2021.1875049
13. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., … Houlsby, N. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Retrieved from http://arxiv.org/abs/2010.11929
14. Fang, Y., Zhang, H., Xie, J., Lin, M., Ying, L., Pang, P., & Ji, W. (2020, August 1). Sensitivity of chest CT for COVID-19: Comparison to RT-PCR. Radiology, Vol. 296, pp. E115–E117. Radiological Society of North America Inc. https://doi.org/10.1148/radiol.2020200432
15. Gulum, M. A., Trombley, C. M., & Kantardzic, M. (2021). A review of explainable deep learning cancer detection models in medical imaging. Applied Sciences (Switzerland), 11(10). https://doi.org/10.3390/app11104573
16. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem, 770–778. https://doi.org/10.1109/CVPR.2016.90
17. Ibrahim, D. M., Elshennawy, N. M., & Sarhan, A. M. (2021). Deep-chest: Multi-classification deep learning model for diagnosing COVID-19, pneumonia, and lung cancer chest diseases. Computers in Biology and Medicine, 132, 104348. https://doi.org/10.1016/j.compbiomed.2021.104348
18. Kanne, J. P., Bai, H., Bernheim, A., Chung, M., Haramati, L. B., Kallmes, D. F., … Sverzellati, N. (2021, June 1). COVID-19 imaging: What we know now and what remains unknown. Radiology, Vol. 299, pp. E262–E279. Radiological Society of North America Inc. https://doi.org/10.1148/radiol.2021204522
19. Karaman, A., Karaboga, D., Pacal, I., Akay, B., Basturk, A., Nalbantoglu, U., Sahin, O. (2022). Hyper-parameter optimization of deep learning architectures using artificial bee colony (ABC) algorithm for high performance real-time automatic colorectal cancer (CRC) polyp detection. Applied Intelligence. https://doi.org/10.1007/s10489-022-04299-1
20. Karaman, A., Pacal, I., Basturk, A., Akay, B., Nalbantoglu, U., Coskun, S., Sahin, O., & Karaboga, D. (2023). Robust real-time polyp detection system design based on YOLO algorithms by optimizing activation functions and hyper-parameters with artificial bee colony (ABC). Expert Systems with Applications, 221. https://doi.org/10.1016/j.eswa.2023.119741
21. Ozkok, F. O., & Celik, M. (2022). A hybrid CNN-LSTM model for high resolution melting curve classification. Biomedical Signal Processing and Control, 71, 103168. https://doi.org/10.1016/J.BSPC.2021.103168
22. PACAL, İ. (2022). Deep Learning Approaches for Classification of Breast Cancer in Ultrasound (US) Images. Journal of the Institute of Science and Technology, 1917–1927. https://doi.org/10.21597/jist.1183679
23. Pacal, I., & Karaboga, D. (2021). A robust real-time deep learning based automatic polyp detection system. Computers in Biology and Medicine, 134. https://doi.org/10.1016/J.COMPBIOMED.2021.104519
24. Pacal, I., Karaboga, D., Basturk, A., Akay, B., & Nalbantoglu, U. (2020). A comprehensive review of deep learning in colon cancer. Computers in Biology and Medicine, 126. https://doi.org/10.1016/J.COMPBIOMED.2020.104003
25. Pacal, I., Karaman, A., Karaboga, D., Akay, B., Basturk, A., Nalbantoglu, U., & Coskun, S. (2022). An efficient real-time colonic polyp detection with YOLO algorithms trained by using negative samples and large datasets. Computers in Biology and Medicine, 141. https://doi.org/10.1016/J.COMPBIOMED.2021.105031
26. Pascarella, G., Strumia, A., Piliego, C., Bruno, F., del Buono, R., Costa, F., … Agrò, F. E. (2020a, August 1). COVID-19 diagnosis and management: a comprehensive review. Journal of Internal Medicine, Vol. 288, pp. 192–206. Blackwell Publishing Ltd. https://doi.org/10.1111/joim.13091
27. Pascarella, G., Strumia, A., Piliego, C., Bruno, F., del Buono, R., Costa, F., … Agrò, F. E. (2020b, August 1). COVID-19 diagnosis and management: a comprehensive review. Journal of Internal Medicine, Vol. 288, pp. 192–206. Blackwell Publishing Ltd. https://doi.org/10.1111/joim.13091
28. Revel, M.-P., Parkar, A. P., Prosch, H., Silva, M., Sverzellati, N., Gleeson, F., & Brady, A. (n.d.). COVID-19 patients and the radiology department-advice from the European Society of Radiology (ESR) and the European Society of Thoracic Imaging (ESTI). https://doi.org/10.1007/s00330-020-06865-y/Published
29. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., … Fei-Fei, L. (2015). ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115(3), 211–252. https://doi.org/10.1007/s11263-015-0816-y
30. Sedik, A., Hammad, M., Abd El-Samie, F. E., Gupta, B. B., & Abd El-Latif, A. A. (2022). Efficient deep learning approach for augmented detection of Coronavirus disease. Neural Computing and Applications, 34(14), 11423–11440. https://doi.org/10.1007/s00521-020-05410-8
31. Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 1–14.
32. Subramanian, N., Elharrouss, O., Al-Maadeed, S., & Chowdhury, M. (2022, April 1). A review of deep learning-based detection methods for COVID-19. Computers in Biology and Medicine, Vol. 143. Elsevier Ltd. https://doi.org/10.1016/j.compbiomed.2022.105233
33. Tahir, A. M., Chowdhury, M. E. H., Khandakar, A., Rahman, T., Qiblawey, Y., Khurshid, U., … Hamid, T. (2021). COVID-19 infection localization and severity grading from chest X-ışını images. Computers in Biology and Medicine, 139. https://doi.org/10.1016/j.compbiomed.2021.105002
34. Wang, J., Zhu, H., Wang, S. H., & Zhang, Y. D. (2021). A Review of Deep Learning on Medical Image Analysis. Mobile Networks and Applications, 26(1), 351–380. https://doi.org/10.1007/s11036-020-01672-7
35. Wang, T., Lei, Y., Fu, Y., Wynne, J. F., Curran, W. J., Liu, T., & Yang, X. (2021). A review on medical imaging synthesis using deep learning and its clinical applications. Journal of Applied Clinical Medical Physics, 22(1), 11–36. https://doi.org/10.1002/acm2.13121
36. Xie, X., Zhong, Z., Zhao, W., Zheng, C., Wang, F., & Liu, J. (2020). Chest CT for Typical Coronavirus Disease 2019 (COVID-19) Pneumonia: Relationship to Negative RT-PCR Testing. Radiology, 296(2), E41–E45. https://doi.org/10.1148/radiol.2020200343