Stroke Classification in Brain Computed Tomography Images Using Vision Transformers and GAN-based Data Augmentation

Öz

This study presents an innovative approach to stroke classification. The research utilizes brain computed tomography (CT) images to distinguish between three classes: “no stroke” “ischemic stroke” and “hemorrhagic stroke” employing Vision Transformers (ViTs), a deep learning-based method incorporating attention mechanisms. In this work, ViTs were effectively applied as a powerful method for image-based classification. To enhance model performance, various training strategies and data augmentation techniques were implemented. Specifically, GAN-based architectures such as SRGAN (Super-Resolution GAN) and BSRGAN (Blind Super-Resolution GAN) were used to expand the dataset and improve its diversity. These GAN-based augmentation techniques significantly improved the model’s overall performance and classification accuracy. The Vision Transformer model was rigorously evaluated through multi-class classification tasks using a range of performance metrics. In the three-class classification task, the model achieved 99.06% accuracy, 98.18% precision, 98.94% recall, and a 98.54% F1-score. For the binary classification of ischemic vs. hemorrhagic stroke, the model reported 99.78% accuracy, 99.02% precision, 99.66% recall, and a 99.26% F1-score. In the binary classification of stroke presence, the model achieved 98.68% accuracy, 97.80% precision, 98.54% recall, and a 98.14% F1-score. These findings demonstrate the potential of Vision Transformers to assist in faster and more reliable stroke diagnosis and highlight their contribution to the development of decision support systems in medical applications.

Anahtar Kelimeler

• Stroke classification
• vision transformers
• CT imaging
• data augmentation
• deep learning

Beyin Bilgisayarlı Tomografi Görüntülerinde Görü Dönüştürücüler ve GAN Tabanlı Veri Artırma Kullanılarak İnme Sınıflandırması

Öz

Bu çalışma, inme sınıflandırması için yenilikçi bir yaklaşım sunmaktadır. Araştırmada, beyin bilgisayarlı tomografi (BT) görüntüleri kullanılarak “inme yok”, “iskemik inme” ve “hemorajik inme” olmak üzere üç farklı sınıfı ayırt etmeyi amaçlayan, dikkat mekanizmaları içeren derin öğrenme tabanlı bir yöntem olan Görü Dönüştürücüler (GD) uygulanmıştır. GD modelleri, bu çalışmada görüntü verisi sınıflandırması için güçlü ve etkili bir yöntem olarak kullanılmıştır. Modelin performansını artırmak amacıyla çeşitli eğitim stratejileri ve veri artırma teknikleri uygulanmıştır. Özellikle, GAN tabanlı SRGAN (Süper Çözünürlük GAN) ve BSRGAN (Kör Süper Çözünürlük GAN) mimarileri, veri setini genişletmek ve çeşitliliği artırmak için kullanılmıştır. Bu GAN tabanlı artırma teknikleri, modelin genel başarımını ve sınıflandırma doğruluğunu önemli ölçüde iyileştirmiştir. Görü Dönüştürücü modeli, çok sınıflı sınıflandırma görevleri kapsamında çeşitli performans ölçütleriyle kapsamlı biçimde değerlendirilmiştir. Üç sınıflı sınıflandırma görevinde model, %99,06 doğruluk, %98,18 hassasiyet, %98,94 duyarlılık ve %98,54 F1 skoru elde etmiştir. Hemorajik ve iskemik inme sınıflandırmasında modelin doğruluğu %99,78, hassasiyeti %99,02, duyarlılığı %99,66 ve F1 skoru %99,26 olarak raporlanmıştır. İkili “inme var/yok” sınıflandırmasında ise model, %98,68 doğruluk, %97,80 hassasiyet, %98,54 duyarlılık ve %98,14 F1 skoru elde etmiştir. Bu bulgular, Görü Dönüştürücüler’in hızlı ve güvenilir inme teşhisine katkı sunma potansiyelini ve tıbbi uygulamalarda karar destek sistemlerinin gelişimine önemli ölçüde katkı sağlayabileceğini göstermektedir.

Anahtar Kelimeler

• İnme sınıflandırması
• görü dönüştürücüler
• BT görüntüleme
• veri artırma
• derin öğrenme

Kaynakça

1. World Health Organization. The top 10 causes of death. Retrieved from https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death, 2018.
2. Benjamin EJ, Blaha MJ, Chiuve SE, Cushman M, Das SR, Deo R et al. Heart disease and stroke statistics—2017 update: a report from the American Heart Association. Circulation 2017; 135(10): e146-e603.
3. Donnan GA, Fisher M, Macleod M, Davis SM. Stroke. Lancet 2008; 371(9624): 1612-1623.
4. González RG. Clinical MRI of acute ischemic stroke. J Magn Reson Imaging 2012; 36(2): 259-271.
5. Akbarzadeh MA, Sanaie S, Kuchaki Rafsanjani M, Hosseini MS. Role of imaging in early diagnosis of acute ischemic stroke: a literature review. Egypt J Neurol Psychiatr Neurosurg 2021; 57: 1-8.
6. Moonis M, Fisher M. Imaging of acute stroke. Cerebrovasc Dis 2001; 11(3): 143-150.
7. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88.
8. Suganyadevi S, Seethalakshmi V, Balasamy K. A review on deep learning in medical image analysis. Int J Multimed Inf Retr 2022; 11(1): 19-38.

9. Gautam A, Raman B. Towards effective classification of brain hemorrhagic and ischemic stroke using CNN. Biomed Signal Process Control 2021; 63: 102178.
10. Neethi AS, Niyas S, Kannath SK, Mathew J, Anzar AM, Rajan J. Stroke classification from computed tomography scans using 3D convolutional neural network. Biomed Signal Process Control 2022; 76: 103720.
11. Shakunthala M, HelenPrabha K. Classification of ischemic and hemorrhagic stroke using Enhanced-CNN deep learning technique. J Intell Fuzzy Syst 2023; Preprint: 1-16.
12. Chen R, Cai Y, Wu J, Liu H, Peng Z, Xie Y et al. Artificial intelligence-based identification of brain CT medical images. In: AOPC 2022: Biomedical Optics; January 2023. Vol. 12560. pp. 53-58.
13. Wieser M, Siegismund D, Heyse S, Steigele S. Vision transformers show improved robustness in high-content image analysis. In: 2022 9th Swiss Conference on Data Science (SDS); June 2022. pp. 71-72.
14. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al. Attention is all you need. In: Advances in Neural Information Processing Systems; 2017. Vol. 30.
15. Dosovitskiy A. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
16. Takahashi S, Sakaguchi Y, Kouno N, Takasawa K, Ishizu K, Akagi Y et al. Comparison of vision transformers and convolutional neural networks in medical image analysis: A systematic review. J Med Syst 2024; 48(1): 1-22.
17. Azad R, Kazerouni A, Heidari M, Aghdam EK, Molaei A, Jia Y et al. Advances in medical image analysis with vision transformers: a comprehensive review. Med Image Anal 2024; 91: 103000.
18. Simo AMD, Kouanou AT, Monthe V, Nana MK, Lonla BM. Introducing a deep learning method for brain tumor classification using MRI data towards better performance. Inform Med Unlocked 2024; 44: 101423.
19. Chavva IR, Crawford AL, Mazurek MH, Yuen MM, Prabhat AM, Payabvash S et al. Deep learning applications for acute stroke management. Ann Neurol 2022; 92(4): 574-587.
20. Okimoto N, Yasaka K, Fujita N, Watanabe Y, Kanzawa J, Abe O. Deep learning reconstruction for improving the visualization of acute brain infarct on computed tomography. Neuroradiology 2024; 66(1): 63-71.
21. Zhu G, Chen H, Jiang B, Chen F, Xie Y, Wintermark M. Application of deep learning to ischemic and hemorrhagic stroke computed tomography and magnetic resonance imaging. Semin Ultrasound CT MR 2022; 43(2): 147-152.
22. Miyamoto N, Ueno Y, Yamashiro K, Hira K, Kijima C, Kitora N et al. Stroke classification and treatment support system artificial intelligence for usefulness of stroke diagnosis. Front Neurol 2023; 14: 1295642.
23. Shakunthala M, HelenPrabha K. Classification of ischemic and hemorrhagic stroke using Enhanced-CNN deep learning technique. J Intell Fuzzy Syst 2023; Preprint: 1-16.
24. Altıntaş M, Öziç MÜ. Performance evaluation of different deep learning models for classifying ischemic, hemorrhagic, and normal computed tomography images: transfer learning approaches. Konya J Eng Sci 2024; 12(2): 465-477.
25. Koska İÖ, Koska Ç, Fernandes A. Automatic stroke classification: Domain knowledge injection augmented transfer learning approach. Anatol Clin J Med Sci 2024; 29(3): 260-267.
26. Çınar N, Kaya B, Kaya M. Classification of brain ischemia and hemorrhagic stroke using a hybrid method. In: 2023 4th International Conference on Data Analytics for Business and Industry (ICDABI); IEEE; 2023. pp. 279-284.
27. Katar O, Yıldırım O, Eroğlu Y. Vision transformer model for efficient stroke detection in neuroimaging. In: 2023 4th International Informatics and Software Engineering Conference (IISEC); IEEE; 2023. pp. 1-6.
28. Çınar N, Kaya B, Kaya M. Brain stroke detection from CT images using transfer learning method. In: 2023 13th International Conference on Advanced Computer Information Technologies (ACIT); IEEE; 2023. pp. 595-599.
29. Azad R, Kazerouni A, Heidari M, Aghdam EK, Molaei A, Jia Y et al. Advances in medical image analysis with vision transformers: a comprehensive review. Med Image Anal 2024; 91: 103000.
30. Henry EU, Emebob O, Omonhinmin CA. Vision transformers in medical imaging: A review. arXiv preprint arXiv:2211.10043, 2022.
31. Li Z, Wang Y, Yu J, Guo Y, Cao L, Gao J et al. Brain MRI image classification for Alzheimer’s disease diagnosis based on DenseNet and Vision Transformer. arXiv preprint arXiv:2103.03732, 2021.
32. Liang S, Zhang W, Gu Y. A hybrid and fast deep learning framework for COVID-19 detection via 3D chest CT images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2021. pp. 508-512.
33. Xia Y, Yao J, Lu L, Huang L, Xie G, Xiao J et al. Effective pancreatic cancer screening on non-contrast CT scans via anatomy-aware transformers. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2021, 24th International Conference; 27 September–1 October 2021; Strasbourg, France. Springer International Publishing; 2021. Vol. 24. pp. 259-269.
34. Ayoub M, Liao Z, Hussain S, Li L, Zhang CW, Wong KK. End-to-end vision transformer architecture for brain stroke assessment based on multi-slice classification and localization using computed tomography. Comput Med Imaging Graph 2023; 109: 102294.
35. Abbaoui W, Retal S, Ziti S, El Bhiri B. Automated ischemic stroke classification from MRI scans: Using a vision transformer approach. J Clin Med 2024; 13(8): 2323.
36. Koç U, et al. Artificial intelligence in healthcare competition (Teknofest-2021): Stroke data set. Eurasian J Med 2022; 54(3): 248-253.
37. Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data 2019; 6(1): 1-48.
38. Perez L, Wang J. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621, 2017.
39. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S et al. Generative adversarial nets. In: Advances in Neural Information Processing Systems (NIPS); 2014. Vol. 27. pp. 2672-2680.
40. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A et al. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017. pp. 4681-4690.
41. Zhang K, Liang X, Gao H, Van Gool L, Timofte R. Designing a practical degradation model for deep blind image super-resolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); 2021. pp. 4791-4800.
42. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004; 13(4): 600-612.
43. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
44. Guo MH, Xu TX, Liu JJ, Liu ZN, Jiang PT, Mu TJ et al. Attention mechanisms in computer vision: A survey. Comput Vis Media 2022; 8(3): 331-368.
45. Ridnik T, Ben-Baruch E, Noy A, Zelnik-Manor L. ImageNet-21K pretraining for the masses. arXiv preprint arXiv:2104.10972, 2021.
46. Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems (NIPS); 2014. Vol. 27. pp. 3320-3328.
47. Yalçın S, Vural H. Brain stroke classification and segmentation using encoder-decoder based deep convolutional neural networks. Comput Biol Med 2022; 149: 105941.
48. Karataş AF, Doğan V, Kılıç V. Artificial intelligence-based cerebrovascular disease detection on brain computed tomography images. Avrupa Bilim ve Teknoloji Dergisi 2022; (41): 175-182.