Classification of Microscopic Fungi Images Using Vision Transformers for Enhanced Detection of Fungal Infections

Abstract

Fungi play a pivotal role in our ecosystem and human health, serving as both essential contributors to environmental sustainability and significant agents of disease. The importance of precise fungi detection cannot be overstated, as it underpins effective disease management, agricultural productivity, and the safeguarding of global food security. This research explores the efficacy of vision transformer-based architectures for the classification of microscopic fungi images of various fungal types to enhance the detection of fungal infections. The study compared the pre-trained base Vision Transformer (ViT) and Swin Transformer models, evaluating their capability in feature extraction and fine-tuning. The incorporation of transfer learning and fine-tuning strategies, particularly with data augmentation, significantly enhances model performance. Utilizing a comprehensive dataset with and without data augmentation, the study reveals that Swin Transformer, particularly when fine-tuned, exhibits superior accuracy (98.36%) over ViT model (96.55%). These findings highlight the potential of vision transformer-based models in automating and refining the diagnosis of fungal infections, promising significant advancements in medical imaging analysis.

Keywords

• Vision transformer
• Swin transformer
• Image classification
• Fungal infection
• Microscopic images

References

1. Lange L. The importance of fungi and mycology for addressing major global challenges. IMA Fungus 2014;5:463–71. https://doi.org/10.5598/imafungus.2014.05.02.10.
2. Almeida F, Rodrigues ML, Coelho C. The still underestimated problem of fungal diseases worldwide. Front Microbiol 2019;10:1–5. https://doi.org/10.3389/fmicb.2019.00214.
3. Ravikant KT, Gupte S, Kaur M. A Review on Emerging Fungal Infections and Their Significance. J Bacteriol Mycol Open Access 2015;1:39–41. https://doi.org/10.15406/jbmoa.2015.01.00009.
4. Brown GD, Denning DW, Gow NAR, Levitz SM, Netea MG, White TC. Hidden killers: Human fungal infections. Sci Transl Med 2012;4:1–9. https://doi.org/10.1126/scitranslmed.3004404.
5. Grosjean P, Weber R. Fungus balls of the paranasal sinuses: A review. Eur Arch Oto-Rhino-Laryngology 2007;264:461–70. https://doi.org/10.1007/s00405-007-0281-5.
6. Hernandez H, Martinez LR. Relationship of environmental disturbances and the infectious potential of fungi. Microbiol (United Kingdom) 2018;164:233–41. https://doi.org/10.1099/mic.0.000620.
7. Kristensen K, Ward LM, Mogensen ML, Cichosz SL. Using image processing and automated classification models to classify microscopic gram stain images. Comput Methods Programs Biomed Updat 2023;3:100091. https://doi.org/10.1016/j.cmpbup.2022.100091.
8. Zhang Y, Jiang H, Ye T, Juhas M. Deep Learning for Imaging and Detection of Microorganisms. Trends Microbiol 2021;29:569–72. https://doi.org/10.1016/j.tim.2021.01.006.

9. Kumar S, Arif T, Alotaibi AS, Malik MB, Manhas J. Advances Towards Automatic Detection and Classification of Parasites Microscopic Images Using Deep Convolutional Neural Network: Methods, Models and Research Directions. Arch Comput Methods Eng 2023;30:2013–39. https://doi.org/10.1007/s11831-022-09858-w.
10. Tahir MW, Zaidi NA, Rao AA, Blank R, Vellekoop MJ, Lang W. A fungus spores dataset and a convolutional neural network based approach for fungus detection. IEEE Trans Nanobioscience 2018;17:281–90. https://doi.org/10.1109/TNB.2018.2839585.
11. Mital ME, Ruzcko Tobias R, Villaruel H, Maningo JM, Kerwin Billones R, Vicerra RR, et al. Transfer learning approach for the classification of conidial fungi (genus aspergillus) thru pre-trained deep learning models. IEEE Reg 10 Annu Int Conf Proceedings/TENCON 2020;2020-Novem:1069–74. https://doi.org/10.1109/TENCON50793.2020.9293803.
12. Mohammad-Rahimi H, Rokhshad R, Bencharit S, Krois J, Schwendicke F. Deep learning: A primer for dentists and dental researchers. J Dent 2023;130:104430. https://doi.org/10.1016/j.jdent.2023.104430.
13. Demir HO, Parlat SZ, Gumus A. Ethereum Blockchain Smart Contract Vulnerability Detection Using Deep Learning. ISAS 2023 - 7th Int Symp Innov Approaches Smart Technol Proc 2023:1–5. https://doi.org/10.1109/ISAS60782.2023.1039179.
14. Kayan CE, Yuksel Aldogan K, Gumus A. Intensity and phase stacked analysis of a Φ-OTDR system using deep transfer learning and recurrent neural networks. Appl Opt 2023;62:1753. https://doi.org/10.1364/ao.481757.
15. Ahmad A, Saraswat D, El Gamal A. A survey on using deep learning techniques for plant disease diagnosis and recommendations for development of appropriate tools. Smart Agric Technol 2023;3:100083. https://doi.org/10.1016/j.atech.2022.100083.
16. Aslani S, Jacob J. Utilisation of deep learning for COVID-19 diagnosis. Clin Radiol 2023;78:150–7. https://doi.org/10.1016/j.crad.2022.11.006.
17. Duzyel O, Catal MS, Kayan CE, Sevinc A, Gumus A. Adaptive resizer-based transfer learning framework for the diagnosis of breast cancer using histopathology images. Signal, Image Video Process 2023;17:4561–70. https://doi.org/10.1007/s11760-023-02692-y.
18. Müjdat Tiryaki V. Mass segmentation and classification from film mammograms using cascaded deep transfer learning. Biomed Signal Process Control 2023;84:104819. https://doi.org/10.1016/j.bspc.2023.104819.
19. Zieliski B, Sroka-Oleksiak A, Rymarczyk D, Piekarczyk A, Brzychczy-Woch M. Deep learning approach to describe and classify fungi microscopic images. PLoS One 2020;15:1–16. https://doi.org/10.1371/journal.pone.0234806.
20. Gaikwad SS, Rumma SS, Hangarge M. Fungi Classification using Convolution Neural Network. Turkish J Comput Math Educ 2021;12:4563–9.
21. Picek L, Sulc M, Matas J, Jeppesen TS, Heilmann-Clausen J, Lassoe T, et al. Danish Fungi 2020 - Not Just Another Image Recognition Dataset. Proc - 2022 IEEE/CVF Winter Conf Appl Comput Vision, WACV 2022 2022:3281–91. https://doi.org/10.1109/WACV51458.2022.00334.
22. Rahman MA, Clinch M, Reynolds J, Dangott B, Meza Villegas DM, Nassar A, et al. Classification of fungal genera from microscopic images using artificial intelligence. J Pathol Inform 2023;14. https://doi.org/10.1016/j.jpi.2023.100314.
23. Koo T, Kim MH, Jue MS. Automated detection of superficial fungal infections from microscopic images through a regional convolutional neural network. PLoS One 2021;16:1–11. https://doi.org/10.1371/journal.pone.0256290.
24. Gao W, Li M, Wu R, Du W, Zhang S, Yin S, et al. The design and application of an automated microscope developed based on deep learning for fungal detection in dermatology. Mycoses 2021;64:245–51. https://doi.org/10.1111/myc.13209.
25. Sopo C, Hajati F, Gheisari S. Defungi: Direct mycological examination of microscopic fungi images. Arxiv 2021. https://doi.org/10.48550/arXiv.2109.07322.
26. Nawarathne UMMPK, Kumari HMNS. Classification of Fungi Images Using Different Convolutional Neural Networks. 2023 8th Int Conf Inf Technol Res 2023.
27. Cinar I, Taspinar YS. Detection of Fungal Infections from Microscopic Fungal Images Using Deep Learning Techniques. Proc Int Conf Adv Technol 2023. https://doi.org/10.58190/icat.2023.12.
28. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention Is All You Need. Adv Neural Inf Process Syst 2017;30.
29. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR 2021 - 9th Int. Conf. Learn. Represent., 2021.
30. Khan S, Naseer M, Hayat M, Zamir SW, Khan FS, Shah M. Transformers in Vision: A Survey. ACM Comput Surv 2022;54. https://doi.org/10.1145/3505244.
31. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, et al. Swin transformer: Hierarchical vision transformer using shifted windows. 2021 IEEE/CVF Int Conf Comput Vis 2021.
32. Iman M, Arabnia HR, Rasheed K. A Review of Deep Transfer Learning and Recent Advancements. Technologies 2023;11:1–14. https://doi.org/10.3390/technologies11020040.
33. Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, et al. A Comprehensive Survey on Transfer Learning. Proc IEEE 2021;109:43–76. https://doi.org/10.1109/JPROC.2020.3004555.
34. Öztürk C, Taşyürek M, Türkdamar MU. Transfer learning and fine-tuned transfer learning methods’ effectiveness analyse in the CNN-based deep learning models. Concurr Comput Pract Exp 2023;35:1–26. https://doi.org/10.1002/cpe.7542.
35. Deng J, Dong W, Socher R, Li L-J, Kai Li, Li Fei-Fei. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conf Comput Vis Pattern Recognit 2010:248–55. https://doi.org/10.1109/cvpr.2009.5206848.
36. Maurício J, Domingues I, Bernardino J. Comparing Vision Transformers and Convolutional Neural Networks for Image Classification: A Literature Review. Appl Sci 2023;13. https://doi.org/10.3390/app13095521.
37. Liu Y, Zhang Y, Wang Y, Hou F, Yuan J, Tian J, et al. A Survey of Visual Transformers. IEEE Trans Neural Networks Learn Syst 2023;PP:1–21. https://doi.org/10.1109/TNNLS.2022.3227717.
38. Jamil S, Jalil Piran M, Kwon OJ. A Comprehensive Survey of Transformers for Computer Vision. Drones 2023;7:1–27. https://doi.org/10.3390/drones7050287.