A deep-XAI method for histopathological image classification: Utilizing transformer based FNet architecture and LIME method

Öz

The purpose of the study is to improve the cancer detection in medical images using the Fourier Net (FNet) architecture and the Local Interpretable Model-agnostic Explanations (LIME) method. The FNet architecture excels in extracting features from high-dimensional images and anatomical representations. LIME, on the other hand, is an algorithm to make the model's decisions interpretable. After applying the FNet architecture to the existing data, the LIME explainability method has been applied to determine whether the model outputs meaningful results from the image. Using deep learning techniques, the proposed algorithm represents cancer types with distinctive and robust features. An additional assessment by an expert pathologist was carried out to prove the results obtained after the LIME interpretation. Thus, medical professionals and researchers will be able to evaluate whether this method developed using FNet and LIME can provide a more interpretable and effective approach to cancer diagnosis. The proposed study lays the foundation for developing effective systems that assist doctors and pathologists in evaluating histopathological tissue images. Additionally, this study aims to enhance the reliability of machine learning methods.

Anahtar Kelimeler

• FNet
• explainable artificial intelligence
• lung cancer
• colon cancer
• pathological assessment

Histopatolojik Görüntü Sınıflandırması için Açıklanabilir Derin Öğrenme Yöntemi: Transformer Tabanlı FNet Mimarisi ve LIME Metodunun Kullanımı

Öz

Çalışmanın amacı, Fourier Net (FNet) mimarisi ve Yerel Yorumlanabilir Model-Agnostik Açıklamalar (LIME) yöntemi kullanılarak tıbbi görüntülerde kanser tespitini geliştirmektir. FNet mimarisi, yüksek boyutlu görüntülerden ve anatomik temsillerden öne çıkan özellikleri başarıyla çıkarma yeteneğine sahiptir. Öte yandan, LIME, modelin kararlarını yorumlanabilir hale getirmek için kullanılan bir algoritmadır. Mevcut verilere FNet mimarisi uygulandıktan sonra, modelin görüntülerden anlamlı sonuçlar üretip üretmediğini değerlendirmek amacıyla LIME açıklanabilirlik yöntemi uygulanmıştır. Önerilen algoritma, derin öğrenme teknikleri kullanılarak kanser türlerini belirgin ve güçlü özelliklerle temsil etmektedir. Ayrıca, LIME yorumlamasından elde edilen sonuçların doğruluğunu kanıtlamak amacıyla uzman bir patolog tarafından ek bir değerlendirme gerçekleştirilmiştir. Bu sayede, tıp uzmanları ve araştırmacılar, FNet ve LIME kullanılarak geliştirilen bu yöntemin kanser teşhisinde daha yorumlanabilir ve etkili bir yaklaşım sağlayıp sağlamadığını değerlendirebileceklerdir. Önerilen çalışma, doktorlar ve patologların histopatolojik doku görüntülerini değerlendirmesine yardımcı olacak etkili sistemlerin geliştirilmesi için bir temel oluşturmaktadır. Ek olarak, bu çalışma makine öğrenimi yöntemlerinin güvenilirliğini artırmayı amaçlamaktadır.

Anahtar Kelimeler

• FNet
• açıklanabilir yapay zeka
• akciğer kanseri
• kolon kanseri
• patolojik değerlendirme

Kaynakça

1. [1] D. Collaborators, “Global , regional , and national life expectancy , all-cause mortality , and cause-specifi c mortality for 249 causes of death , 1980 – 2015 : a systematic analysis for the Global Burden of Disease Study 2015,” pp. 1980–2015, 2017, doi: 10.1016/S0140-6736(16)31012-1.
2. [2] B. F. Ferlay J, Ervik M, Lam F, Laversanne M, Colombet M, Mery L, Piñeros M, Znaor A, Soerjomataram I, “WHO.” [Online]. Available: https://gco.iarc.who.int
3. [3] D. Hanahan and R. A. Weinberg, “The Hallmarks of Cancer,” Cell, vol. 100, no. 1, pp. 57–70, 2000, doi: https://doi.org/10.1016/S0092-8674(00)81683-9.
4. [4] P. L. Nunez and R. Srinivasan, “A theoretical basis for standing and traveling brain waves measured with human EEG with implications for an integrated consciousness,” Clin. Neurophysiol., vol. 117, no. 11, pp. 2424–2435, 2006, doi: 10.1016/j.clinph.2006.06.754.
5. [5] K. Kurishima et al., “Lung cancer patients with synchronous colon cancer,” Mol. Clin. Oncol., pp. 137– 140, 2017, doi: 10.3892/mco.2017.1471.
6. [6] M. del Re et al., “Implications of KRAS mutations in acquired resistance to treatment in NSCLC,” Oncotarget, vol. 9, no. 5, pp. 6630–6643, 2018, doi: 10.18632/oncotarget.23553.
7. [7] D. Crosby et al., “Early detection of cancer,” Science (80- . )., vol. 375, no. 6586, p. eaay9040, 2022, doi: 10.1126/science.aay9040.
8. [8] C. Bladder et al., “Bladder Cancer Early Detection , Diagnosis , and Staging Can Bladder Cancer Be Found Early,” Am. Cancer Soc., no. cancer.org, pp. 1–24, 2023, [Online]. Available: https://www.cancer.org/content/dam/CRC/PDF/Public/8661.00.pdf

9. [9] E. Sümer, M. Engin, M. Ağıldere, and H. Oğul, “Monitoring Nodule Progression in Chest X-ray Images,” Pamukkale Univ. J. Eng. Sci., vol. 24, no. 5, pp. 934–941, 2018, doi: 10.5505/pajes.2018.89166.
10. [10] B. H. M. van der Velden, H. J. Kuijf, K. G. A. Gilhuijs, and M. A. Viergever, “Explainable artificial intelligence (XAI) in deep learning-based medical image analysis,” Med. Image Anal., vol. 79, p. 102470, 2022, doi: 10.1016/j.media.2022.102470.
11. [11] S. Mangal, A. Chaurasia, and A. Khajanchi, “Convolution Neural Networks for diagnosing colon and lung cancer histopathological images.” 2020.
12. [12] M. Masud, N. Sikder, A.-A. Nahid, A. K. Bairagi, and M. A. AlZain, “A Machine Learning Approach to Diagnosing Lung and Colon Cancer Using a Deep Learning-Based Classification Framework,” Sensors, vol. 21, no. 3, 2021, doi: 10.3390/s21030748.
13. [13] K. Adu, Y. Yu, J. Cai, K. Owusu-Agyemang, B. A. Twumasi, and X. Wang, “DHS-CapsNet: Dual horizontal squash capsule networks for lung and colon cancer classification from whole slide histopathological images,” Int. J. Imaging Syst. Technol., vol. 31, no. 4, pp. 2075–2092, 2021, doi: https://doi.org/10.1002/ima.22569.
14. [14] M. Ali and R. Ali, “Multi-Input Dual-Stream Capsule Network for Improved Lung and Colon Cancer Classification,” Diagnostics, vol. 11, no. 8, 2021, doi: 10.3390/diagnostics11081485.
15. [15] N. yahia Ibrahim and A. S. Talaat, “An Enhancement Technique to Diagnose Colon and Lung Cancer by using Double CLAHE and Deep Learning,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 8, 2022, doi: https://doi.org/10.14569/IJACSA.2022.0130833.
16. [16] M. A. Talukder, M. M. Islam, M. A. Uddin, A. Akhter, K. F. Hasan, and M. A. Moni, “Machine learning-based lung and colon cancer detection using deep feature extraction and ensemble learning,” Expert Syst. Appl., vol. 205, p. 117695, 2022, doi: https://doi.org/10.1016/j.eswa.2022.117695.
17. [17] M. S. N. Raju and B. S. Rao, “Classification of Colon and Lung Cancer through Analysis of Histopathology Images Using Deep Learning Models,” Ing. des Syst. d’Information, vol. 27, no. 6, pp. 967–971, 2022, doi: 10.18280/isi.270613.
18. [18] N. Kumar, M. Sharma, V. P. Singh, C. Madan, and S. Mehandia, “An empirical study of handcrafted and dense feature extraction techniques for lung and colon cancer classification from histopathological images,” Biomed. Signal Process. Control, vol. 75, p. 103596, 2022, doi: https://doi.org/10.1016/j.bspc.2022.103596.
19. [19] R. R. Wahid, C. Nisa’, R. P. Amaliyah, and E. Y. Puspaningrum, “Lung and colon cancer detection with convolutional neural networks on histopathological images,” AIP Conf. Proc., vol. 2654, no. 1, p. 20020, Feb. 2023, doi: 10.1063/5.0114327.
20. [20] S. Tummala, S. Kadry, A. Nadeem, H. T. Rauf, and N. Gul, “An Explainable Classification Method Based on Complex Scaling in Histopathology Images for Lung and Colon Cancer,” Diagnostics, vol. 13, no. 9, 2023, doi: 10.3390/diagnostics13091594.
21. [21] S. Mehmood et al., “Malignancy Detection in Lung and Colon Histopathology Images Using Transfer Learning With Class Selective Image Processing,” IEEE Access, vol. 10, pp. 25657–25668, 2022, doi: 10.1109/ACCESS.2022.3150924.
22. [22] J. Fan, J. Lee, and Y. Lee, “A Transfer Learning Architecture Based on a Support Vector Machine for Histopathology Image Classification,” Appl. Sci., vol. 11, no. 14, 2021, doi: 10.3390/app11146380.
23. [23] T. Aitazaz, A. Tubaishat, F. Al-Obeidat, B. Shah, T. Zia, and A. Tariq, “Transfer learning for histopathology images: an empirical study,” Neural Comput. Appl., vol. 35, no. 11, pp. 7963–7974, 2023, doi: 0.1007/s00521-022-07516-7.
24. [24] M. S. Ahmed, K. N. Iqbal, and M. G. R. Alam, “Interpretable Lung Cancer Detection using Explainable AI Methods,” in 2023 International Conference for Advancement in Technology (ICONAT), 2023, pp. 1–6. doi: 10.1109/ICONAT57137.2023.10080480.
25. [25] J. Lee-Thorp, J. Ainslie, I. Eckstein, and S. Ontañón, “FNet: Mixing Tokens with Fourier Transforms,” NAACL 2022 - 2022 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. Proc. Conf., pp. 4296–4313, 2022, doi: 10.18653/v1/2022.naaclmain.319.
26. [26] A. A. Borkowski, M. M. Bui, L. B. Thomas, C. P. Wilson, L. A. DeLand, and S. M. Mastorides, “Lung and Colon Cancer Histopathological Image Dataset (LC25000),” pp. 1–2, 2019.
27. [27] H. Lemjabbar-Alaoui, O. U. I. Hassan, Y.-W. Yang, and P. Buchanan, “Lung cancer: Biology and treatment options,” Biochim. Biophys. Acta - Rev. Cancer, vol. 1856, no. 2, pp. 189–210, 2015, doi: https://doi.org/10.1016/j.bbcan.2015.08.002.
28. [28] J. H. Schiller et al., “Comparison of Four Chemotherapy Regimens for Advanced Non–Small-Cell Lung Cancer,” N. Engl. J. Med., vol. 346, no. 2, pp. 92–98, 2002, doi: 10.1056/NEJMoa011954.
29. [29] A. Banerjee, S. Pathak, V. D. Subramanium, D. G., R. Murugesan, and R. S. Verma, “Strategies for targeted drug delivery in treatment of colon cancer: current trends and future perspectives,” Drug Discov. Today, vol. 22, no. 8, pp. 1224–1232, 2017, doi: https://doi.org/10.1016/j.drudis.2017.05.006.
30. [30] F. A. Spanhol, L. S. Oliveira, C. Petitjean, and L. Heutte, “A Dataset for Breast Cancer Histopathological Image Classification,” IEEE Trans. Biomed. Eng., vol. 63, no. 7, pp. 1455–1462, 2016, doi: 10.1109/TBME.2015.2496264.
31. [31] A. Vaswani, N. Shazeer, and N. Parmar, “Attention is All You Need,” in 31st Conference on Neural Information Processing Systems (NIPS), Long Beach, 2015.
32. [32] E. Yazan and M. F. Talu, “Integration of attention mechanisms into segmentation architectures and their application on breast lymph node images,” Pamukkale Univ. J. Eng. Sci., vol. 29, no. 3, pp. 248–255, 2023, doi: 10.5505/pajes.2022.07838.
33. [33] G. Çelik and M. F. Talu, “Generating the image viewed from EEG signals,” Pamukkale Univ. J. Eng. Sci., vol. 27, no. 2, pp. 129–138, 2021, doi: 10.5505/pajes.2020.76399.
34. [34] B. H. M. van der Velden, H. J. Kuijf, K. G. A. Gilhuijs, and M. A. Viergever, “Explainable artificial intelligence (XAI) in deep learning-based medical image analysis,” Medical Image Analysis, vol. 79. Elsevier B.V., Jul. 01, 2022. doi: 10.1016/j.media.2022.102470.
35. [35] M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?’ Explaining the Predictions of Any Classifier,” NAACL-HLT 2016 - 2016 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. Proc. Demonstr. Sess., pp. 97–101, 2016, doi: 10.18653/v1/n16-3020.
36. [36] D. Kaplun, A. Krasichkov, P. Chetyrbok, N. Oleinikov, A. Garg, and H. S. Pannu, “Cancer Cell Profiling Using Image Moments and Neural Networks with Model Agnostic Explainability: A Case Study of Breast Cancer Histopathological (BreakHis) Database,” Mathematics, vol. 9, no. 20, 2021, doi: 10.3390/math9202616.
37. [37] J. C. Kumar, Vinay Abbas, Abul K Aster, “Robbins Basic Pathology, Ninth Edition,” in Robbins Basic Pathology. [Online]. Available: https://books.google.com.tr/books?id=aLV9nU_7X6UC&printsec=copyright&redir_esc=y#v=onepage&q&f=false
38. [38] J. M. JR Goldblum, LW Lamps, Rosai and Ackerman’s Surgical Pathology. 2017.