Dikkat mekanizmalarının bölütleme mimarilerine entegrasyonu ve meme lenf düğümü görüntülerinde uygulanması

Öz

Motorize mikroskopların yaygınlaşması, hastadan alınan dokunun otomatik taranarak tek bir büyük görsele aktarılması ve bölütlemeye özel derin/çekişmeli ağların üretilmesi gibi yenilikler bölütleme probleminde uzman etiketlemesine oldukça yakın çıktıları otomatik üretebilme ümidini arttırmıştır. Özellikle dikkat modüllerinin klasik 3DUNet veya GAN mimarilerine entegrasyonuyla bölütleme performanslarının iyileştirildiği bilinmektedir. Bu çalışmada histopatolojik görüntü bölütleme probleminin çözümünde dört farklı dikkat modülü (DAF, DAF3D, DANet ve MSA) etkileri analiz edilmiştir. DAF ve DAF3D modüllerinde tekil (SLF) ve çoklu (MLF) katman özellikleri birlikte kullanılırken, DANet ve MSA modüllerinde pozisyon ve kanal dikkat olmak üzere iki farklı mekanizma kullanılmaktadır. Yapılan deneysel çalışmalar sonucunda, DAF3D dikkat modülünün bölütleme doğruluğunu en yüksek seviyeye çıkardığı (0.76 mIoU ve 0.89 PA) görülmüştür. Aynı zamanda yaklaşımlar arasında en düşük bölütleme maliyetine (1 görüntü için 0.156 saniye) sahip olan yöntem yine DAF3D olmuştur.

Anahtar Kelimeler

• Dikkat mekanizması
• Histopatoloji
• Bölütleme
• DANet
• DAF
• DAF3D

Integration of attention mechanisms into segmentation architectures and their application on breast lymph node images

Abstract

Innovations such as the widespread use of motorized microscopes, the automatic scanning of the tissue taken from the patient and transferring it to a single large image, and the production of deep/adversarial networks specific to segmentation have increased the hope of automatically producing outputs very close to expert labeling in the segmentation problem. Particularly, it is known that segmentation performances are improved by integrating attention modules into classical 3D-UNet or GAN architectures. In this study, the effects of four different attention modules (DAF, DAF3D, DANet and MSA) were analyzed in solving the histopathological image segmentation problem. While single (SLF) and multiple (MLF) layer features are used together in DAF and DAF3D modules, two different mechanisms, position attention module and channel attention module, are used in DANet and MSA modules. As a result of the experimental studies, it has been seen that the DAF3D attention module maximizes the segmentation accuracy (0.76 mIoU and 0.89 PA). At the same time, the method with the lowest segmentation cost (0.156 seconds for 1 image) among the approaches was again DAF3D.

Keywords

• Attention mechanism
• Histopathology
• Segmentation
• DANet
• DAF
• DAF3D

References

1. [1] Ray S, Turi RH. “Determination of number of clusters in kmeans clustering and application in colour image segmentation”. In Proceedings of the 4th international conference on advances in pattern recognition and digital techniques, Calcutta, India, 22-29 December 1999.
2. [2] Park SH, Yun ID, Lee SU. “Color image segmentation based on 3-D clustering: morphological approach”. Pattern Recognition, 31(8), 1061-1076, 1998.
3. [3] Zhang X, Shan Y, Wei W, Zhu Z. “An Image Segmentation Method Based on Improved Watershed Algorithm”. 2010 International Conference on Computational and Information Sciences, Chengdu, China, 17-19 December 2010.
4. [4] Tobias OJ, Seara R. “Image segmentation by histogram thresholding using fuzzy sets”. IEEE Transactions on Image Processing, 11(12), 1457-1465, 2002.
5. [5] Derraz F, Beladgham M, Khelif MH. “Application of active contour models in medical image segmentation”. International Conference on Information Technology: Coding and Computing, Las Vegas, Nevada, USA, 5-7 April 2004.
6. [6] Al-Milaji Z, Ersoy I, Hafiane A, Palaniappan K, Bunyak F. “Integrating segmentation with deep learning for enhanced classification of epithelial and stromal tissues in H&E images”. Pattern Recognition Letters, 119, 214-221, 2019.
7. [7] Pan X, Li L, Yang H, Liu Z, Yang J, Zhao L, Fan Y. “Accurate segmentation of nuclei in pathological images via sparse reconstruction and deep convolutional networks”. Neurocomputing, 229, 88-99, 2017.
8. [8] Bulten W, Hulsbergen-van de Kaa, CA, van der Laak J, Litjens GJ. “Automated segmentation of epithelial tissue in prostatectomy slides using deep learning”. In Medical Imaging 2018: Digital Pathology, 10581, 219-225, 2018.

9. [9] Matuszewski DJ, Sintorn IM. “Reducing the U-Net size for practical scenarios: Virus recognition in electron microscopy images”. Computer methods and programs in biomedicine, 178, 31-39, 2019.
10. [10] Kalapahar A, Silva-Rodríguez J, Colomer A, López-Mir F, Naranjo V. "Gleason Grading of Histology Prostate Images Through Semantic Segmentation via Residual U-Net". 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 25-28 October, 2020.
11. [11] AskariHemmat M, Honari S, Rouhier L, Perone CS, CohenAdad J, Savaria Y, David JP. “U-net fixed-point quantization for medical image segmentation”. International Workshops, LABELS 2019, HAL-MICCAI 2019, and CuRIOUS 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, 13-17 October, 2019.
12. [12] Badrinarayanan V, Kendall A, Cipolla R. "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481-2495, 2017
13. [13] Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. “Semantic image segmentation with deep convolutional nets and fully connected crfs”. arXiv, 2016. https://arxiv.org/pdf/1412.7062.pdf
14. [14] Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs”. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834-848, 2017.
15. [15] Bozdağ Z. Histopatolojik Görüntülerde Tümör Bölütlenmesi. Doktora Tezi, İnönü Üniversitesi, Malatya, Türkiye, 2021.
16. [16] Wang X, Girshick R, Gupta A, He K. “Non-local Neural Networks.” IEEE 2018 Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18-23 June 2018.
17. [17] Galassi A, Lippi M, Torroni P. “Attention in natural language processing”. IEEE Transactions on Neural Networks and Learning Systems, 32(10), 4297-4308, 2020.
18. [18] Zhang H, Goodfellow I, Metaxas D, Odena A. “Self-Attention Generative Adversarial Networks”. 36th International Conference on Machine Learning, ICML, Long Beach, California, USA, 9-15 June 2019.
19. [19] Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Tang X. “Residual attention network for image classification”. 2017 IEEE conference on computer vision and pattern recognition, Honolulu, Hawaii 21-26 July, 2017.
20. [20] Wang Y, Deng Z, Hu X, Zhu L, Yang X, Xu X, Ni D. “Deep attentional features for prostate segmentation in ultrasound”. In International Conference on Medical Image Computing and Computer-Assisted Interventio, Granada, Spain, 16-20 September, 2018.
21. [21] Wang Y, Dou H, Hu X, Zhu L, Yang X, Xu M, Ni D. “Deep attentive features for prostate segmentation in 3D transrectal ultrasound”. IEEE transactions on medical imaging, 38(12), 2768-2778, 2019.
22. [22] Zhao H, Zhang Y, Liu S, Shi J, Loy CC, Lin D, Jia J. “Psanet: Point-wise spatial attention network for scene parsing”. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8-14 September, 2018.
23. [23] Li H, Xiong P, An J, Wang L. “Pyramid attention network for semantic segmentation”. arXiv, 2018. https://arxiv.org/pdf/1805.10180.pdf
24. [24] Yu Y, Ji Z, Fu Y, Guo J, Pang Y, Zhang Z. “Stacked semanticguided attention model for fine-grained zero-shot learning”. 32nd Conference on Neural Information Processing Systems, Montreal, Canada, 3-8 December, 2018.
25. [25] Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H. “Dual attention network for scene segmentation”. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA 16-20 June, 2019.
26. [26] Wu F, Chen F, Jing XY, Hu CH, Ge Q, Ji Y. “Dynamic attention network for semantic segmentation”. Neurocomputing, 384, 182-191, 2020
27. [27] Sinha A, Dolz J. “Multi-scale self-guided attention for medical image segmentation”. IEEE journal of biomedical and health informatics, 25(1), 121-130, 2020
28. [28] Huang Z, Wang X, Huang L, Huang C, Wei Y, Liu W. “Ccnet: Criss-cross attention for semantic segmentation”. 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Korea, 27 October-2 November, 2019.
29. [29] Tao A, Sapra K, Catanzaro B. “Hierarchical multi-scale attention for semantic segmentation”. arXiv, 2020. https://arxiv.org/pdf/2005.10821.pdf
30. [30] Chen LC, Yang Y, Wang J, Xu W, Yuille AL. “Attention to scale: Scale-aware semantic image segmentation”. 2016 IEEE conference on computer vision and pattern recognition Las Vegas, NV, USA, 27-30 June 2016.
31. [31] Li R, Su J, Duan C, Zheng S. “Linear attention mechanism: An efficient attention for semantic segmentation”. arXiv, 2020 https://arxiv.org/ftp/arxiv/papers/2007/2007.14902.pdf
32. [32] Islam M, Vibashan VS, Jose V, Wijethilake N, Utkarsh U, Ren H. “Brain tumor segmentation and survival prediction using 3D attention UNet”. 5th International Workshop, BrainLes 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, 17 October 2019.
33. [33] Zhang J, Yu L, Chen D, Pan W, Shi C, Niu Y, Cheng Y. “Dense GAN and multi-layer attention based lesion segmentation method for COVID-19 CT images”. Biomedical Signal Processing and Control, 2021. https://doi.org/10.1016/j.bspc.2021.102901
34. [34] Camelyon16 Veri Kümesi. https://camelyon16.grand-challenge.org/ (08.05.2022 He K, Zhang X, Ren S, Sun J. “Deep residual learning for image recognition”. 2016 IEEE conference on computer vision and pattern recognition, Las Vegas, Nevada. USA, 27-30 June 2016.