Multi-scale Residual Segmentation Network for Histopathological Image

Year 2024, Volume: 15 Issue: 3, 623 - 632

Zehra Bozdağ , Muhammed Fatih Talu

https://doi.org/10.24012/dumf.1500666

Abstract

Deep learning is used in all areas of the image processing like object detection/localization, synthetic image generation, segmentation, tracking, and others. It is frequently used especially in medical image segmentation field since it provides rapid response during the treatment process. The fact that natural images contain different types of noise, patterns, and structures and the lack of distinctive quantitative information still makes the segmentation problem very challenging. The classical networks having high parameters have a long training time. The need of less training time for high parameter networks and high segmentation accuracy has led us to develop a new network. In this study, a state-of-the-art autoencoder network (MSRSegNet) is proposed to perform segmentation. Unlike conventional autoencoder approaches, it consists of encoder, fusion and decoder blocks. In encoder and decoder blocks, Multi-scale Residual Blocks are used to share information between blocks and to detect features on different scales. In fusion block, Atrous Spatial Pyramid Pooling (ASPP) module is used to preserve multi-scale contextual information. Information sharing between blocks has increased the ability of the proposed method to capture global features. The performance parameters of mean intersection over unit (mIOU) and pixel accuracy (PA) is used to compare the results. As a result, it was observed that the proposed segmentation network has high accuracy (69% mIoU) and fast segmentation performance (0.061sec. for an image with 256x256)

Keywords

deep learning, histopathological image segmentaion, image processing

Histopatolojik Görüntü İçin Çok Ölçekli Artık Bölütleme Ağı

Abstract

Derin öğrenme, nesne algılama/yer bulma, sentetik görüntü oluşturma, segmentasyon ve izleme gibi görüntü işlemeyle ilgili tüm alanlarda kullanılmaktadır. Özellikle tedavi sürecinde hızlı yanıt sağladığı için tıbbi görüntü segmentasyonu alanında sıkça kullanılmaktadır. Doğal histopatolojik görüntülerin farklı türde gürültüler, yapılar içermesi ve belirleyici niceliksel bilgilerin eksikliği, segmentasyon problemini çok zor hale getirmektedir. Yüksek parametreli klasik ağların uzun bir eğitim süresi vardır. Daha kısa bir eğitim süresi ve yüksek segmentasyon doğruluğu gereksinimi, bizi yeni bir hibrit ağ geliştirmeye yönlendirdi. Bu çalışma, histopatolojik görüntülerde segmentasyon gerçekleştirmek için bir oto-enkoder ağı (MSRSegNet) önermektedir. Geleneksel oto-enkoder yaklaşımlarından farklı olarak, kodlayıcı, füzyon ve kod çözme bloklarından oluşur. Kodlayıcı ve kod çözücü bloklarda, bloklar arasında bilgi paylaşmak ve farklı ölçeklerde özellikleri tespit etmek için çok ölçekli artık bloklar kullanılır. Füzyon bloğunda, çok ölçekli bağlamsal bilgileri korumak için Atrous Uzaysal Piramit Havuzlama (AUPH) modülü kullanılmaktadır. Bloklar arasındaki bilgi paylaşımı, önerilen yöntemin global özellikleri yakalama yeteneğini artırmıştır. Sonuçları karşılaştırmak için ortalama birlik üzerinden kesişim (mIoU) ve piksel doğruluğu (PA) performans parametreleri kullanılmıştır. Sonuç olarak, önerilen segmentasyon ağının yüksek doğruluğa (%69 mIoU) ve hızlı segmentasyon performansına (256x256 boyutunda bir görüntü için 0.061 saniye) sahip olduğu gözlemlenmiştir.

Keywords

derin ögrenme, histopatolojik görüntü bölütleme, görüntü işleme

References

• [1] World Health Organization, “WHO | Breast cancer,” Who, 2018. https://www.who.int/cancer/prevention/diagnosis-screening/breast-cancer/en/.
• [2] M. N. Gurcan, L. E. Boucheron, A. Can, A. Madabhushi, N. M. Rajpoot, and B. Yener, “Histopathological Image Analysis: A Review,” IEEE Rev. Biomed. Eng., 2009, doi: 10.1109/RBME.2009.2034865.
• [3] Z. Gandomkar, P. Brennan, and C. Mello-Thoms, “Computer-based image analysis in breast pathology,” J. Pathol. Inform., vol. 7, no. 1, p. 43, 2016, doi: 10.4103/2153-3539.192814.
• [4] K. Das, S. Conjeti, A. G. Roy, J. Chatterjee, and D. Sheet, “Multıple Instance Learnıng Of Deep Convolutıonal Neural Networks For Breast Hıstopathology Whole Slıde Classıfıcatıon Kausik Das , Sailesh Conjeti , Abhijit Guha Roy Department of Electrical Engineering , IIT Kharagpur , India School of Medical Science an,” no. Isbi, pp. 578–581, 2018.
• [5] F. Gu, N. Burlutskiy, M. Andersson, and L. K. Wilén, “Multi-resolution Networks for Semantic Segmentation in Whole Slide Images,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 11039 LNCS, pp. 11–18, 2018, doi: 10.1007/978-3-030-00949-6_2.
• [6] J. W. Wei, L. J. Tafe, Y. A. Linnik, L. J. Vaickus, N. Tomita, and S. Hassanpour, “Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks,” Sci. Rep., vol. 9, no. 1, p. 3358, 2019, doi: 10.1038/s41598-019-40041-7.
• [7] Y. Celik, M. Talo, O. Yildirim, M. Karabatak, and U. R. Acharya, “Automated Invasive Ductal Carcinoma Detection Based Using Deep Transfer Learning with Whole-Slide Images,” Pattern Recognit. Lett., 2020, doi: 10.1016/j.patrec.2020.03.011.
• [8] K. Paeng, S. Hwang, S. Park, M. Kim, and S. Kim, “A Unified Framework for Tumor Proliferation Score Prediction in Breast Histopathology,” Dec. 2016.
• [9] J. Wei, J. Wei, C. Jackson, B. Ren, A. Suriawinata, and S. Hassanpour, “Automated detection of celiac disease on duodenal biopsy slides: A deep learning approach,” J. Pathol. Inform., vol. 10, no. 1, p. 7, 2019, doi: 10.4103/jpi.jpi_87_18.
• [10] B. E. Bejnordi et al., “Deep Learnıng-Based Assessment Of Tumor-Assocıated Stroma For Dıagnosıng Breast Cancer In Hıstopathology Images Diagnostic Image Analysis Group , Radboud University Medical Center , Nijmegen , Netherlands , Beth Israel Deaconess Medical Center , Harvard M,” 2017 IEEE 14th Int. Symp. Biomed. Imaging (ISBI 2017), pp. 929–932, 2017, doi: 10.1109/ISBI.2017.7950668.
• [11] Y. Song, Q. Li, H. Huang, D. Feng, M. Chen, and W. Cai, “Low Dimensional Representation of Fisher Vectors for Microscopy Image Classification,” IEEE Trans. Med. Imaging, vol. 36, no. 8, pp. 1636–1649, 2017, doi: 10.1109/TMI.2017.2687466.
• [12] X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local Neural Networks,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 7794–7803, 2018, doi: 10.1109/CVPR.2018.00813.
• [13] S. Wan et al., “Integrated local binary pattern texture features for classification of breast tissue imaged by optical coherence microscopy,” Med. Image Anal., vol. 38, pp. 104–116, 2017, doi: 10.1016/j.media.2017.03.002.
• [14] P. J. Sudharshan, C. Petitjean, F. Spanhol, L. E. Oliveira, L. Heutte, and P. Honeine, “Multiple instance learning for histopathological breast cancer image classification,” Expert Syst. Appl., vol. 117, pp. 103–111, 2019, doi: 10.1016/j.eswa.2018.09.049.
• [15] B. E. Bejnordi, G. Litjens, M. Hermsen, N. Karssemeijer, and J. A. W. M. van der Laak, “A multi-scale superpixel classification approach to the detection of regions of interest in whole slide histopathology images,” Med. Imaging 2015 Digit. Pathol., vol. 9420, p. 94200H, 2015, doi: 10.1117/12.2081768.
• [16] C. Higgins and C. Higgins, “Applications and challenges of digital pathology and whole slide imaging Applications and challenges of digital pathology,” vol. 0295, 2015, doi: 10.3109/10520295.2015.1044566.
• [17] H. Lin, S. Member, H. Chen, S. Graham, and S. Member, “Fast ScanNet : Fast and Dense Analysis of Multi-Gigapixel Whole-Slide Images for Cancer Metastasis Detection,” IEEE Trans. Med. Imaging, vol. PP, no. c, p. 1, 2018, doi: 10.1109/TMI.2019.2891305.
• [18] D. K. Das and P. K. Dutta, “Efficient automated detection of mitotic cells from breast histological images using deep convolution neutral network with wavelet decomposed patches,” Comput. Biol. Med., vol. 104, pp. 29–42, Jan. 2019, doi: 10.1016/j.compbiomed.2018.11.001.
• [19] G. ÇELİK, “Histopatolojik Görüntülerden Kolon Kanseri Tespiti için EfficientNetB0 ve DVM Tabanlı Yaklaşım,” Fırat Üniversitesi Mühendislik Bilim. Derg., 2023, doi: 10.35234/fumbd.1323422.
• [20] O. Jimenez-del-Toro et al., “Analysis of Histopathology Images,” in Biomedical Texture Analysis, Elsevier, 2017, pp. 281–314.
• [21] J.-M. Chen et al., “Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review,” Tumor Biol., vol. 39, no. 3, p. 101042831769455, Mar. 2017, doi: 10.1177/1010428317694550.
• [22] W. T. Xiao, L. J. Chang, and W. M. Liu, “Semantic Segmentation of Colorectal Polyps with DeepLab and LSTM Networks,” 2018, doi: 10.1109/ICCE-China.2018.8448568.
• [23] M. Sebai, T. Wang, and S. A. Al-Fadhli, “PartMitosis: A Partially Supervised Deep Learning Framework for Mitosis Detection in Breast Cancer Histopathology Images,” IEEE Access, vol. 8, pp. 45133–45147, 2020, doi: 10.1109/ACCESS.2020.2978754.
• [24] B. Olimov, K. Sanjar, S. Din, A. Ahmad, A. Paul, and J. Kim, “FU-Net: fast biomedical image segmentation model based on bottleneck convolution layers,” 2021, doi: 10.1007/s00530-020-00726-w.
• [25] H. Zunair and A. Ben Hamza, “Sharp U-Net: Depthwise convolutional network for biomedical image segmentation,” Comput. Biol. Med., 2021, doi: 10.1016/j.compbiomed.2021.104699.
• [26] Z. Bozdag and M. F. Talu, “Pyramidal position attention model for histopathological image segmentation,” Biomed. Signal Process. Control, 2023, doi: 10.1016/j.bspc.2022.104374.
• [27] Z. Bozdağ and F. M. Talu, “Pyramidal Nonlocal Network for Histopathological Image of Breast Lymph Node Segmentation,” Int. J. Comput. Intell. Syst., 2020, doi: 10.2991/ijcis.d.201030.001.
• [28] J. M. J. Valanarasu, V. A. Sindagi, I. Hacihaliloglu, and V. M. Patel, “KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation,” IEEE Trans. Med. Imaging, 2022, doi: 10.1109/TMI.2021.3130469.
• [29] A. Srivastava et al., “MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation,” IEEE J. Biomed. Heal. Informatics, 2022, doi: 10.1109/JBHI.2021.3138024.
• [30] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” 2017, doi: 10.1109/CVPR.2017.243.
• [31] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conf. Comput. Vis. Pattern Recognit., pp. 770–778, Dec. 2015, doi: 10.1109/CVPR.2016.90.
• [32] C. Szegedy et al., “Going deeper with convolutions,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2015, pp. 1–9, doi: 10.1109/CVPR.2015.7298594.
• [33] T. Takikawa, D. Acuna, V. Jampani, and S. Fidler, “Gated-SCNN: Gated Shape CNNs for Semantic Segmentation,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2019, pp. 5228–5237, doi: 10.1109/ICCV.2019.00533.
• [34] J. Fu et al., “Dual Attention Network for Scene Segmentation,” 2019 IEEE/CVF Conf. Comput. Vis. Pattern Recognit., pp. 3141–3149, Sep. 2018, doi: 10.1109/CVPR.2019.00326.
• [35] Z. Zhu, M. Xu, S. Bai, T. Huang, and X. Bai, “Asymmetric Non-Local Neural Networks for Semantic Segmentation,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2019, pp. 593–602, doi: 10.1109/ICCV.2019.00068.
• [36] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in MICCAI2015, 2015, pp. 234–241.
• [37] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2015, pp. 3431–3440, doi: 10.1109/CVPR.2015.7298965.
• [38] A. Pfeuffer, K. Schulz, and K. Dietmayer, “Semantic Segmentation of Video Sequences with Convolutional LSTMs,” IEEE Intell. Veh. Symp. Proc., May 2019, doi: 10.1109/IVS.2019.8813852.
• [39] J. Li, F. Fang, K. Mei, and G. Zhang, “Multi-scale Residual Network for Image Super-Resolution,” Eur. Conf. Comput. Vis., pp. 527–542, 2018, doi: 10.1007/978-3-030-01237-3_32.
• [40] R. Lan, L. Sun, Z. Liu, H. Lu, C. Pang, and X. Luo, “MADNet: A Fast and Lightweight Network for Single-Image Super Resolution,” IEEE Trans. Cybern., 2021, doi: 10.1109/TCYB.2020.2970104.
• [41] Y. Chen, R. Xia, K. Yang, and K. Zou, “MFFN: image super-resolution via multi-level features fusion network,” Vis. Comput., 2024, doi: 10.1007/s00371-023-02795-0.
• [42] C. He et al., “Camouflaged Object Detection with Feature Decomposition and Edge Reconstruction,” 2023, doi: 10.1109/CVPR52729.2023.02111.
• [43] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation,” 2018, doi: 10.1007/978-3-030-01234-2_49.
• [44] B. Ehteshami Bejnordi et al., “Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer,” JAMA, vol. 318, no. 22, p. 2199, Dec. 2017, doi: 10.1001/jama.2017.14585.
• [45] L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation,” pp. 833–851, Feb. 2018, doi: 10.1007/978-3-030-01234-2_49.
• [46] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017, vol. 2017-Janua, pp. 6230–6239, doi: 10.1109/CVPR.2017.660.
• [47] H. Zhao, X. Qi, X. Shen, J. Shi, and J. Jia, “ICNet for Real-Time Semantic Segmentation on High-Resolution Images,” 2018, doi: 10.1007/978-3-030-01219-9_25.
• [48] V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., 2017, doi: 10.1109/TPAMI.2016.2644615.