Multiscale Hybrid Attention Network for Automated Segmentation of Colorectal Polyps

Abstract

Accurate and reliable segmentation of colorectal polyps plays a critical role in the early diagnosis of colorectal cancer. Although deep learning–based methods have shown significant progress in recent years, the variations in polyp size, shape, color, and texture can adversely affect segmentation performance. In this study, a hybrid network model is proposed for automatic polyp segmentation. The proposed approach utilizes multi-level single-layer feature maps extracted from a DAF3D-based structure, which has demonstrated strong performance in deep feature extraction. These feature maps are first processed with a channel attention module and then with a reverse attention module to generate a multi-layer feature representation. While the channel attention module emphasizes the most informative channels, the reverse attention module enhances boundary regions, thereby improving segmentation accuracy. To evaluate the effect of these modules, scenarios using only channel attention, only reverse attention, and both modules together were compared, revealing that the integrated use of both modules achieved the highest performance. Additionally, the proposed model was compared with several polyp segmentation networks from the literature under the same backbone architecture and identical training conditions. Experiments conducted on multiple datasets demonstrate that the hybrid architecture provides significantly superior performance in mean Dice and mean IoU metrics compared to other methods. The findings indicate that the proposed approach is an effective, stable, and highly accurate method for polyp segmentation.

Keywords

• Polyp segmentation
• Attention
• Chanal attention modüle
• Reverse attention modüle

Ethical Statement

This study is a research paper that does not include a survey and the datasets used in the implementation phase were obtained from publicly accessible websites; therefore, it does not require ethical committee approval.

Kolorektal Poliplerin Otomatik Segmentasyonu için Çok Ölçekli Hibrit Dikkat Ağı

Abstract

Kolorektal poliplerin doğru ve güvenilir bir şekilde bölütlenmesi, kolorektal kanserin erken tanısında kritik bir öneme sahiptir. Son yıllarda derin öğrenme tabanlı yöntemlerle önemli ilerlemeler kaydedilmiş olsa da poliplerin boyut, şekil, renk ve doku açısından gösterdiği çeşitlilik, segmentasyon başarımını olumsuz etkileyebilmektedir. Bu çalışmada, otomatik polip segmentasyonu için hibrit bir ağ modeli önerilmektedir. Önerilen yaklaşımda, derin özellik çıkarımında başarılı olduğu bilinen DAF3D tabanlı yapıda elde edilen çok seviyeli tek katmanlı özellik haritaları kullanılmaktadır. Bu özellik haritaları önce kanal dikkat modülü, ardından ters dikkat modülü ile işlenerek çok katmanlı bir özellik temsiline dönüştürülmektedir. Kanal dikkat modülü en bilgilendirici kanalları öne çıkarırken, ters dikkat modülü sınır bölgelerini belirginleştirerek segmentasyon doğruluğunu artırmaktadır. Modüllerin etkisini değerlendirmek amacıyla yalnızca kanal dikkat, yalnızca ters dikkat ve her iki modülün birlikte kullanıldığı senaryolar karşılaştırılmış; her iki modülün birlikte kullanıldığı durumda en yüksek başarımın elde edildiği görülmüştür. Ayrıca önerilen model, literatürde yer alan çeşitli polip segmentasyon ağlarıyla aynı omurga mimarisi ve eşit eğitim koşulları altında karşılaştırılmıştır. Birden fazla veri seti üzerinde gerçekleştirilen deneyler, hibrit yapının mean Dice ve mean IoU metriklerinde diğer yöntemlere kıyasla anlamlı derecede üstün performans gösterdiğini ortaya koymaktadır. Elde edilen bulgular, önerilen yaklaşımın polip segmentasyonu için etkili, kararlı ve yüksek doğruluk sağlayan bir yöntem olduğunu göstermektedir.

Keywords

• Polip segmentasyonu
• Dikkat
• Kanal dikkat modülü
• Ters dikkat modülü

Ethical Statement

Bu çalışma, anket çalışması içermeyen ve uygulama aşamasında kullanılan veri setleri herkes tarafından erişilebilir sitelerden alınmış bir araştırma makalesi niteliğinde olup, etik kurul onayı gerektirmemektedir.

References

1. Silva J, Histace A, Romain O, Dray X, Granado B. Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. International Journal of Computer Assisted Radiology and Surgery. 2014;9(2):283-293.
2. Haggar FA, Boushey RP. Colorectal cancer epidemiology: incidence, mortality, survival, and risk factors. Clinics in colon and rectal surgery.2009;22(04):191-197
3. Heresbach D, Barrioz T, Lapalus M, Coumaros D, Bauret P, Potier P, et. Al. Miss rate for colorectal neoplastic polyps: a prospective multicenter study of back-to-back video colonoscopies, Endoscopy. 2008;40:284–290.
4. Leufkens A, Oijen van M, Vleggaar F, Siersema P. Factors influencing the miss rate of polyps in a back-to-back colonoscopy study, Endoscopy. 2012;44:470-475
5. Karkanis SA, Iakovidis DK, Maroulis DE, Karras DA, Tzivras M. Computer-aided tumor detection in endoscopic video using color wavelet features, IEEE Trans. Inf. Technol. Biomed. 2003;7:141–152.
6. Nawarathna R, Oh JH, Muthukudage J, Tavanapong W, Wong J, Groen PC, et. al. Abnormal image detection in endoscopy videos using a filter bank and local binary patterns, Neurocomputing. 2014;144:70–91
7. Mamonov AV, Figueiredo IN, Figueiredo PN, Tsai YHR. Automated Polyp Detection in Colon Capsule Endoscopy, IEEE Trans. Med. Imaging. 2014;33:1488–1502.
8. Tajbakhsh N, Gurudu SR, Liang J. Automated Polyp Detection in Colonoscopy Videos Using Shape and Context Information, IEEE Trans. Med. Imaging. 2016;35:630–644.

9. Yu L, Chen H, Dou Q, Qin J, Heng PA. Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE journal of biomedical and health informatics. 2016;21(1):65-75
10. Brandao P, Mazomenos E, Ciuti G, Caliò R, Bianchi F, Menciassi A, et. al. Fully convolutional neural networks for polyp segmentation in colonoscopy. In Medical Imaging 2017: Computer-Aided Diagnosis. 2017;10134:101-107
11. Akbari M, Mohrekesh M, Nasr-Esfahani E, Soroushmehr SR, Karimi N, Samavi S, et. al. Polyp segmentation in colonoscopy images using fully convolutional network. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2018;69-72
12. Zhang L, Dolwani S, Ye X. Automated polyp segmentation in colonoscopy frames using fully convolutional neural network and textons. In Annual Conference on Medical Image Understanding and Analysis. 2017;707-717. Springer, Cham.
13. Jha D, Smedsrud PH, Riegler MA, Johansen D, De Lange T, Halvorsen P, et. al. Resunet++: An advanced architecture for medical image segmentation. IEEE International Symposium on Multimedia (ISM) 2019. San Diego, IEEE;2019. p. 225-2255.
14. Su J, Li J, Zhang Y, Xia C, Tian Y. Selectivity or invariance: Boundary-aware salient object detection. International Conference on Computer Vision 2019. Seoul, IEEE/CVF; 2019. p. 3799-3808
15. Yang B, Zhang X, Chen L, Yang H, Gao Z. Edge guided salient object detection. Neurocomputing. 2017;221:60-71.
16. Fang, Y., Chen, C., Yuan, Y., & Tong, K. Y. Selective feature aggregation network with area-boundary constraints for polyp segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention 2019. China, Springer, Cham; 2019 p. 302-310.
17. Murugesan B, Sarveswaran K, Shankaranarayana SM, Ram K, Joseph J, Sivaprakasam M. Psi-Net: Shape and boundary aware joint multi-task deep network for medical image segmentation. 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2019. Strasbourg, IEEE; 2019 p. 7223-7226.
18. Kim T, Lee H, Kim D. Uacanet: Uncertainty augmented context attention for polyp segmentation. 29th ACM International Conference on Multimedia 2021. Chengdu, ACM; 2021 p. 2167-2175.
19. Bozdağ Z. Histopatolojik görüntülerde tümör bölütlenmesi. [Phd. thesis], Malatya: İnönü Üniversitesi; 2021.
20. Kayaoğlu M, Şengür A, Ciğerim SÇ, Bor S. Detection of Cervical Vertebrae Using Object Detection and Semantic Segmentation Methods in Lateral Cephalometric Radiographs, Turkish Journal of Nature and Science. 2025;14(2): 26–36.
21. Tomar NK, Jha D, Ali S, Johansen HD, Johansen D, Riegler MA, et. al. DDANet: Dual decoder attention network for automatic polyp segmentation. International Conference on Pattern Recognition 2021. Kolkata, Springer, Cham; p. 307-314.
22. Tomar NK, Jha D, Riegler MA, Johansen HD, Johansen D, Rittscher J, et. al. Fanet: A feedback attention network for improved biomedical image segmentation. IEEE Transactions on Neural Networks and Learning Systems. 2022;34(11): 9375-9388.
23. Tomar NK, Jha D, Bagci U, Ali S. TGANet: Text-guided attention for improved polyp segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention 2022. Switzerland, Cham: Springer Nature; p.151-160.
24. Xu Q, Duan W, He N. DCSAU-Net: A Deeper and More Compact Split-Attention U-Net for Medical Image Segmentation. Computers in Biology and Medicine. 2023;154: 106626.
25. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño, F. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Computerized medical imaging and graphics. 2015;43:99-111.
26. Huang CH, Wu HY, Lin YL. Hardnet-mseg: A simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps. arXiv preprint arXiv:2101.07172.
27. Chao P, Kao CY, Ruan YS, Huang CH, Lin YL. Hardnet: A low memory traffic network. International Conference on Computer Vision 2019. Seoul, IEEE/CVF; 2019. p. 3552-3561.
28. Wu Z, Su L, Huang Q. Cascaded partial decoder for fast and accurate salient object detection. International Conference on Computer Vision and Pattern Recognation 2019. Seoul, IEEE/CVF; 2019. p. 3907-3916.
29. Chen S, Tan X, Wang B, Hu X. Reverse attention for salient object detection. In Proceedings of the European conference on computer vision (ECCV). 2018. p. 234-250.
30. Fan DP, Ji GP, Zhou T, Chen G, Fu H, Shen J. et al. Pranet: Parallel reverse attention network for polyp segmentation. In International conference on medical image computing and computer-assisted intervention (pp. 263-273). 2020; Springer, Cham.
31. Gao SH, Cheng MM, Zhao K, Zhang XY, Yang MH, Torr P. Res2net: A new multi-scale backbone architecture. IEEE transactions on pattern analysis and machine intelligence. 2019; 43(2): 652-662.
32. Lou A, Guan S, Loew M. Caranet: Context axial reverse attention network for segmentation of small medical objects. Journal of Medical Imaging. 2023;10(1):14005-14005.
33. Bernal J, Sánchez J, Vilarino F. Towards automatic polyp detection with a polyp appearance model. Pattern Recognition. 2012;45(9): 3166-3182.
34. Jha D, Smedsrud PH, Riegler MA, Halvorsen P, Lange TD, Johansen D, et. al.Kvasir-seg: A segmented polyp dataset. International Conference on Multimedia Modeling. 2020;pp. 451-462. Springer, Cham.
35. Vazquez D, Bernal J, S´anchez FJ, Fern´andez-Esparrach GL´opez, AM, Romero A, Drozdzal M, et. al. A benchmark for endoluminal scene segmentation of colonoscopy images. Journal of Healthcare Engineering. 2017; 2017(1): 4037190.
36. Wang Y, Dou H, Hu X, Zhu L, Yang X, Xu M, et. al. Deep attentive features for prostate segmentation in 3D transrectal ultrasound. IEEE transactions on medical imaging. 2019;38(12): 2768-2778.
37. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, et. al. Dual attention network for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3146-3154, 2019.