TriaNet: A Tri-Fusion Attention Network for Segmenting Polyps with Ambiguous Boundaries

Abstract

Colorectal cancer (CRC) is one of the most common and deadly types of cancer worldwide. During standard colonoscopy procedures to detect polyps, which are early-stage precancerous lesions critical for disease prevention, challenges exist, such as overlooking polyps and the inability to accurately segment polyps with weak borders that are integrated with surrounding tissue using current computer-aided methods. This study proposes a new deep learning architecture, called TriaNet (Tri-Fusion Attention Network), to enhance the segmentation accuracy of polyps with weak borders. The fundamental innovation of TriaNet is its unique “triple-fusion” attention mechanism, which combines three complementary information streams. The proposed method dynamically fuses edge feature information obtained from a hybrid block containing Scharr, DoG, and Gabor filters, the semantic feature map from the decoder structure, and an instantaneous boundary map derived from a Scharr operator applied to an upper layer prediction. Furthermore, Deformable Alignment layers are utilized in skip connections to enhance the model's ability to adapt to variable polyp morphologies. The TriaNET architecture was tested on four different benchmark datasets, including Kvasir-SEG, CVC-ColonDB, ETIS-LaribPolypDB, and CVC-300, which demonstrated superior performance compared to state-of-the-art methods.

Keywords

• Polyp Segmentation
• Attention Network
• U-Net
• Fusion Module

References

1. Li, S., Ren, Y., Yu, Y., Jiang, Q., He, X., & Li, H., “A survey of deep learning algorithms for colorectal polyp segmentation”, Neurocomputing, 614, 128767, 2025.
2. Pacal, I., Karaboga, D., Basturk, A., Akay, B., & Nalbantoglu, U. “A Comprehensive review of deep learning in colon cancer”, Computers in Biology and Medicine, 126, 104003, 2020.
3. Islam, M. R., Ahamed, M. F., Islam, M. R., Nahiduzzaman, M., & Ahsan, M., “Detection, localization, segmentation, and classification in colorectal cancer screening using deep learning: A systematic review”, Biomedical Signal Processing and Control, 110, 108202, 2025.
4. Maas, M. H., et al., “A computer-aided polyp detection system in screening and surveillance colonoscopy: an international, multicentre, randomised, tandem trial”, The Lancet Digital Health, 6(3), e157-e165, 2024.
5. Bui, N. T., Hoang, D. H., Nguyen, Q. T., Tran, M. T., & Le, N., “Meganet: Multi-scale edge-guided attention network for weak boundary polyp segmentation”, In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 7985-7994, 2024.
6. Hu, Z., Tang, J., Wang, Z., Zhang, K., Zhang, L., & Sun, Q., “Deep learning for image-based cancer detection and diagnosis− A survey”, Pattern Recognition, 83, 134-149, 2018.
7. Nault, J. C., Bioulac–Sage, P. A. U. L. E. T. T. E., & Zucman–Rossi, J. E. S. S. I. C. A., “Reviews in basic and clinical gastroenterology and hepatology”, Gastroenterology, 144, 888-902. 2013.
8. Ronneberger, O., Fischer, P., & Brox, T. “U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234-241, Cham: Springer international publishing, October 2015.

9. Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N., & Liang, J., “Unet++: Redesigning skip connections to exploit multiscale features in image segmentation”, IEEE transactions on medical imaging, 39(6), 1856-1867, 2019.
10. Jha, D., Riegler, M. A., Johansen, D., Halvorsen, P., & Johansen, H. D., ”Doubleu-net: A deep convolutional neural network for medical image segmentation”, In 2020 IEEE 33rd International Symposium on computer-based medical systems (CBMS), pp. 558-564, IEEE, July 2020.
11. Huang, C. H., Wu, H. Y., & Lin, Y. L., “Hardnet-MSEG: A simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean Dice and 86 fps”, arXiv preprint arXiv:2101.07172, 2021.
12. Yu, T., & Wu, Q., “Hardnet-CPS: colorectal polyp segmentation based on harmonic densely united network”, Biomedical Signal Processing and Control, 85, 104953, 2023.
13. Ta, N., Chen, H., Lyu, Y., & Wu, T., “BLE-Net: Boundary learning and enhancement network for polyp segmentation”, Multimedia Systems, 29(5), 3041-3054, 2023.
14. Zhou, T., Zhou, Y., He, K., Gong, C., Yang, J., Fu, H., & Shen, D., “Cross-level feature aggregation network for polyp segmentation”, Pattern Recognition, 140, 109555, 2023.
15. Zhao, X., et al., “M2SNet: Multi-scale in multi-scale subtraction network for medical image segmentation”, arXiv preprint arXiv:2303.10894, 2023.
16. Chen, W., Zhang, R., Zhang, Y., Bao, F., Lv, H., Li, L., & Zhang, C., “Pact-Net: Parallel CNNs and Transformers for medical image segmentation”, Computer Methods and Programs in Biomedicine, 242, 107782, 2023.
17. Fan, D. P., Ji, G. P., Zhou, T., Chen, G., Fu, H., Shen, J., & Shao, L. “PraNet: Parallel reverse attention network for polyp segmentation”, In International conference on medical image computing and computer-assisted intervention, pp. 263-273, Cham: Springer International Publishing, September 2020.
18. Wei, J., Hu, Y., Zhang, R., Li, Z., Zhou, S. K., & Cui, S. “Shallow attention network for polyp segmentation”, In International conference on medical image computing and computer-assisted intervention, pp. 699-708, Cham: Springer International Publishing, September 2021.
19. Fang, Y., Chen, C., Yuan, Y., & Tong, K. Y. “Selective feature aggregation network with area-boundary constraints for polyp segmentation”, In International conference on medical image computing and computer-assisted intervention, pp. 302-310. Cham: Springer International Publishing, October 2019.
20. Gao, S. H., Cheng, M. M., Zhao, K., Zhang, X. Y., Yang, M. H., & Torr, P., “Res2net: A new multi-scale backbone architecture”, IEEE transactions on pattern analysis and machine intelligence, 43(2), 652-662, 2019.
21. Jha, D., Smedsrud, P. H., Riegler, M. A., Halvorsen, P., De Lange, T., Johansen, D., & Johansen, H. D. “Kvasir-SEG: A segmented polyp dataset”, In International conference on multimedia modeling, pp. 451-462, Cham: Springer International Publishing, December 2019.
22. Bernal, J., Sánchez, J., & Vilarino, F. “Impact of image preprocessing methods on polyp localization in colonoscopy frames”, In 2013 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp. 7350-7354, IEEE. July 2013.
23. Chien-Hsiang Huang, Hung-Yu Wu, Youn-Long Lin, “Dataset: ETIS-Larib Polyp DB”, https://doi.org/10.57702/pqx39a6l, 2024.
24. Vázquez, D., et al., “A benchmark for endoluminal scene segmentation of colonoscopy images”, Journal of healthcare engineering, 2017(1), 4037190, 2017.