UKnow-Net: Knowledge-Enhanced U-Net for Improved Retinal Vessel Segmentation

Abstract

Retinal vessel segmentation plays a critical role in diagnosing and managing ophthalmic and systemic diseases, as abnormalities in retinal vasculature can indicate disease progression. Traditional manual segmentation by expert ophthalmologists is time-consuming, labor-intensive, and prone to variability, underscoring the need for automated methods. While deep learning approaches like U-Net have advanced retinal vessel segmentation, they often struggle to generalize across diverse datasets due to differences in image acquisition techniques, resolutions, and patient demographics. To address these challenges, I propose UKnow-Net, a knowledge-enhanced U-Net architecture designed to improve retinal vessel segmentation across multiple datasets. UKnow-Net employs a multi-step process involving knowledge distillation and enhancement techniques. First, I train four specialized teacher networks separately on four publicly available retinal vessel segmentation datasets—DRIVE, CHASE_DB1, DCA1, and CHUAC—allowing each to specialize in the unique features of its respective dataset. These teacher networks generate pseudo-labels representing their domain-specific knowledge. We then train a student network using the ensemble of pseudo-labels from all teacher networks, effectively distilling the collective expertise into a unified model capable of generalizing across different datasets. Experiments demonstrate that UKnow-Net outperforms traditional handcrafted networks (such as U-Net, UNet++, and Attention U-Net) and several state-of-the-art models in key performance metrics, including sensitivity, specificity, F1 score, and Intersection over Union (IoU). Specifically, our two variants, UKnowNet-A and UKnowNet-B, show well performance; UKnowNet-A, trained solely on pseudo-labels, achieved higher sensitivity across all datasets, indicating a superior ability to detect true positives, while UKnowNet-B, which combines pseudo-labels with ground truth annotations, achieved balanced precision and recall, leading to higher F1 scores and IoU metrics. The integration of pseudo-labels effectively transfers the collective expertise of the teacher networks to the student network, enhancing generalization and robustness. I aim to ensure fair comparison and reproducibility in future research by publicly sharing our source code and model weights.

Keywords

• Retinal Vessel Segmentation
• Knowledge Distillation and Enhancement
• Semi-supervised Learning

References

1. Abràmoff, M. D., Garvin, M. K., & Sonka, M. (2010). Retinal imaging and image analysis. IEEE Reviews in Biomedical Engineering, 3, 169-208. https://doi.org/10.1109/rbme.2010.2084567
2. Amritesh, Owais, M. M., Vemula, V., Amit, A., & Natarajan, S. (2023, May 3-5). Localised Land-Use Classification Using U-Net and Satellite Imaging. In: A. J. Kulkarni, & N. Cheikhrouhou (Eds.), Proceedings of the 2nd International Conference on Information Science and Applications (ICISA 2023), (pp. 235-248), Pune, India. https://doi.org/10.1007/978-981-99-6984-5_15
3. Anand, V., Gupta, S., Koundal, D., Nayak, S. R., Barsocchi, P., & Bhoi, A. K. (2022). Modified U-net architecture for segmentation of skin lesion. Sensors, 22(3), 867. https://doi.org/10.3390/s22030867
4. Carballal, A., Novoa, F. J., Fernandez-Lozano, C., García-Guimaraes, M., Aldama-López, G., Calviño-Santos, R., Vazquez-Rodriguez, J. M., & Pazos, A. (2018). Automatic multiscale vascular image segmentation algorithm for coronary angiography. Biomedical Signal Processing and Control, 46, 1-9. https://doi.org/10.1016/j.bspc.2018.06.007
5. Cervantes-Sanchez, F., Cruz-Aceves, I., Hernandez-Aguirre, A., Hernandez-Gonzalez, M. A., & Solorio-Meza, S. E. (2019). Automatic segmentation of coronary arteries in X-ray angiograms using multiscale analysis and artificial neural networks. Applied Sciences, 9(24), 5507. https://doi.org/10.3390/app9245507
6. Chaudhuri, S., Chatterjee, S., Katz, N., Nelson, M., & Goldbaum, M. (1989). Detection of blood vessels in retinal images using two-dimensional matched filters. IEEE Transactions on Medical Imaging, 8(3), 263-269. https://doi.org/10.1109/42.34715
7. Fraz, M. M., Remagnino, P., Hoppe, A., Uyyanonvara, B., Rudnicka, A. R., Owen, C. G., & Barman, S. A. (2012). Blood vessel segmentation methodologies in retinal images–a survey. Computer Methods and Programs in Biomedicine, 108(1), 407-433. https://doi.org/10.1016/j.cmpb.2012.03.009
8. Fu, H., Xu, Y., Lin, S., Kee Wong, D. W., & Liu, J. (2016, October 17-21). Deepvessel: Retinal vessel segmentation via deep learning and conditional random field. In: S. Ourselin, L. Joskowicz, M. R. Sabuncu, G. Unal, & W. Wells (Eds.), Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2016), (Part II, pp. 132-139). Athens, Greece. https://doi.org/10.1007/978-3-319-46723-8_16

9. Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network. https://doi.org/10.48550/arXiv.1503.02531
10. Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., & Maier-Hein, K. H. (2018, September 16). No New-Net. In: A. Crimi, S. Bakas, H. Kuijf, F. Keyvan, M. Reyes, & T. van Walsum (Eds.), Proceedings of the 4th International Workshop on Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries (BrainLes 2018), (Part II, pp. 234-244), Granada, Spain. https://doi.org/10.1007/978-3-030-11726-9_21
11. Jaeger, P. F., Kohl, S. A. A., Bickelhaupt, S., Isensee, F., Kuder, T. A., Schlemmer, H.-P., & Maier-Hein, K. H. (2019, December 13). Retina U-Net: Embarrassingly simple exploitation of segmentation supervision for medical object detection. In: Proceedings of the Machine Learning for Health Workshop (ML4H), (pp. 171-183), Vancouver, Canada.
12. Kamran, S. A., Hossain, K. F., Tavakkoli, A., Zuckerbrod, S. L., Sanders, K. M., & Baker, S. A. (2021, September 27 - October 1). RV-GAN: Segmenting retinal vascular structure in fundus photographs using a novel multi-scale generative adversarial network. In: M. de Bruijne, P. C. Cattin, S. Cotin, N. Padoy, S. Speidel, Y. Zheng, & C. Essert (Eds.), Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2021), (Part VIII, pp. 34-44), Strasbourg, France. https://doi.org/10.1007/978-3-030-87237-3_4
13. Kuş, Z., & Kiraz, B. (2023). Evolutionary architecture optimization for retinal vessel segmentation. IEEE Journal of Biomedical and Health Informatics, 27(2), 5895-5903 https://doi.org/10.1109/JBHI.2023.3314981
14. Li, J., Gao, G., Liu, Y., & Yang, L. (2023). MAGF-Net: A multiscale attention-guided fusion network for retinal vessel segmentation. Measurement, 206, 112316. https://doi.org/10.1016/j.measurement.2022.112316
15. Liskowski, P., & Krawiec, K. (2016). Segmenting retinal blood vessels with deep neural networks. IEEE Transactions on Medical Imaging, 35(11), 2369-2380. https://doi.org/10.1109/TMI.2016.2546227
16. Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60-88. https://doi.org/10.1016/j.media.2017.07.005
17. Liu, W., Yang, H., Tian, T., Cao, Z., Pan, X., Xu, W., Jin, Y., & Gao, F. (2022). Full-resolution network and dual-threshold iteration for retinal vessel and coronary angiograph segmentation. IEEE Journal of Biomedical and Health Informatics, 26(9), 4623-4634. https://doi.org/10.1109/JBHI.2022.3188710
18. Mendonca, A. M., & Campilho, A. (2006). Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction. IEEE Transactions on Medical Imaging, 25(9), 1200-1213. https://doi.org/10.1109/TMI.2006.879955
19. Mou, L., Zhao, Y., Chen, L., Cheng, J., Gu, Z., Hao, H., Qi, H., Zheng, Y., Frangi, A., & Liu, J. (2019, October 13–17). CS-Net: Channel and spatial attention network for curvilinear structure segmentation. In: D. Shen, T. Liu, T. M. Peters, L. H. Staib, C. Essert, S. Zhou, P-T. Yap, & A. Khan (Eds.), Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2019), (Part I, pp. 721-730), Shenzhen, China. https://doi.org/10.1007/978-3-030-32239-7_80
20. Mu, Y., Li, K., Sun, Y., & Bao, Y. (2024). Semantic segmentation of corn leaf blotch disease images based on U-Net integrated with RFB structure and dual attention mechanism. Agronomy (Basel, Switzerland), 14(11), 2652. https://doi.org/10.3390/agronomy14112652
21. Niemeijer, M., Staal, J., van Ginneken, B., Loog, M., & Abramoff, M. D. (2004, February 14-19). Comparative study of retinal vessel segmentation methods on a new publicly available database. In: J. M. Fitzpatrick & M. Sonka (Eds.), Proceedings of the Medical Imaging 2004: Image Processing. SPIE Proceedings, (Vol. 5370, pp. 648-656), San Diego, California, United States. https://doi.org/10.1117/12.535349
22. Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., Glocker, B., & Rueckert, D. (2018, July 4-6). Attention u-net: Learning where to look for the pancreas. In: Proceedings of the 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherlands. https://doi.org/10.48550/arXiv.1804.03999
23. Patton, N., Aslam, T. M., MacGillivray, T., Deary, I. J., Dhillon, B., Eikelboom, R. H., Yogesan, K., & Constable, I. J. (2006). Retinal image analysis: concepts, applications and potential. Progress in Retinal and Eye Research, 25(1), 99-127. https://doi.org/10.1016/j.preteyeres.2005.07.001
24. Qin, D., Bu, J.-J., Liu, Z., Shen, X., Zhou, S., Gu, J.-J., Wang, Z.-H., Wu, L., & Dai, H.-F. (2021). Efficient medical image segmentation based on knowledge distillation. IEEE Transactions on Medical Imaging, 40(12), 3820-3831. https://doi.org/10.1109/TMI.2021.3098703
25. Qu, Z., Zhuo, L., Cao, J., Li, X., Yin, H., & Wang, Z. (2023). TP-net: Two-path network for retinal vessel segmentation. IEEE Journal of Biomedical and Health Informatics, 27(4), 1979-1990. https://doi.org/10.1109/JBHI.2023.3237704
26. Ronneberger, O., Fischer, P., & Brox, T. (2015, October 5-9). U-net: Convolutional networks for biomedical image segmentation. In: N. Navab, J. Hornegger, W. M. Wells, & A. F. Frangi (Eds.), Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), (Part III, pp. 234-241), Munich, Germany. https://doi.org/10.1007/978-3-319-24574-4_28
27. Samuel, P. M., & Veeramalai, T. (2021). VSSC Net: vessel specific skip chain convolutional network for blood vessel segmentation. Computer Methods and Programs in Biomedicine, 198, 105769. https://doi.org/10.1016/j.cmpb.2020.105769
28. Shen, L., Margolies, L. R., Rothstein, J. H., Fluder, E., McBride, R., & Sieh, W. (2019). Deep learning to improve breast cancer detection on screening mammography. Scientific Reports, 9(1), 12495. https://doi.org/10.1038/s41598-019-48995-4
29. Staal, J., Abràmoff, M. D., Niemeijer, M., Viergever, M. A., & van Ginneken, B. (2004). Ridge-based vessel segmentation in color images of the retina. IEEE Transactions on Medical Imaging, 23(4), 501-509. https://doi.org/10.1109/TMI.2004.825627
30. Sun, K., Xiao, B., Liu, D., & Wang, J. (2019, June 15-20). Deep high-resolution representation learning for human pose estimation. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5693-5703), Long Beach, CA, USA. https://doi.org/10.1109/CVPR.2019.00584
31. Wang, L., & Yoon, K.-J. (2021). Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(6), 3048-3068. https://doi.ieeecomputersociety.org/10.1109/TPAMI.2021.3055564
32. Wang, W., Zhong, J., Wu, H., Wen, Z., & Qin, J. (2020, October 4–8). RVSeg-net: An efficient feature pyramid cascade network for retinal vessel segmentation. In: A. L. Martel, P. Abolmaesumi, D. Stoyanov, D. Mateus, M. A. Zuluaga, S. K. Zhou, D. Racoceanu, & L. Joskowicz (Eds.), Proceedings of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2020), (Part V, pp. 796-805), Lima, Peru. https://doi.org/10.1007/978-3-030-59722-1_77
33. Wu, H., Wang, W., Zhong, J., Lei, B., Wen, Z., & Qin, J. (2021). SCS-net: A scale and context sensitive network for retinal vessel segmentation. Medical Image Analysis, 70, 102025. https://doi.org/10.1016/j.media.2021.102025
34. Zana, F., & Klein, J.-C. (2001). Segmentation of vessel-like patterns using mathematical morphology and curvature evaluation. IEEE Transactions on Image Processing, 10(7), 1010-1019. https://doi.org/10.1109/83.931095
35. Zhang, S., Fu, H., Yan, Y., Zhang, Y., Wu, Q., Yang, M., Tan, M., & Xu, Y. (2019, October 13-17). Attention guided network for retinal image segmentation. In: D. Shen, T. Liu, T. M. Peters, L. H. Staib, C. Essert, S. Zhou, P-T. Yap, & A. Khan (Eds.), Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2019), (Part I, pp. 797-805), Shenzhen, China. https://doi.org/10.1007/978-3-030-32239-7_88
36. Zhang, Z., & Lu, B. (2024). Efficient skin lesion segmentation with boundary distillation. Medical & Biological Engineering & Computing, 62(9), 2703–2716. https://doi.org/10.1007/s11517-024-03095-y
37. Zhou, Y., Yu, H., & Shi, H. (2021, September 27–October 1). Study Group Learning: Improving Retinal Vessel Segmentation Trained with Noisy Labels. In: M. de Bruijne, P. C. Cattin, S. Cotin, N. Padoy, S. Speidel, Y. Zheng, & C. Essert (Eds.), Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2021), (Part I, pp. 57-67), Strasbourg, France. https://doi.org/10.1007/978-3-030-87193-2_6
38. Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N., & Liang, J. (2018, September 20). Unet++: A nested u-net architecture for medical image segmentation. In: D. Stoyanov, Z. Taylor, G. Carneiro, T. Syeda-Mahmood, A. Martel, L. Maier-Hein, J. M. R. S. Tavares, A. Bradley, J. P. Papa, V. Belagiannis, J. C. Nascimento, Z. Lu, S. Conjeti, M. Moradi, H. Greenspan, & A. Madabhushi (Eds.), Proceedings of the 4th International Workshop and 8th International Workshop on Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support (DLMIA 2018, ML-CDS 2018), (pp. 3-11). Granada, Spain. https://doi.org/10.1007/978-3-030-00889-5_1