LUPU-Net: a new improvement proposal for encoder-decoder architecture

Abstract

Many network designs in recent years have offered deeper layered solutions. However, models that achieve high-performance results with fewer layers are preferred due to causing less processing load for the system. The U-Net authors succeeded in efficiently creating a model with fewer layers. However, the U-Net architecture also requires improvement to become more efficient. For this purpose, we offer a novel encoder-decoder architecture based on the U-Net and the LU-Net. Furthermore, we propose using a reduced number of up-sampling operations, which were utilized together with the down-sampling operations intensively in the encoder section in our previous research, in the encoder part. The proposed architecture was evaluated on the IOSTAR dataset for the segmentation of retinal vessels. The preprocessing and data augmentation processes were applied to the images before training. The U-Net, LU-Net, and the proposed model were evaluated by using the accuracy, sensitivity, specificity, Dice, and Jaccard metrics. The proposed model achieved performance metric values such as an accuracy of 97.29%, a sensitivity of 81.10%, a specificity of 98.94%, a Dice coefficient of 84.66%, and a Jaccard coefficient of 73.41%. The proposed model obtained improved results compared with the other models, especially for test samples.

Keywords

• Convolutional Neural Network
• LU-Net
• Retinal Vessel
• Semantic Segmentation
• Up sampling
• U-Net

References

1. 1. McCulloch, W.S. and W. Pitts, A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 1943. 5: p. 115-133.
2. 2. Aizenberg, I.N., N.N. Aizenberg, and J. Vandewalle, Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications. 2000, USA: Kluwer Academic Publishers.
3. 3. Fukushima, K., Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 1980. 36 (4): p. 193–202.
4. 4. Simonyan, K. and A. Zisserman, Very deep convolutional networks for large-scale image recognition, in 3rd International Conference on Learning Representations, 2015, San Diego, CA: USA. p. 1-14.
5. 5. Long, J., E. Shelhamer, and T. Darrell, Fully convolutional networks for semantic segmentation, in The IEEE Conference on Computer Vision and Pattern Recognition, 2015, Boston: USA. p. 3431-3440.
6. 6. Ronneberger, O., P. Fischer, and T. Brox, U-Net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention, 2015, Munich: Germany. p. 234-241.
7. 7. Badrinarayanan, V., A. Kendall, and R. Cipolla, SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. 39 (12): p. 2481-2495.
8. 8. Khan, T.M., S.S. Naqvi, M. Arsalan, M.A. Khan, H.A. Khan, et al., Exploiting residual edge information in deep fully convolutional neural networks for retinal vessel segmentation, in International Joint Conference on Neural Networks, 2020, Glasgow: United Kingdom. p. 1-8.

9. 9. Ozgunalp, U., R. Fan, and A. Serener, Semantic segmentation of retinal vessels using SegNet, in 28th Signal Processing and Communications Applications Conference, 2020, Gaziantep: Turkey. p. 1-4.
10. 10. Xian-cheng, W., L. Wei, M. Bingyi, J. He, Z. Jiang, et al., Retina blood vessel segmentation using a U-Net based convolutional neural network, in International Conference on Data Science, 2018, Beijing: China. p. 1-11.
11. 11. Gao, X., Y. Cai, C. Qiu, and Y. Cui, Retinal blood vessel segmentation based on the gaussian matched filter and U-Net, in 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics, 2017, Shanghai: China. p. 1-5.
12. 12. Fu, W., K. Breininger, Z. Pan, and A. Maier, Degenerating U-Net on retinal vessel segmentation. [cited 2021 14 May]; Available from: https://doi.org/10.1007/978-3-658-29267-6_7.
13. 13. Mehta, R. and J. Sivaswamy, M-Net: A convolutional neural network for deep brain structure segmentation, in IEEE 14th International Symposium on Biomedical Imaging, 2017, Melbourne: Australia. p. 437-440.
14. 14. Li, L., M. Verma, Y. Nakashima, H. Nagahara, and R. Kawasaki, IterNet: Retinal image segmentation utilizing structural redundancy in vessel networks, in IEEE Winter Conference on Applications of Computer Vision, 2020, Colorado: USA. p. 3656-3665.
15. 15. Li, Q., S. Fan, and C. Chen, An intelligent segmentation and diagnosis method for diabetic retinopathy based on improved U-Net network. Journal of Medical Systems, 2019. 43: p. 304.
16. 16. Cai, Y., Y. Li, X. Gao, and Y. Guo, Retinal vessel segmentation method based on improved deep U-Net, in Chinese Conference on Biometric Recognition, 2019, Zhuzhou: China. p. 321-328.
17. 17. He, K., X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in IEEE Conference on Computer Vision and Pattern Recognition, 2016, Las Vegas: USA. p. 770-778.
18. 18. Li, D., D.A. Dharmawan, B.P. Ng, and S. Rahardja, Residual U-Net for retinal vessel segmentation, in IEEE International Conference on Image Processing, 2019, Taipei: Taiwan. p. 1425-1429.
19. 19. Xiao, X., S. Lian, Z. Luo, and S. Li, Weighted Res-UNet for high-quality retina vessel segmentation, in 9th International Conference on Information Technology in Medicine and Education, 2018, Hangzhou: China. p. 327-331.
20. 20. Huang, G., Z. Liu, L. Van Der Maaten, and K.Q. Weinberger, Densely connected convolutional networks, in IEEE Conference on Computer Vision and Pattern Recognition, 2017, Honolulu: USA. p. 2261-2269.
21. 21. Cheng, Y., M. Ma, L. Zhang, C. Jin, L. Ma, and Y. Zhou, Retinal blood vessel segmentation based on densely connected U-Net. Mathematical Biosciences and Engineering, 2020. 17 (4): p. 3088-3108.
22. 22. Wang, C., Z. Zhao, Q. Ren, Y. Xu, and Y. Yu, Dense U-Net based on patch-based learning for retinal vessel segmentation. Entropy, 2019. 21 (2): p. 168.
23. 23. Zhou, Z., M.M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, UNet++: A nested U-Net architecture for medical image segmentation, in 4th International Workshop on Deep Learning in Medical Image Analysis, 2018, Granada: Spain. p. 3-11.
24. 24. Arpacı, S.A. and S. Varlı, Retinal vessel segmentation with differentiated U-Net network, in 28th Signal Processing and Communications Applications Conference, 2020, Gaziantep: Turkey. p. 1-4.
25. 25. Zhang, J., J. Du, H. Liu, X. Hou, Y. Zhao, et al., LU-NET: An improved U-Net for ventricular segmentation. IEEE Access, 2019. 7: p. 92539-92546.
26. 26. Hu, J., L. Shen, and G. Sun, Squeeze-and-excitation networks, in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Salt Lake City: USA. p. 7132-7141.
27. 27. Kingma, D.P. and J.L. Ba, ADAM: A method for stochastic optimization. [cited 2021 14 May]; Available from: https://arxiv.org/pdf/1412.6980.pdf .
28. 28. Keras library. [cited 2021 14 May]; Available from: https://keras.io/ .
29. 29. TensorFlow library. [cited 2021 14 May]; Available from: https://www.tensorflow.org/ .
30. 30. IOSTAR retinal vessel segmentation dataset. [cited 2019 5 April]; Available from: http://www.retinacheck.org/download-iostar-retinal-vessel-segmentation-dataset .
31. 31. Zhang, J., B. Dashtbozorg, E. Bekkers, J.P.W. Pluim, R. Duits, et al., Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores. IEEE Transactions on Medical Imaging, 2016. 35 (12): p. 2631-2644.
32. 32. Abbasi-Sureshjani, S., I. Smit-Ockeloen, J. Zhang, and B. Ter Haar Romeny, Biologically-inspired supervised vasculature segmentation in SLO retinal fundus images, in 12th International Conference Image Analysis and Recognition, 2015, Niagara Falls: Canada. p. 325-334.
33. 33. Zhou, M., K. Jin, S. Wang, J. Ye, and D. Qian, Color retinal image enhancement based on luminosity and contrast adjustment. IEEE Transactions on Biomedical Engineering, 2018. 65 (3): p. 521-527.
34. 34. OpenCV library. [cited 2021 14 May]; Available from: https://opencv.org/ .
35. 35. Bloice, M.D., C. Stocker, and A. Holzinger, Augmentor: An Image Augmentation Library for Machine Learning. [cited 2021 14 May]; Available from: https://arxiv.org/abs/1708.04680 .
36. 36. Soomro, T.A., A. J. Afifi, J. Gao, O. Hellwich, M. Paul, and L. Zheng, Strided U-Net Model: Retinal Vessels Segmentation using Dice Loss, in Digital Image Computing: Techniques and Applications, 2018, Canberra: Australia. p. 1-8.
37. 37. Sorensen, T., A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on Danish commons. Biologiske Skrifter, 1948.5: p.1–34.
38. 38. Jaccard, P., Lois de distribution florale dans la zone alpine. Bull. Société Vaudoise Sci. Nat., 1902. 38: p. 69–130.
39. 39. Meyer, M.I., P. Costa, A. Galdran, A.M. Mendonça, and A. Campilho, A deep neural network for vessel segmentation of scanning laser ophthalmoscopy images, in International Conference on Image Analysis and Recognition, 2017, Montreal: Canada. p. 507-515.
40. 40. Guo, C., M. Szemenyei, Y. Yi, Y. Xue, W. Zhou, et al., Dense residual network for retinal vessel segmentation, in IEEE International Conference on Acoustics, Speech and Signal Processing, 2020, Barcelona: Spain. p. 1374-1378.
41. 41. Brea, L.S., D.A. De Jesus, S. Klein, and Tv. Walsum, Deep learning-based retinal vessel segmentation with cross-modal evaluation, in Proceedings of the Third Conference on Medical Imaging with Deep Learning, 2020, Montreal: Canada. p. 709-720.
42. 42. Kim, J.U., H.G. Kim, and Y.M. Ro, Iterative deep convolutional encoder-decoder network for medical image segmentation, in 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2017, Jeju: Korea (South). p. 685-688.
43. 43. Khan, K.B., A.A. Khaliq, A. Jalil, M.A. Iftikhar, N. Ullah, et al., A review of retinal blood vessels extraction techniques: challenges, taxonomy, and future trends. Pattern Analysis and Applications, 2019. 22 (3): p. 767-802.
44. 44. Oliveira, A., S. Pereira, and C.A. Silva, Augmenting data when training a CNN for retinal vessel segmentation: How to warp?, in IEEE 5th Portuguese Meeting on Bioengineering, 2017, Coimbra. p. 1-4.
45. 45. Arpacı, S.A. and S. Varlı, Diabetic retinopathy classification with deep learning, in 4th International Scientific Research Congress, 2019, Yalova: Turkey. p. 311-321.