BRAIN TUMOR SEGMENTATION USING U-NET-BASED DEEP LEARNING MODELS

Abstract

When examining segmentation methods developed for brain tumor detection, a significant gap exists in the comparative performance analysis of standard and hybrid U-Net models. This study systematically evaluates four Convolutional Neural Network (CNN) models to address this gap, particularly regarding the practical effectiveness of integrating complex, pre-trained encoders: Classic U-Net, U-Net+VGG-16, U-Net+ResNet50, and U-Net++. The BraTS 2019 dataset was used for training and testing the models (with 5-fold cross-validation), and the Figshare dataset was used for additional validation, performing tumor segmentation and generating predictions. U-Net++ outperformed both the conventional U-Net and the hybrid models, indicating its higher segmentation performance under the conditions evaluated in this study. While U-Net++ achieved superior results in Dice coefficient, sensitivity, specificity, and Jaccard Index values (0.920, 0.890, 0.973, and 0.850, respectively), the U-Net+VGG-16 hybrid model (0.697, 0.659, 0.862, and 0.535, respectively) and the U-Net+ResNet50 hybrid model (0.585, 0.550, 0.800, and 0.414) produced lower Dice scores. When we examine the primary reasons for this poor performance output, we identify and discuss these as the incompatibility of 2D encoder blocks with 3D medical data and the lack of unoptimized transfer learning. We believe this finding is of critical importance for future model designs. Our study demonstrates that the U-Net++ architecture, with its dense and intertwined skip connections, provides a more effective solution than simple hybrid models built with powerful classifiers such as VGG-16 and ResNet50. This allows us to obtain extremely important results for precise brain tumor segmentation.

Keywords

• Brain Tumor
• Computer-Aided Diagnosis
• Convolutional Neural Networks
• Image Processing
• Magnetic Resonance
• U-Net.

References

1. B. Alberts, A. Johnson, J. Lewis, M. Raff, K. Roberts and P. Walter, “Molecular biology of the cell (6th ed.)”, Garland Science, 2014.
2. R. Paudyal, A. D. Shah, O. Akin, R. K. G. Do, A. S. Konar, V. Hatzoglou, U. Mahmood, N. Lee, R. J. Wong, S. Banerjee, J. Shin, H. Veeraraghavan and A. Shukla-Dave, “Artificial intelligence in CT and MR imaging for oncological applications”, Cancers, vol. 15, no. 9, pp. 2573, 2023, doi: https://doi.org/10.3390/cancers15092573.
3. C. Liao and L. Kong "A comprehensive survey of semantic segmentation based on computer vision", Proc. SPIE 13486, Fourth International Conference on Computer Vision, Application, and Algorithm (CVAA 2024), 1348612, 2025, doi: https://doi.org/10.1117/12.3055943.
4. A. I. Khan, J. L. Shah and M. M. Bhat, “Coronet: A deep neural network for detection and diagnosis of COVID-19 from chest X-ray images”, Computer Methods and Programs in Biomedicine, vol. 196, pp. 105581, 2020, doi: https://doi.org/10.1016/j.cmpb.2020.105581.
5. A. Esteva, K. Chou, S. Yeung, N. Naik, A. Madani, A. Mottaghi, Y. Liu, E. Topol, J. Dean and R. Socher, “Deep learning-enabled medical computer vision”, NPJ Digital Medicine, vol. 4, no. 1, 1-9, 2021, doi: https://doi.org/10.1038/s41746-020-00376-2.
6. O. Ronneberger, P. Fischer and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation”, In: Navab, N., Hornegger, J., Wells, W., Frangi, A. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015, Lecture Notes in Computer Science, 9351. Springer, Cham., 2015, doi: https://doi.org/10.1007/978-3-319-24574-4_28.
7. Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox and O. Ronneberger, “3D U-Net: Learning dense volumetric segmentation from sparse annotation”, Lecture Notes in Computer Science, vol. 9901, 424-432, 2016, doi: https://doi.org/10.1007/978-3-319-46723-8_49.
8. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition”, Computer Vision and Pattern Recognition, pp. 1-14, 2015, doi: https://doi.org/10.48550/arXiv.1409.1556.

9. K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 770-778, 2016, doi: https://doi.org/10.1109/CVPR.2016.90.
10. H. Dong, G. Yang, F. Liu, Y. Mo and Y. Guo, “Automatic brain tumor segmentation using U-Net based fully convolutional networks”, Computer Vision and Pattern Recognition, pp. 1-12, 2017, doi: https://doi.org/10.48550/arXiv.1705.03820.
11. V. Iglovikov and A. Shvets, “TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation”,Computer Vision and Pattern Recognition, 1-5, 2018, doi: https://doi.org/10.48550/arXiv.1801.05746.
12. Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh and J. Liang, “UNet++: A nested U-Net architecture for medical image segmentation”, In: Stoyanov, D., et al. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. DLMIA ML-CDS 2018 2018, Lecture Notes in Computer Science, 11045. Springer, Cham., 2018, doi: https://doi.org/10.1007/978-3-030-00889-5_1.
13. S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi and B. H. Menze, “Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge”, Computer Vision and Pattern Recognition, pp. 1-49, 2019, doi: https://doi.org/10.17863/CAM.38755.
14. C. Jun, “Brain tumor dataset”, Figshare, 2017, https://figshare.com/articles/dataset/brain_tumor_dataset/1512427.
15. H. Mujtaba, “Introduction to ResNet or residual network”, https://www.mygreatlearning.com/blog/resnet/#sh1. Great Learning, 2020, Access Date: 01.8.2021.
16. A. Zhang, Z. C. Lipton, M. Li and A. J. Smola, “Dive into deep learning”, Machine Learning, pp. 1-1185, 2020, doi: https://doi.org/10.48550/arXiv.2106.11342.
17. Jiang, Z., Ding, C., Liu, M., Tao, D. (2020). Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task. In: Crimi, A., Bakas, S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2019. Lecture Notes in Computer Science, vol 11992.
18. E. Shelhamer, J. Long and T. Darrell, “Fully convolutional networks for semantic segmentation," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 640-651, 2017, doi: https://doi.org/10.1109/TPAMI.2016.2572683.
19. L. Liu and K. Jia, "BTIS-Net: Efficient 3D U-Net for Brain Tumor Image Segmentation," in IEEE Access, vol. 12, pp. 133392-133405, 2024.
20. C.Y. Lee, S. Xie, P. Gallagher, Z. Zhang and Z. Tu, “Deeply-supervised nets”, In Artificial Intelligence and Statistics, vol. 38, pp. 562–570, 2015, doi: https://doi.org/10.48550/arXiv.1409.5185.
21. M. Awad and R. Khanna, Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers. New York, NY, USA: Apress, 2015.
22. L.C. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A. L. Yuille, “Rethinking atrous convolution for semantic image segmentation”, Computer Vision and Pattern Recognition, pp. 1-14, 2017, doi: https://doi.org/10.48550/arXiv.1706.05587.
23. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto and H. Adam, “MobileNets: Efficient convolutional neural networks for mobile vision applications”, Computer Vision and Pattern Recognition, pp. 1-9, 2017, doi: https://doi.org/10.48550/arXiv.1704.04861.
24. B. Jena, S. Jain, G. K. Nayak, and S. Saxena, "Analysis of depth variation of U-NET architecture for brain tumor segmentation," Multimedia Tools and Applications, vol. 82, pp. 10723-10743, Sep. 2023. doi: https://doi.org/10.1007/s11042-022-13730-1
25. N. Tajbakhsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B. Gotway and J. Liang, “Convolutional neural networks for medical image analysis: Full training or fine tuning?”, IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1299-1312, 2016, doi: https://doi.org/10.1109/TMI.2016.2535302.
26. C. H. Sudre, W. Li, T. Vercauteren, S. Ourselin and M. J. Cardoso, “Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations”, Lecture Notes in Computer Science (MICCAI), vol. 10553, pp. 240–248, 2017, doi: https://doi.org/10.1007/978-3-319-67558-9_28.
27. A. Krizhevsky, I. Sutskever and E. Hinton, “ImageNet classification with deep convolutional neural networks”, Communications of the ACM, vol. 60, pp. 84–90, 2017, doi: https://doi.org/10.1145/3065386.
28. M. Ferguson, R. Ak, T. L. Yung-Tsun and K. H. Law, “Automatic localization of casting defects with convolutional neural networks”, IEEE International Conference on Big Data, 2017, Boston.
29. Jiang, Y., Zhang, Y., Lin, X., Dong, J., Cheng, T., Liang, J.: Swinbts: A method for 3d multimodal brain tumor segmentation using swin transformer. Brain Sci. 12(6), 797 (2022)
30. Liu, X., Gao, P., Yu, T., Wang, F., Yuan, R.-Y.: Cswin-unet: Transformer unet with cross-shaped windows for medical image segmentation. Information Fusion 113, 102634 (2025)
31. Rasool, N., Bhat, J.I., Wani, N.A., Ahmad, N., Alshara, M.: Transresunet: Revolutionizing glioma brain tumor segmentation through transformer-enhanced residual unet. IEEE Access (2024)