Karaciğer tümör segmentasyonu için veri füzyonunun ResUNet modeli üzerindeki etkisinin araştırılması

Öz

Tıbbi görüntü segmentasyonu, hastalığın teşhisi veya hastalıklı bölgenin konumlandırılması amacıyla, görüntüdeki renk ve şekil farklılıklarını kullanarak bölgeleri ayırma işlemidir. Bu işlem manuel veya otomatik olarak gerçekleştirilebilir. Günümüzde makine öğrenmesi ve derin öğrenme tekniklerini kullanan otomatik segmentasyon yöntemlerinde, performansı artırmak amacıyla alana özgü modeller geliştirilmekte olup, tıbbi veri setlerinde U-Net tabanlı segmentasyon mimarileri sınırlı ve dengesiz veri ile bile etkili sonuçlar verebilmektedir. Ancak U-Net mimarisinde, derin ağlarda gradyan sönmesi gibi eğitim zorlukları ortaya çıkabilmektedir; bu noktada ResNet mimarisi, daha derin bir yapı sağlayarak derinlik gereksinimlerini karşılamaktadır. Bu mimarilerin birleştirilmesiyle oluşan hibrit ResUNet mimarisi U-Net’in segmentasyon gücünü ResNet’in artık bağlantıları ile birleştirerek, hem derin ağların avantajını kullanmakta hem de eğitim sürecindeki zorlukları hafifletmektedir. Bu çalışmada, otomatik karaciğer tümör segmentasyonu amacıyla, Temel Bileşenler Analizi (PCA) ve Ayrık Dalgacık Dönüşümü (DWT) ile kanal bazlı birleştirilen veriler üzerinde hibrit ResUNet modeli uygulanmıştır. Her kanalın özgün ve ayırt edici örüntülerini koruyarak özellik temsillerini zenginleştirmek amacıyla kanal bazlı veri füzyonu kullanılmıştır. PCA ve DWT tabanlı her iki füzyon yöntemi de, verileri farklı uzaylara taşıyarak modelin görüntüdeki farklı yapıları ayırt etme kapasitesini güçlendirmiştir. Sonuçlar, her iki yöntemin de iki farklı veri setinde birbirine yakın dice benzerlik katsayısı değerleri elde ederek karşılaştırılabilir performans sergilediğini ortaya koymaktadır.

Anahtar Kelimeler

• Tıbbi görüntü segmentasyonu
• Ayrık Dalgacık Dönüşümü ile görüntü füzyonu
• Karaciğer tümörü segmentasyonu
• Temel Bileşenler Analizi ile görüntü füzyonu
• ResUNet modeli

Investigating the impact of data fusion on the ResUNet model for liver tumor segmentation

Öz

Medical image segmentation is employed to separate regions in images based on color and shape differences for disease diagnosis or localization of pathological areas. It can be performed manually or automatically. Automatic segmentation methods leverage machine learning and deep learning techniques, with domain-specific models developed to enhance performance; U-Net-based architectures can achieve effective results even with limited and imbalanced medical datasets. However, U-Net models may face training challenges such as vanishing gradients in deep networks, which can be addressed by ResNet architectures providing deeper structures. The hybrid ResUNet combines the segmentation capabilities of U-Net with the residual connections of ResNet, thus exploiting the advantages of deep networks while mitigating training difficulties. In this study, for automatic liver tumor segmentation, the hybrid ResUNet was applied to channel-based fused data obtained using Principal Component Analysis (PCA) and Discrete Wavelet Transform (DWT). Channel-based data fusion preserves the unique and distinctive patterns of each channel, enriching feature representations, and both PCA- and DWT-based fusion methods transform the data into different spaces, enhancing the model’s ability to differentiate various structures. The results demonstrate that both methods achieve comparable performance, yielding similar dice similarity coefficient values across two different datasets.

Anahtar Kelimeler

• Medical image segmentation
• Discrete Wavelet Transform image fusion
• Liver tumor segmentation
• Principal Component Analysis image fusion
• ResUNet model

Kaynakça

1. 1. Sayiner M., Golabi P., Younossi Z.M., Disease burden of hepatocellular carcinoma: a global perspective, Digestive Diseases and Sciences, 64, 910-917, 2019.
2. 2. Li X., Huang Y., Tian J., H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes, IEEE Transactions on Medical Imaging, 37 (12), 2663-2674, 2018.
3. 3. Sefti R., Sbibih D., Jennane R., An automatic B-snake model based on deep learning for medical image segmentation, Expert Systems with Applications, 2025.
4. 4. Karakoyun M., Gülcü Ş., Kodaz H., D-MOSG: Discrete multi-objective shuffled gray wolf optimizer for multi-level image thresholding, Engineering Science and Technology, an International Journal, 24 (6), 1455-1466, 2021.
5. 5. Mittal M., Garg A., Sofat S., Goyal L.M., Deep learning based enhanced tumor segmentation approach for MR brain images, Applied Soft Computing, 78, 346-354, 2019.
6. 6. Fan T., Wang G., Li Y., Wang H., Ma-net: A multi-scale attention network for liver and tumor segmentation, IEEE Access, 8, 179656-179665, 2020.
7. 7. Pereira S., Pinto A., Alves V., Silva C.A., Brain tumor segmentation using convolutional neural networks in MRI images, IEEE Transactions on Medical Imaging, 35 (5), 1240-1251, 2016.
8. 8. Liu Y., Stojadinovic S., Hrycushko B., Wardak Z., Lau S., Lu W., Gu X., A deep convolutional neural network-based automatic delineation strategy for multiple brain metastases stereotactic radiosurgery, PloS One, 12 (10), e0185844, 2017.

9. 9. Shin H.C., Tenenholtz N.A., Rogers J.K., Schwarz C.G., Senjem M.L., Gunter J.L., Michalski M., Medical image synthesis for data augmentation and anonymization using generative adversarial networks, Simulation and Synthesis in Medical Imaging: Third International Workshop, SASHIMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings 3, 1-11, Springer International Publishing, 2018.
10. 10. Mok T.C., Chung A.C., Learning data augmentation for brain tumor segmentation with coarse-to-fine generative adversarial networks, BrainLesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part I 4, Springer, 2019.
11. 11. Dvornik N., Mairal J., Schmid C., On the importance of visual context for data augmentation in scene understanding, IEEE Transactions on Pattern Analysis and Machine Intelligence, 43 (6), 2014-2028, 2019.
12. 12. Agarwal M., Mahajan R., Medical images contrast enhancement using quad weighted histogram equalization with adaptive gamma correction and homomorphic filtering, Procedia Computer Science, 115, 509-517, 2017.
13. 13. Goodfellow I.J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y., Generative adversarial nets, Advances in Neural Information Processing Systems, 27, 2014.
14. 14. Nalepa J., Marcinkiewicz M., Kawulok M., Data augmentation for brain-tumor segmentation: a review, Frontiers in Computational Neuroscience, 13, 83, 2019.
15. 15. Mouraviev A., Generative adversarial network for MRI super resolution.
16. 16. Bi L., Kim J., Kumar A., Feng D., Automatic liver lesion detection using cascaded deep residual networks, arXiv preprint arXiv:1704.02703, 2017.
17. 17. Li, Y., Daho, M. E. H., Conze, P. H., Zeghlache, R., Le Boité, H., Tadayoni, R., Quellec, G., A review of deep learning-based information fusion techniques for multimodal medical image classification, Computers in Biology and Medicine, 177, 108635, 2024.
18. 18. Aakaaram V., Bachu S., MRI and CT image fusion using synchronized anisotropic diffusion equation with DT-CWT decomposition, 2022 Smart Technologies, Communication and Robotics (STCR), 1-5, 2022.
19. 19. Affane A., Kucharski A., Chapuis P., Freydier S., Lebre M.A., Vacavant A., Fabijańska A., Segmentation of liver anatomy by combining 3D U-net approaches, Applied Sciences, 11 (11), 4895, 2021.
20. 20. Huang Q., Sun J., Ding H., Wang X., Wang G., Robust liver vessel extraction using 3D U-Net with variant dice loss function, Computers in Biology and Medicine, 101, 153-162, 2018.
21. 21. Shen Y., Sheng V.S., Wang L., Duan J., Xi X., Zhang D., Cui Z., Empirical comparisons of deep learning networks on liver segmentation, Computers, Materials & Continua, 62 (3), 2020.
22. 22. Bilic P., Christ P., Li H.B., Vorontsov E., Ben-Cohen A., Kaissis G., Menze B., The liver tumor segmentation benchmark (LiTS), Medical Image Analysis, 84, 102680, 2023.
23. 23. Moghbel M., Mashohor S., Mahmud R., Saripan M.I.B., Review of liver segmentation and computer assisted detection/diagnosis methods in computed tomography, Artificial Intelligence Review, 50, 497-537, 2018.
24. 24. Çavga S.H., Performance of neural networks and heuristic models for disease prediction from liver enzymes: Application to biochemistry device output, Journal of the Faculty of Engineering and Architecture of Gazi University, 39 (4), 2263–2270, 2024.
25. 25. Grover P., 3D Liver segmentation, Kaggle Dataset, Available from: https://www.kaggle.com/datasets/prathamgrover/3d-liver-segmentation/data, Accessed on 09.01.2025.
26. 26. Tahir J., LiTS17, Kaggle Dataset, Available from: https://www.kaggle.com/datasets/javariatahir/litstrain-val, Accessed on 09.01.2025.
27. 27. Aymaz S., A new hybrid approach for multi-focus image fusion using CNN and SVM methods, Journal of the Faculty of Engineering and Architecture of Gazi University, 39 (2), 1123–1136, 2024.
28. 28. Salau A.O., Jain S., Eneh J.N., A review of various image fusion types and transform, Indonesian Journal of Electrical Engineering and Computer Science, 24 (3), 1515-1522, 2021.
29. 29. Zou Y., Chen S., Che C., Zhang J., Zhang Q., Breast cancer histopathology image classification based on dual-stream high-order network, Biomedical Signal Processing and Control, 78, 104007, 2022.
30. 30. Ünel F.B., Yalpir S., Reduction of mass appraisal criteria with principal component analysis and integration to GIS, International Journal of Engineering and Geosciences, 4 (3), 94-105, 2019.
31. 31. Abdulkareem M.B., Design and development of multimodal medical image fusion using discrete wavelet transform, 2nd International Conference on Inventive Communication and Computational Technologies (ICICCT), IEEE, 2018.
32. 32. Jaeger P.F., Kohl S.A., Bickelhaupt S., Isensee F., Kuder T.A., Schlemmer H.P., Maier-Hein K.H., Retina U-Net: Embarrassingly simple exploitation of segmentation supervision for medical object detection, Machine Learning for Health Workshop, PMLR, 171-183, Nisan 2020.
33. 33. Ghosh S., Chaki A., Santosh K., Improved U-Net architecture with VGG-16 for brain tumor segmentation, Physical and Engineering Sciences in Medicine, 44 (3), 703-712, 2021.
34. 34. Jin Q., Meng Z., Pham T.D., Chen Q., Wei L., Su R., DUNet: A deformable network for retinal vessel segmentation, Knowledge-Based Systems, 178, 149-162, 2019.
35. 35. Jin Q., Meng Z., Sun C., Cui H., Su R., RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans, Frontiers in Bioengineering and Biotechnology, 8, 605132, 2020.
36. 36. Weng Y., Zhou T., Li Y., Qiu X., Nas-unet: Neural architecture search for medical image segmentation, IEEE Access, 7, 44247-44257, 2019.
37. 37. Azad R., Asadi-Aghbolaghi M., Fathy M., Escalera S., Bi-directional ConvLSTM U-Net with densely connected convolutions, IEEE/CVF International Conference on Computer Vision Workshops, 0-0, 2019.
38. 38. Ding Y., Chen F., Zhao Y., Wu Z., Zhang C., Wu D., A stacked multi-connection simple reducing net for brain tumor segmentation, IEEE Access, 7, 104011-104024, 2019.
39. 39. Siddique N., Paheding S., Elkin C.P., Devabhaktuni V., U-net and its variants for medical image segmentation: A review of theory and applications, IEEE Access, 9, 82031-82057, 2021.
40. 40. Wightman R., Touvron H., Jégou H., Resnet strikes back: An improved training procedure in timm, arXiv preprint arXiv:2110.00476, 2021.
41. 41. Zhang C., Benz P., Argaw D.M., Lee S., Kim J., Rameau F., Kweon I.S., Resnet or densenet? Introducing dense shortcuts to resnet, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 3550-3559, 2021.
42. 42. Roy S.K., Manna S., Song T., Bruzzone L., Attention-based adaptive spectral–spatial kernel ResNet for hyperspectral image classification, IEEE Transactions on Geoscience and Remote Sensing, 59 (9), 7831-7843, 2020.
43. 43. Yu H., Sun H., Tao J., Qin C., Xiao D., Jin Y., Liu C., A multi-stage data augmentation and AD-ResNet-based method for EPB utilization factor prediction, Automation in Construction, 147, 104734, 2023.
44. 44. Keles A., Keles M.B., Keles A., COV19-CNNet and COV19-ResNet: diagnostic inference engines for early detection of COVID-19, Cognitive Computation, 2021, 1-11.
45. 45. McNeely-White D., Beveridge J.R., Draper B.A., Inception and ResNet features are (almost) equivalent, Cognitive Systems Research, 59, 312-318, 2020.
46. 46. Zhang K., Tang B., Deng L., Liu X., A hybrid attention improved ResNet based fault diagnosis method of wind turbines gearbox, Measurement, 179, 109491, 2021.
47. 47. Nguyen G.N., Le Viet N.H., Elhoseny M., Shankar K., Gupta B.B., Abd El-Latif A.A., Secure blockchain enabled Cyber–physical systems in healthcare using deep belief network with ResNet model, Journal of Parallel and Distributed Computing, 153, 150-160, 2021.
48. 48. Lu Y., Qin X., Fan H., Lai T., Li Z., WBC-Net: A white blood cell segmentation network based on UNet++ and ResNet, Applied Soft Computing, 101, 107006, 2021.
49. 49. Zhang K., Tang B., Deng L., Tan Q., Yu H., A fault diagnosis method for wind turbines gearbox based on adaptive loss weighted meta-ResNet under noisy labels, Mechanical Systems and Signal Processing, 161, 107963, 2021.
50. 50. Mandal B., Okeukwu A., Theis Y., Masked face recognition using ResNet-50, arXiv preprint arXiv:2104.08997, 2021.
51. 51. Gao M., Qi D., Mu H., Chen J., A transfer residual neural network based on ResNet-34 for detection of wood knot defects, Forests, 12(2), 212, 2021.
52. 52. Ma L., Shuai R., Ran X., Liu W., Ye C., Combining DC-GAN with ResNet for blood cell image classification, Medical & Biological Engineering & Computing, 58, 1251-1264, 2020.
53. 53. Chen X., Yao L., Zhang Y., Residual attention U-Net for automated multi-class segmentation of COVID-19 chest CT images, arXiv preprint arXiv:2004.05645, 2020.
54. 54. Lu Z., Bai Y., Chen Y., Su C., Lu S., Zhan T., Wang S., The classification of gliomas based on a pyramid dilated convolution ResNet model, Pattern Recognition Letters, 133, 173-179, 2020.
55. 55. Vuola A.O., Akram S.U., Kannala J., Mask-RCNN and U-net ensembled for nuclei segmentation, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), IEEE, 2019.
56. 56. Kerfoot E., Clough J., Oksuz I., Lee J., King A.P., Schnabel J.A., Left-ventricle quantification using residual U-Net, Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges: 9th International Workshop, STACOM 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers 9, Springer International Publishing, 371-380, 2019.
57. 57. Li D., Dharmawan D.A., Ng B.P., Rahardja S., Residual U-Net for retinal vessel segmentation, 2019 IEEE International Conference on Image Processing (ICIP), 1425-1429, 2019.
58. 58. Zhang Z., Liu Q., Wang Y., Road extraction by deep residual U-Net, IEEE Geoscience and Remote Sensing Letters, 15 (5), 749-753, 2018.
59. 59. Sharma S., Sharma S., Athaiya A., Activation functions in neural networks, Towards Data Sci, 6 (12), 310-316, 2017.
60. 60. Gustineli M., A survey on recently proposed activation functions for deep learning, arXiv preprint arXiv:2204.02921, 2022.
61. 61. Makris, A., Kontopoulos, I., Tserpes, K., COVID-19 detection from chest X-Ray images using Deep Learning and Convolutional Neural Networks, in 11th Hellenic Conference on Artificial Intelligence, 2020.
62. 62. Sankaran, K. S., Thangapandian, M., Vasudevan, N., Brain tumor grade identification using deep Elman neural network with adaptive fuzzy clustering-based segmentation approach, Multimedia Tools and Applications, 80(16), 25139-25169, 2021.
63. 63. Rahman, Z., Hussain, A., Shah, H., Arshad, M., Urdu news clustering using K-Mean algorithm on the basis of Jaccard coefficient and Dice coefficient similarity, 2022.
64. 64. Wang, L., Wang, C., Sun, Z., Chen, S., An improved dice loss for pneumothorax segmentation by mining the information of negative areas, IEEE Access, 8, 167939-167949, 2020.
65. 65. Lei, T., Wang, R., Zhang, Y., Wan, Y., Liu, C., Nandi, A. K., DefED-Net: Deformable encoder-decoder network for liver and liver tumor segmentation, IEEE Transactions on Radiation and Plasma Medical Sciences, 6 (1), 68-78, 2021.
66. 66. Zhang, C., Hua, Q., Chu, Y., Wang, P., Liver tumor segmentation using 2.5D UV-Net with multi-scale convolution, Computers in Biology and Medicine, 133, 104424, 2021.
67. 67. Chen, X., Zhang, R., Yan, P., Feature fusion encoder decoder network for automatic liver lesion segmentation, in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019, IEEE.
68. 68. Han, X., Automatic liver lesion segmentation using a deep convolutional neural network method, arXiv preprint arXiv:1704.07239, 2017.
69. 69. Alirr, O.I., Deep learning and level set approach for liver and tumor segmentation from CT scans. Journal of Applied Clinical Medical Physics, 2020, 21 (10), 200-209.
70. 70. Tummala, B.M. and S.S. Barpanda, Liver tumor segmentation from computed tomography images using multiscale residual dilated encoder‐decoder network. International Journal of Imaging Systems and Technology, 32 (2), 600-613, 2022.
71. 71. Sks-zod, Resized Liver Tumor, Kaggle Dataset, Available from: https://www.kaggle.com/datasets/skszod/resized-liver-tumor, Accessed on 10.12.2024.
72. 72. Şeker Ertuğrul Ü., Kodaz H., Discrete Wavelet Transform-Based Data Fusion with ResUNet Model for Liver Tumor Segmentation, Electronics, 14 (13), 2589, 2025.