Fusion and CNN based classification of liver focal lesions using magnetic resonance imaging phases

Abstract

The diagnosis and follow-up of focal liver lesions have an important place in radiology practice and in planning the treatment of patients. Lesions detected in the liver can be benign or malign. While benign lesions do not require any treatment, some treatments and surgical operations may be required for malign lesions. Magnetic resonance imaging provides some advantages over other imaging modalities in the detection and characterization of focal liver lesions with its superior soft tissue contrast. Additionally, different phases help make a clear diagnosis of different contrast agent retention properties in magnetic resonance imaging. This study aims to classify focal liver lesions based on convolutional neural networks by fusing magnetic resonance liver images obtained in pre-contrast, venous, arterial, and delayed phases. Magnetic resonance imaging data were obtained from Selcuk University, Faculty of Medicine, Department of Radiology in Turkey. The experiments were performed using 460 magnetic resonance images in four phases of 115 patients. Two experiments were conducted. Two-dimensional discrete wavelet transform was used to fuse the phases in both experiments. In the first experiment, the best model was determined using the original data, different number of convolution layers and different activation functions. In the second experiment, the best-found model was used. Additionally, the number of data was increased using data augmentation methods in this experiment. The results were compared with other state-of-the art methods and the superiority of the proposed method was proved. As a result of the classification, 96.66% accuracy, 86.67% sensitivity and 98.76% specificity rates were obtained. When the results are examined, CNN efficiency increases by fusing MR liver images taken in different phases.

Keywords

• Fusion
• Convolutional Neural Networks
• Discrete Wavelet Transform
• Segmentation
• Liver Lesion Classification

References

1. [1] Kabe GK, Song Y, Liu Z. Optimization of FireNet for liver lesion classification. Electronics 2020;9:1–16.[CrossRef]
2. [2] Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, et al. Convolutional neural networks for medical image analysis: Full training or fine tun-ing? IEEE Trans Med Imaging 2016;35:1299–1312. [CrossRef]
3. [3] Rofsky NM, Lee VS, Laub G, Pollack MA, Krinsky GA, Thomasson D, et al. Abdominal MR imaging with a volumetric interpolated breath-hold exami-nation. Radiology 1999;212:876–884. [CrossRef]
4. [4] Low RN. Abdominal MRI advances in the detection of liver tumours and characterization. Lancet Oncol 2007;8:525–535. [CrossRef]
5. [5] Galea N, Cantisani V, Taouli B. Liver lesion detec-tion and characterization: Role of diffusion‐weighted imaging. J Magn Reason Imaging 2013;37:1260–1276. [CrossRef]
6. [6] Li Z, Mao Y, Huang W, Li H, Zhu J, Li W, et al. Texture-based classification of different single liver lesion based on SPAIR T2W MRI images. BMC Med Imaging 2017;17:42.
7. [7] Niraj LK, Patthi B, Singla A, Gupta R, Ali I, Dhama K, et al. MRI in dentistry- A future towards radia-tion free imaging - systematic review. J Clin Diagn Res 2016;10:14–19. [CrossRef]
8. [8] Albiin N. MRI of focal liver lesions. Curr Med Imaging 2012;8:107–116. [CrossRef]

9. [9] Ozturk AE, Ceylan M. Fusion and ANN based classi-fication of liver focal lesions using phases in magnetic resonance imaging. 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI); 2015 Apr 16–19; Brooklyn, USA: IEEE; 2015. pp. 415–419.[CrossRef]
10. [10] Morlet J, Arens G, Fourgeau E, Giard D. Wave propa-gation and sampling theory-part II: Sampling theory and complex waves. Geophysics 1982;47:222–236. [CrossRef]
11. [11] Mojsilovic A, Popovic M, Sevic D. Classification of the ultrasound liver images with the 2N/spl times/1-D wavelet transform. Proceedings of 3rd IEEE International Conference on Image Processing; 1996 Sept 16-19; Lausanne, Switzerland: IEEE; 1996. pp. 367–370.
12. [12] Beura S, Majhi B, Dash R. Mammogram classi-fication using two dimensional discrete wavelet transform and gray-level co-occurrence matrix for detection of breast cancer. Neurocomputing 2015;154:1–14. [CrossRef]
13. [13] Uppal MTN. Classification of mammograms for breast cancer detection using fusion of discrete cosine transform and discrete wavelet transform features. Biomed Res 2016;27:322–327.
14. [14] Sarhan AM. Brain tumor classification in magnetic resonance images using deep learning and wavelet transform. J Biomed Eng 2020;13:102–112.[CrossRef]
15. [15] Yoshida H, Keserci B, Casalino DD, Coskun A, Ozturk O, Savranlar A. Segmentation of liver tumors in ultrasound images based on scale-space analysis of the continuous wavelet transform. 1998 IEEE Ultrasonics Symposium; 1998 Oct 5-8; Sendai, Japan: IEEE; 1998. pp. 1713–1716.
16. [16] Kutlu H, Avcı E. A novel method for classifying liver and brain tumors using convolutional neural net-works, discrete wavelet transform and long short-term memory networks. Sensors 1992;19:1–16. [CrossRef]
17. [17] Abd El Kader I, Xu G, Shuai Z, Saminu S, Javaid I, Salim Ahmad I. Differential deep convolutional neural network model for brain tumor classification. Brain Sci 2021;11:352. [CrossRef]
18. [18] Alakwaa W, Nassef M, Badr A. Lung cancer detec-tion and classification with 3D convolutional neu-ral network (3D-CNN). Int J Adv Comput Sci Appl 2017;8:409–417. [CrossRef]
19. [19] Alkhaleefah M, Wu CC. A hybrid CNN and RBF-based SVM approach for breast cancer classifica-tion in mammograms. 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC); 2018 Oct 7-10; Miyazaki, Japan: IEEE; 2018. pp. 894–899. [CrossRef]
20. [20] Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, Greenspan H. GAN-based syn-thetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 2018;321:321–331. [CrossRef]
21. [21] Li J, Liang B, Wang Y. A hybrid neural network for hyperspectral image classification. Remote Sens Lett 2020;11:96–105. [CrossRef]
22. [22] Jiang X, Chang L, Zhang YD. Classification of Alzheimer’s disease via eight-layer convolutional neural network with batch normalization and dropout techniques. J Med Imaging & Health Infor 2020;10:1040–1048. [CrossRef]
23. [23] Elgendi M, Nasir MU, Tang Q, Smith D, Grenier JP, Batte C, et al. The effectiveness of image aug-mentation in deep learning networks for detecting COVID- 19: A geometric transformation perspec-tive. Front Med 2021;8:1–12. [CrossRef]
24. [24] Ceylan M, Ozbay Y, Yıldırım E. A new approach for biomedical image segmentation: Combined complex-valued artificial neural network case study: Lung segmentation on chest CT images. 5th Cairo International Biomedical Engineering Conference; 2010 Dec 16-18; Cairo, Egypt: IEEE; 2010. pp. 33–36. [CrossRef]
25. [25] Nayak DR, Dash R, Majhi B. Brain MR image clas-sification using two-dimensional discrete wave-let transform and AdaBoost with random forests. Neurocomputing 2016;177:188–197. [CrossRef]
26. [26] Deepa R, Rajaguru H, Babu CG. Analysis on wavelet feature and softmax discriminant classi-fier for the detection of epilepsy. ICCSSS 2020: First International Conference on Circuits, Signals, Systems and Securities; 2020 Dec 11-12; Tamil Nadu, India: IOP Science; 2020. 012036. [CrossRef]
27. [27] Haghighat MBA, Aghagolzadeh A, Seyedarabi H. Multi-focus image fusion for visual sensor networks in DCT domain. Comput Electr Eng 2011;37:789–
28. 797. [CrossRef]
29. [28] Cihan M, Ceylan M. NE3D-CNN: A new 3D con-volutional neural network for hyperspectral image classification and remote sensing application. Eur J Lipid Sci Technol, 65–71.
30. [29] Sun Y, Xue B, Zhang, M, Yen GG. Evolving deep con-volutional neural networks for image classification, IEEE Transactions on Evolutionary Computation 2009;24(2):394–407. [CrossRef]
31. [30] Zhang YD, Dong Z, Chen X, Jia W, Du S, Muhammad K, et al. Image based fruit category classification by
32. 13- layer deep convolutional neural network and data augmentation. Multimed Tools Appl 2019;78:3613–3632. [CrossRef]
33. [31] Cihan M, Ceylan M, Ornek AH. Spectral-spatial classification for non-invasive health status detection of neonates using hyperspectral imaging and deep convolutional neural networks. Spectrosc Lett 2022;55:336–349. [CrossRef]
34. [32] He M, Li B, Chen H. Multi-scale 3D deep convolu-tional neural network for hyperspectral image clas-sification. 2017 IEEE International Conference on Image Processing (ICIP); 2017 Sept 17-20; Beijing, China: IEEE; 2017. pp. 3904–3908. [CrossRef]
35. [33] Cihan M, Ceylan M, Soylu H, Konak M. Fast evalua-tion of unhealthy and healthy neonates using hyper-spectral features on 700-850 Nm wavelengths, ROI extraction, and 3D-CNN. IRBM 2022;43:362–371. [CrossRef]
36. [34] Goyal P, Dollár P, Girshick R, Noordhuis P, Wesolowski L, Kyrola A, et al. Accurate, large mini-batch sgd: Training imagenet in 1 hour, 2017.