Performance Comparison of Different Pre-Trained Deep Learning Models in Classifying Brain MRI Images

Abstract

A brain tumor is a collection of abnormal cells formed as a result of uncontrolled cell division. If tumors are not diagnosed in a timely and accurate manner, they can cause fatal consequences. One of the commonly used techniques to detect brain tumors is magnetic resonance imaging (MRI). MRI provides easy detection of abnormalities in the brain with its high resolution. MR images have traditionally been studied and interpreted by radiologists. However, with the development of technology, it becomes more difficult to interpret large amounts of data produced in reasonable periods. Therefore, the development of computerized semi-automatic or automatic methods has become an important research topic. Machine learning methods that can predict by learning from data are widely used in this field. However, the extraction of image features requires special engineering in the machine learning process. Deep learning, a sub-branch of machine learning, allows us to automatically discover the complex hierarchy in the data and eliminates the limitations of machine learning. Transfer learning is to transfer the knowledge of a pre-trained neural network to a similar model in case of limited training data or the goal of reducing the workload. In this study, the performance of the pre-trained Vgg-16, ResNet50, Inception v3 models in classifying 253 brain MR images were evaluated. The Vgg-16 model showed the highest success with 94.42% accuracy, 83.86% recall, 100% precision and 91.22% F1 score. This was followed by the ResNet50 model with an accuracy of 82.49%.The findings obtained in this study were compared with similar studies in the literature and it was found that it showed higher success than most studies.

Keywords

• Brain MRI Classification
• Transfer Learning
• Convolutional Neural Networks

Beyin MR Görüntülerini Sınıflandırmada Farklı Önceden Eğitilmiş Derin Öğrenme Modellerinin Performans Karşılaştırması

Abstract

Beyin tümörleri, beyin hücrelerinin kontrolsüz bölünmeleri sonucu meydana gelen kitlelerdir. Tümörler zamanında ve doğru teşhis edilmezlerse ölümcül sonuçlara neden olabilir. Beyin tümörlerini tespit etmede yaygın olarak kullanılan tekniklerden biri olan MRI’dir. MRI, sağladığı yüksek çözünürlük ile beyindeki anormalliklerin kolay tespitine imkân verir. MR görüntüleri geleneksel olarak radyologlar tarafından incelenip yorumlanır. Ancak teknolojinin gelişmesi ile birlikte üretilen çok miktarda veriyi makul sürelerde yorumlamak daha zor hale gelmektedir. Bu nedenle bilgisayarlı yarı otomatik ya da otomatik yöntemlerin geliştirilmesi önemli bir araştırma konusu haline gelmiştir. Verilerden öğrenerek tahmin yapabilen makine öğrenmesi yöntemleri bu alanda yaygın olarak kullanılmaktadır. Ancak makine öğrenmesi için görüntü özelliklerinin çıkarımı özel bir mühendislik gerektirir. Makine öğrenmesinin bir alt dalı olan derin öğrenme, veri içerisindeki karmaşık hiyerarşiyi otomatik olarak keşfetmeye imkân sağlar ve makine öğrenmesinin sınırlılıklarını ortadan kaldırır. Transfer öğrenme ise eldeki eğitim verisinin az olması halinde ya da iş yükünü azaltmak için daha önceden eğitilmiş bir derin sinir ağının bilgilerinin benzer başka bir modele aktarılmasıdır. Bu çalışmada önceden eğitilmiş Vgg-16, ResNet50 ve Inception v3 modellerinin sınıflamadaki performansları değerlendirilmiştir. Vgg-16 modeli %94.42 doğruluk, %83.86 recall, %100 precision ve %91.22 F1 skoru ile en yüksek başarıyı göstermiştir. Bunu %82.49 doğrulukla ResNet50 modeli izlemektedir. Bu çalışmada elde edilen bulgular literatürdeki benzer çalışmalarla karşılaştırılmış ve çoğu çalışmadan daha yüksek başarı gösterdiği görülmüştür. 

Keywords

• Beyin MRI Sınıflama
• Transfer Öğrenme
• Evrişimsel Sinir Ağları

References

1. Abiwinanda, Nyoman, Muhammad Hanif, S. Tafwida Hesaputra, Astri Handayani, and Tati Rajab Mengko. 2019. “Brain Tumor Classification Using Convolutional Neural Network.” Pp. 183–89 in World congress on medical physics and biomedical engineering 2018. Springer.
2. Afshar, Parnian, Arash Mohammadi, and Konstantinos N. Plataniotis. 2018. “Brain Tumor Type Classification via Capsule Networks.” Pp. 3129–3133 in 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE.
3. Agn, Mikael, Per Munck af Rosenschöld, Oula Puonti, Michael J. Lundemann, Laura Mancini, Anastasia Papadaki, Steffi Thust, John Ashburner, Ian Law, and Koen Van Leemput. 2019. “A Modality-Adaptive Method for Segmenting Brain Tumors and Organs-at-Risk in Radiation Therapy Planning.” Medical Image Analysis 54:220–237.
4. Agravat, Rupal R., and Mehul S. Raval. 2018. “Deep Learning for Automated Brain Tumor Segmentation in MRI Images.” Pp. 183–201 in Soft Computing Based Medical Image Analysis, edited by N. Dey, A. S. Ashour, F. Shi, and V. E. Balas. Academic Press.
5. Akkus, Zeynettin, Alfiia Galimzianova, Assaf Hoogi, Daniel L. Rubin, and Bradley J. Erickson. 2017. “Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions.” Journal of Digital Imaging 30(4):449–59.
6. Bauer, Stefan, Roland Wiest, Lutz-P. Nolte, and Mauricio Reyes. 2013. “A Survey of MRI-Based Medical Image Analysis for Brain Tumor Studies.” Physics in Medicine & Biology 58(13):R97.
7. Belaid, Ouiza Nait, and Malik Loudini. 2020. “Classification of Brain Tumor by Combination of Pre-Trained VGG16 CNN.” Journal of Information Technology Management 12(2):13–25.
8. Cancer.Net, “Brain Tumor - Statistics.” Retrieved October 28, 2020 (https://www.cancer.net/cancer-types/brain-tumor/statistics).

9. Cao, Chensi, Feng Liu, Hai Tan, Deshou Song, Wenjie Shu, Weizhong Li, Yiming Zhou, Xiaochen Bo, and Zhi Xie. 2018. “Deep Learning and Its Applications in Biomedicine.” Genomics, Proteomics & Bioinformatics 16(1):17–32.
10. Chakrabarty, N. n.d. “Brain MRI Images for Brain Tumor Detection.” Kaggle. Retrieved May 28, 2020 (https://kaggle.com/navoneel/brain-mri-images-for-brain-tumor-detection).
11. Chaplot, Sandeep, Lalit M. Patnaik, and N. R. Jagannathan. 2006. “Classification of Magnetic Resonance Brain Images Using Wavelets as Input to Support Vector Machine and Neural Network.” Biomedical Signal Processing and Control 1(1):86–92.
12. Charron, Odelin, Alex Lallement, Delphine Jarnet, Vincent Noblet, Jean-Baptiste Clavier, and Philippe Meyer. 2018. “Automatic Detection and Segmentation of Brain Metastases on Multimodal MR Images with a Deep Convolutional Neural Network.” Computers in Biology and Medicine 95:43–54.
13. Chelghoum, Rayene, Ameur Ikhlef, Amina Hameurlaine, and Sabir Jacquir. 2020. “Transfer Learning Using Convolutional Neural Network Architectures for Brain Tumor Classification from MRI Images.” Pp. 189–200 in Artificial Intelligence Applications and Innovations, edited by I. Maglogiannis, L. Iliadis, and E. Pimenidis. Cham: Springer International Publishing.
14. Cheng, Jie-Zhi, Dong Ni, Yi-Hong Chou, Jing Qin, Chui-Mei Tiu, Yeun-Chung Chang, Chiun-Sheng Huang, Dinggang Shen, and Chung-Ming Chen. 2016. “Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans.” Scientific Reports 6(1):1–13.
15. Deepak, S., and P. M. Ameer. 2019. “Brain Tumor Classification Using Deep CNN Features via Transfer Learning.” Computers in Biology and Medicine 111:103345. doi: 10.1016/j.compbiomed.2019.103345.
16. Deepak, S., and P. M. Ameer. 2020. “Automated Categorization of Brain Tumor from MRI Using CNN Features and SVM.” Journal of Ambient Intelligence and Humanized Computing. doi: 10.1007/s12652-020-02568-w.
17. Deniz, Erkan, Abdulkadir Şengür, Zehra Kadiroğlu, Yanhui Guo, Varun Bajaj, and Ümit Budak. 2018. “Transfer Learning Based Histopathologic Image Classification for Breast Cancer Detection.” Health Information Science and Systems 6(1):18.
18. Domingues, Rémi, Maurizio Filippone, Pietro Michiardi, and Jihane Zouaoui. 2018. “A Comparative Evaluation of Outlier Detection Algorithms: Experiments and Analyses.” Pattern Recognition 74:406–21.
19. El-Dahshan, El-Sayed Ahmed, Tamer Hosny, and Abdel-Badeeh M. Salem. 2010. “Hybrid Intelligent Techniques for MRI Brain Images Classification.” Digital Signal Processing 20(2):433–41.
20. Gu, Yu, Xiaoqi Lu, Lidong Yang, Baohua Zhang, Dahua Yu, Ying Zhao, Lixin Gao, Liang Wu, and Tao Zhou. 2018. “Automatic Lung Nodule Detection Using a 3D Deep Convolutional Neural Network Combined with a Multi-Scale Prediction Strategy in Chest CTs.” Computers in Biology and Medicine 103:220–231.
21. Havaei, Mohammad, Axel Davy, David Warde-Farley, Antoine Biard, Aaron Courville, Yoshua Bengio, Chris Pal, Pierre-Marc Jodoin, and Hugo Larochelle. 2017. “Brain Tumor Segmentation with Deep Neural Networks.” Medical Image Analysis 35:18–31.
22. He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep Residual Learning for Image Recognition.” Pp. 770–78 in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
23. Hussein, Sarfaraz, Pujan Kandel, Candice W. Bolan, Michael B. Wallace, and Ulas Bagci. 2019. “Lung and Pancreatic Tumor Characterization in the Deep Learning Era: Novel Supervised and Unsupervised Learning Approaches.” IEEE Transactions on Medical Imaging 38(8):1777–1787.
24. Jain, Rachna, Nikita Jain, Akshay Aggarwal, and D. Jude Hemanth. 2019. “Convolutional Neural Network Based Alzheimer’s Disease Classification from Magnetic Resonance Brain Images.” Cognitive Systems Research 57:147–59. doi: 10.1016/j.cogsys.2018.12.015.
25. Kamnitsas, Konstantinos, Christian Ledig, Virginia FJ Newcombe, Joanna P. Simpson, Andrew D. Kane, David K. Menon, Daniel Rueckert, and Ben Glocker. 2017. “Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation.” Medical Image Analysis 36:61–78.
26. Kaur, Taranjit, and Tapan Kumar Gandhi. 2020. “Deep Convolutional Neural Networks with Transfer Learning for Automated Brain Image Classification.” Machine Vision and Applications 31:1–16.
27. Kleesiek, Jens, Armin Biller, Gregor Urban, U. Kothe, Martin Bendszus, and F. Hamprecht. 2014. “Ilastik for Multi-Modal Brain Tumor Segmentation.” Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge) 12–17.
28. Kumar, Sanjeev, Chetna Dabas, and Sunila Godara. 2017. “Classification of Brain MRI Tumor Images: A Hybrid Approach.” Procedia Computer Science 122:510–17.
29. LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. “Deep Learning.” Nature 521(7553):436–444.
30. Litjens, Geert, Clara I. Sánchez, Nadya Timofeeva, Meyke Hermsen, Iris Nagtegaal, Iringo Kovacs, Christina Hulsbergen-Van De Kaa, Peter Bult, Bram Van Ginneken, and Jeroen Van Der Laak. 2016. “Deep Learning as a Tool for Increased Accuracy and Efficiency of Histopathological Diagnosis.” Scientific Reports 6:26286.
31. Menze, Bjoern H., Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, and Roland Wiest. 2014. “The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS).” IEEE Transactions on Medical Imaging 34(10):1993–2024.
32. Mlynarski, Pawel, Hervé Delingette, Antonio Criminisi, and Nicholas Ayache. 2019. “Deep Learning with Mixed Supervision for Brain Tumor Segmentation.” Journal of Medical Imaging 6(3):034002.
33. Mohsen, Heba, El-Sayed A. El-Dahshan, El-Sayed M. El-Horbaty, and Abdel-Badeeh M. Salem. 2018. “Classification Using Deep Learning Neural Networks for Brain Tumors.” Future Computing and Informatics Journal 3(1):68–71.
34. Moritz, Chad H., Victor M. Haughton, Dietmar Cordes, Michelle Quigley, and M. Elizabeth Meyerand. 2000. “Whole-Brain Functional MR Imaging Activation from a Finger-Tapping Task Examined with Independent Component Analysis.” American Journal of Neuroradiology 21(9):1629–35.
35. Naser, Mohamed A., and M. Jamal Deen. 2020. “Brain Tumor Segmentation and Grading of Lower-Grade Glioma Using Deep Learning in MRI Images.” Computers in Biology and Medicine 121:103758.
36. Nayak, Deepak Ranjan, Ratnakar Dash, and Banshidhar Majhi. 2016. “Brain MR Image Classification Using Two-Dimensional Discrete Wavelet Transform and AdaBoost with Random Forests.” Neurocomputing 177:188–97.
37. Pashaei, Ali, Hedieh Sajedi, and Niloofar Jazayeri. 2018. “Brain Tumor Classification via Convolutional Neural Network and Extreme Learning Machines.” Pp. 314–319 in 2018 8th International conference on computer and knowledge engineering (ICCKE). IEEE.
38. Prastawa, Marcel, Elizabeth Bullitt, Sean Ho, and Guido Gerig. 2004. “A Brain Tumor Segmentation Framework Based on Outlier Detection.” Medical Image Analysis 8(3):275–83.
39. Rajinikanth, V., Suresh Chandra Satapathy, Steven Lawrence Fernandes, and S. Nachiappan. 2017. “Entropy Based Segmentation of Tumor from Brain MR Images–a Study with Teaching Learning Based Optimization.” Pattern Recognition Letters 94:87–95.
40. Ranjan Nayak, Deepak, Ratnakar Dash, and Banshidhar Majhi. 2017. “Stationary Wavelet Transform and Adaboost with SVM Based Pathological Brain Detection in MRI Scanning.” CNS & Neurological Disorders-Drug Targets (Formerly Current Drug Targets-CNS & Neurological Disorders) 16(2):137–49.
41. Rehman, Arshia, Saeeda Naz, Muhammad Imran Razzak, Faiza Akram, and Muhammad Imran. 2020. “A Deep Learning-Based Framework for Automatic Brain Tumors Classification Using Transfer Learning.” Circuits, Systems, and Signal Processing 39(2):757–75. doi: 10.1007/s00034-019-01246-3.
42. Reza, S., and K. M. Iftekharuddin. 2014. “Improved Brain Tumor Tissue Segmentation Using Texture Features.” Proceedings MICCAI BraTS (Brain Tumor Segmentation Challenge) 27–30.
43. Saritha, M., K. Paul Joseph, and Abraham T. Mathew. 2013. “Classification of MRI Brain Images Using Combined Wavelet Entropy Based Spider Web Plots and Probabilistic Neural Network.” Pattern Recognition Letters 34(16):2151–2156.
44. Saxena, Priyansh, Akshat Maheshwari, and Saumil Maheshwari. 2020. “Predictive Modeling of Brain Tumor: A Deep Learning Approach.” Pp. 275–85 in Innovations in Computational Intelligence and Computer Vision. Springer.
45. Shahamat, Hossein, and Mohammad Saniee Abadeh. 2020. “Brain MRI Analysis Using a Deep Learning Based Evolutionary Approach.” Neural Networks 126:218–34.
46. Simonyan, K., and A. Zisserman. 2020. “Very Deep Convolutional Networks for Large-Scale Image Recognition. ArXiv 1409.1556 (09 2014).” URL Https://Arxiv. Org/Abs/1409.1556. Accessed: February.
47. Szegedy, Christian, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. “Rethinking the Inception Architecture for Computer Vision.” Pp. 2818–2826 in Proceedings of the IEEE conference on computer vision and pattern recognition.
48. Tustison, Nick, Max Wintermark, Chris Durst, and Brian Avants. 2013. “Ants and Arboles.” Multimodal Brain Tumor Segmentation 47.
49. Wang, Shuihua, Yudong Zhang, Zhengchao Dong, Sidan Du, Genlin Ji, Jie Yan, Jiquan Yang, Qiong Wang, Chunmei Feng, and Preetha Phillips. 2015. “Feed-Forward Neural Network Optimized by Hybridization of PSO and ABC for Abnormal Brain Detection.” International Journal of Imaging Systems and Technology 25(2):153–164.
50. Yang, Yang, Lin-Feng Yan, Xin Zhang, Yu Han, Hai-Yan Nan, Yu-Chuan Hu, Bo Hu, Song-Lin Yan, Jin Zhang, Dong-Liang Cheng, Xiang-Wei Ge, Guang-Bin Cui, Di Zhao, and Wen Wang. 2018. “Glioma Grading on Conventional MR Images: A Deep Learning Study With Transfer Learning.” Frontiers in Neuroscience 12:804–804. doi: 10.3389/fnins.2018.00804.
51. Yousefi, Mina, Adam Krzyżak, and Ching Y. Suen. 2018. “Mass Detection in Digital Breast Tomosynthesis Data Using Convolutional Neural Networks and Multiple Instance Learning.” Computers in Biology and Medicine 96:283–293.
52. Zhang, Min, Geoffrey S. Young, Huai Chen, Jing Li, Lei Qin, J. Ricardo McFaline-Figueroa, David A. Reardon, Xinhua Cao, Xian Wu, and Xiaoyin Xu. 2020. “Deep-Learning Detection of Cancer Metastases to the Brain on MRI.” Journal of Magnetic Resonance Imaging 52(4):1227–36.
53. Zhang, Yu-Dong, Shuihua Wang, Zhengchao Dong, Preetha Phillip, Genlin Ji, and Jiquan Yang. 2015. “Pathological Brain Detection in Magnetic Resonance Imaging Scanning by Wavelet Entropy and Hybridization of Biogeography-Based Optimization and Particle Swarm Optimization.” Progress In Electromagnetics Research 152:41–58.
54. Zhang, Yudong, Shuihua Wang, Genlin Ji, and Zhengchao Dong. 2013. “An MR Brain Images Classifier System via Particle Swarm Optimization and Kernel Support Vector Machine.” The Scientific World Journal 2013.
55. Zhou, Leilei, Zuoheng Zhang, Yu-Chen Chen, Zhen-Yu Zhao, Xin-Dao Yin, and Hong-Bing Jiang. 2019. “A Deep Learning-Based Radiomics Model for Differentiating Benign and Malignant Renal Tumors.” Translational Oncology 12(2):292–300.
56. Zuo, Haiqiang, Heng Fan, Erik Blasch, and Haibin Ling. 2017. “Combining Convolutional and Recurrent Neural Networks for Human Skin Detection.” IEEE Signal Processing Letters 24(3):289–293.