Classification of Knee X-rays That Can Be Diagnosed Radiographically Using Deep Learning and Machine Learning Methods

Abstract

The aim of this study is to classify knee osteoarthritis, synovial chondromatosis, Osgood-Schlatter disease, os fabella pathologies that can be diagnosed with plain knee X-rays, and normal knee radiographs with deep learning and machine learning methods. This study was performed on 540 knee osteoarthritis, 151 Osgood_Schlatter disease, 191 knee chondromatosis, 152 os fabella and 523 normal knee X-ray images. First, classification was performed with the VGG-16 network, which is a pre-trained deep learning model. Then, the features extracted with the VGG-16 convolution layer were classified with random forest, support vector machines, logistic regression and decision tree machine learning algorithms. With VGG-16 model, 95.3% accuracy, 95.1% sensitivity, 98.7% specificity, 96.8% precision, and 95.9% F1 score results were obtained. In classifying the features extracted from the VGG- 16 convolution layer with machine learning algorithms, 98.2% accuracy, 99.0% sensitivity, 98.9% specificity, 98.2% precision and 98.5% F1 score results were obtained with the logistic regression classifier. In this study, which was conducted to classify radiographically detectable knee pathologies, successful results were obtained with the VGG-16 network. The features extracted from the convolution layer of the VGG-16 model were reclassified with machine learning algorithms, logistic regression, support vector machines and random forest classifiers, and improvements in performance metrics were obtained compared to the VGG-16 model. With this proposed method, the performance of deep learning models can be further improved.

Keywords

• Knee osteoarthritis
• Knee chondromatosis
• Osgood-Schlatter disease
• Os fabella
• Deep learning
• Machine learning

Radyografik Olarak Tanı Konulabilen Diz Röntgenlerinin Derin Öğrenme ve Makine Öğrenmesi Yöntemleri ile Sınıflandırılması

Abstract

Bu çalışmanın amacı, düz diz röntgenleriyle tanısı konulabilen diz osteoartriti, sinovyal kondromatozis, Osgood-Schlatter hastalığı, os fabella patolojileri ve normal diz radyografilerini derin öğrenme ve makine öğrenmesi yöntemleriyle sınıflandırmaktır. Bu çalışma 540 diz osteoartriti, 151 Osgood_Schlatter hastalığı, 191 diz kondromatozisi, 152 os fabella ve 523 normal diz röntgen görüntüsü üzerinde gerçekleştirildi. Öncelikle önceden eğitilmiş derin öğrenme modeli olan VGG-16 ağı ile sınıflandırma yapıldı. Daha sonra VGG-16 evrişim katmanı ile çıkarılan özellikler, rastgele orman, destek vektör makineleri, lojistik regresyon ve karar ağacı makine öğrenmesi algoritmalarıyla sınıflandırıldı. VGG-16 modeli ile %95,3 doğruluk, %95,1 duyarlılık, %98.7 özgüllük, %96,8 kesinlik ve %95,9 F1 skoru sonuçları elde edildi. VGG-16 evrişim katmanından çıkarılan özelliklerin makine öğrenmesi algoritmaları ile sınıflandırılmasında lojistik regresyon sınıflandırıcısı ile %98,2 doğruluk, %99,0 duyarlılık, %98.9 özgüllük, %98,2 kesinlik ve %98,5 F1 skoru sonuçları elde edilmiştir. Radyografik olarak tanısı konulabilen diz patolojilerinin sınıflandırılması amacıyla yapılan bu çalışmada, VGG-16 ağı ile başarılı sonuçlar elde edilmiştir. VGG-16 modeli evrişim katmanı üzerinden çıkarılan özellikler makine öğrenmesi algoritmaları ile yeniden sınıflandırılmış, lojistik regresyon, destek vektör makineleri ve rastgele orman sınıflandırıcıları ile VGG-16 modeline kıyasla performans metriklerinde iyileşmeler elde edilmiştir. Önerilen bu yöntemle, derin öğrenme modellerinin performansı daha da iyileştirilebilir.

Keywords

• Diz osteoartriti
• Diz kondromatozu
• Osgood-Schlatter hastalığı
• Os fabella
• Derin öğrenme
• Makine öğrenmesi

References

1. [1] M. J. Lespasio, N. S. Piuzzi, M. E. Husni, G. F. Muschler, A. Guarino and M. A. Mont, “Knee osteoarthritis: a primer,” The Permanente Journal, vol. 21, no. 4, pp. 16–183, 2017.
2. [2] B. Heidari, “Knee osteoarthritis prevalence, risk factors, pathogenesis and features: Part I.,” Caspian Journal of Internal Medicine, vol. 2, no. 2, pp. 205–12, 2011.
3. [3] R. C. Lawrence et al., “Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part II.,” Arthritis & Rheumatism, vol. 58, no. 1, pp. 26–35, 2008.
4. [4] J. A. Neumann, G. E. Garrigues, B. E. Brigman and W. C. Eward, “Synovial chondromatosis,” JBJS Reviews, vol. 4, no. 5, 2016, Art. no. e2.
5. [5] M. A. Adelani, R. M. Wupperman and G. E. Holt, “Benign synovial disorders,” Journal of the American Academy of Orthopaedic Surgeons, vol. 16, no. 5, pp. 268–275, 2008.
6. [6] A. Robertson, S. C. E. Jones, R. Paes and G. Chakrabarty, “The fabella: a forgotten source of knee pain?,” Knee, vol. 11, no. 3, pp. 243–245, 2004.
7. [7] O. Unluturk, S. Duran and H. Yasar Teke, “Prevalence of the fabella and its general characteristics in Turkish population with magnetic resonance imaging,” Surgical and Radiologic Anatomy, vol. 43, no. 12, pp. 2047–2054, 2021.
8. [8] M. A. Berthaume, E. Di Federico and A. M. J. Bull, “Fabella prevalence rate increases over 150 years, and rates of other sesamoid bones remain constant: a systematic review,” Journal of Anatomy, vol. 235, no. 1, pp. 67–79, 2019.

9. [9] N. H. Sahar, A. S. Ramli, S. F. Badlishah-Sham and M. F. Mohd Miswan, “Osgood-Schlatter disease in adult: would early diagnosis and treatment ımprove the prognosis?,” Journal of Clinical and Health Sciences, vol. 8, no. 1, pp. 76–81, 2023.
10. [10] J. Høgh and B. Lund, “The sequelae of Osgood-Schlatter’s disease in adults,” International Orthopaedics, vol. 12, no. 3, pp. 213–215, 1988.
11. [11] G. L. De Lucena, C. Dos Santos Gomes and R. Oliveira Guerra, “Prevalence and associated factors of osgood-schlatter syndrome in a population-based sample of brazilian adolescents,” The American Journal of Sports Medicine, vol. 39, no. 2, pp. 415–420, 2011.
12. [12] M. Mazzei, "A change detection with machine learning approach for medical image analysis". in MICAD 2022. Lecture Notes in Electrical Engineering, vol. 810. R. Su, Y. Zhang, H. Liu and A. F. Frangi, Eds. Singapore: Springer, 2023, pp. 203-229.
13. [13] S. Suganyadevi, V. Seethalakshmi and K. Balasamy, “A review on deep learning in medical image analysis,” International Journal of Multimedia Information Retrieval, vol. 11, no. 1, pp. 19–38, 2022.
14. [14] P. K. Sethy, S. K. Behera, P. K. Ratha and P. Biswas, “Detection of coronavirus disease (COVID- 19) based on deep features and support vector machine,” International Journal of Mathematical, Engineering and Management Sciences, vol. 5, no. 4, pp. 643–651, 2020.
15. [15] S. S. Abdullah and M. P. Rajasekaran, “Automatic detection and classification of knee osteoarthritis using deep learning approach,” La Radiologia Medica, vol. 127, no. 4, pp. 398–406, 2022.
16. [16] K. O’Shea and R. Nash, “An introduction to convolutional neural networks,” International Journal of Research in Applied Sciences and Engineering Technology, vol. 10, no. 12, pp. 943– 947, 2022.
17. [17] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg and L. Fei-Fei, “ImageNet large scale visual recognition challenge,” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015.
18. [18] F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, “A comprehensive survey on transfer learning,” Proceedings of the IEEE, vol. 109, no. 1, pp. 43–76, 2021.
19. [19] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, vol. 25, 2012.
20. [20] C. Szegedy et al., "Going deeper with convolutions," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 1-9.
21. [21] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA, 2015, pp. 1–14, 2014.
22. [22] K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770–778.
23. [23] A. G. Howard et al., “MobileNets: efficient convolutional neural networks for mobile vision applications,” arXiv: Computer Vision and Pattern Recognition, 2017, [Online]. Available: http://arxiv.org/abs/1704.04861.
24. [24] A. W. Salehi et al., “A study of CNN and transfer learning in medical imaging: advantages, challenges, future scope,” Sustainability, vol. 15, no. 7, 2023, Art. no. 5930.
25. [25] P. M. Shakeel, M. A. Burhanuddin and M. I. Desa, “Automatic lung cancer detection from CT image using improved deep neural network and ensemble classifier,” Neural Computing and Applications, vol. 34, no. 12, pp. 9579–9592, 2022.
26. [26] P. Tiwari et al., “CNN based multiclass brain tumor detection using medical imaging,” Computational Intelligence and Neuroscience, vol. 2022, pp. 1–8, 2022.
27. [27] A. Tiulpin, J. Thevenot, E. Rahtu, P. Lehenkari and S. Saarakkala, “Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach,” Scientific Reports, vol. 8, no. 1, 2018, Art. no. 1727.
28. [28] Y. Wang, X. Wang, T. Gao, L. Du and W. Liu, “An automatic knee osteoarthritis diagnosis method based on deep learning: data from the osteoarthritis initiative,” Journal of Healthcare Engineering, vol. 2021, pp. 1–10, 2021.
29. [29] B. Liu, J. Luo and H. Huang, “Toward automatic quantification of knee osteoarthritis severity using improved Faster R-CNN,” International Journal of Computer Assisted Radiology and Surgery, vol. 15, no. 3, pp. 457–466, 2020.
30. [30] K. Üreten and H. H. Maraş, “Automated classification of rheumatoid arthritis, osteoarthritis, and normal hand radiographs with deep learning methods,” Journal of Digital Imaging, vol. 35, no. 2, pp. 193–199, 2022.
31. [31] S. Duran et al., “Automatic detection of spina bifida occulta with deep learning methods from plain pelvic radiographs,” Research on Biomedical Engineering, vol. 39, no. 3, pp. 655–661, 2023.
32. [32] K. Üreten, H. F. Sevinç, U. İğdeli, A. Onay and Y. Maraş, “Use of deep learning methods for hand fracture detection from plain hand radiographs.,” Turkish Journal of Trauma & Emergency Surgery, vol. 28, no. 2, pp. 196–201, 2022.
33. [33] S. Benyahia, B. Meftah and O. Lézoray, “Multi-features extraction based on deep learning for skin lesion classification,” Tissue Cell, vol. 74, 2022, Art. no. 101701.
34. [34] H. Nasiri and S. Hasani, “Automated detection of COVID-19 cases from chest X-ray images using deep neural network and XGBoost,” Radiography, vol. 28, no. 3, pp. 732–738, 2022.
35. [35] H. Nasiri and S. A. Alavi, “A novel framework based on deep learning and ANOVA feature selection method for diagnosis of COVID-19 cases from chest x-ray images,” Computational Intelligence and Neuroscience, vol. 2022, pp. 1–11, 2022.
36. [36] A. Mahbod, G. Schaefer, C. Wang, R. Ecker and I. Ellinge, "Skin lesion classification using hybrid deep neural networks," in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 2019, pp. 1229-1233.
37. [37] Y. Wang and X. S. Ni, “Predicting class-imbalanced business risk using resampling, regularization, and model emsembling algorithms,” International Journal of Management and Information Technology, vol. 11, no. 1, 2019.
38. [38] Y. Jiao, J. Yuan, Y. Qiang and S. Fei, “Deep embeddings and logistic regression for rapid active learning in histopathological images,” Computer Methods and Programs in Biomedicine, vol. 212, 2021, Art. no. 106464.