Otomatik Kemik Kırığı Tespitinde Evrişimsel Sinir Ağları ve Transformer Modellerinin Performansının Analizi

Abstract

İnsan varlığı için hayati önem taşıyan iskelet ve kas sisteminin en önemli bileşeni kemiklerdir. Bir kemiğin kırılması belirli bir darbeden veya şiddetli bir geriye doğru hareketten kaynaklanabilir. Bu çalışmada, kemik kırığı tespiti, evrişimli sinir ağı (ESA) tabanlı modeller olan Faster R-CNN ve RetinaNet, ayrıca bir transformer tabanlı model olan DETR (Detection Transformer) kullanılarak gerçekleştirilmiştir. Her model için farklı omurga ağları kullanılarak detaylı bir inceleme yapılmıştır. Bu çalışmanın birincil katkıları, CNN ve transformatör tasarımları arasındaki performans farklılıklarının yöntemsel bir değerlendirmesidir. 5145 görüntüden oluşan açık kaynaklı bir veri setinde eğitilen modeller, 750 test görüntüsünde test edilmiştir. Sonuçlara göre, RetinaNet/ResNet101 modeli diğer modellere göre daha üstün performans sergileyerek 0.901 mAP50 oranına ulaşmıştır. Elde edilen sonuçlar, eğitilen modellerin bilgisayar destekli tanı (BDT) sistemlerinde kullanılabilecek vaat edici sonuçlar sunmaktadır.

Keywords

• Kemik kırığı
• vision transformer
• DETR
• Faster R-CNN
• RetinaNet

Analyzing the Performance of Convolutional Neural Networks and Transformer Models in Automated Bone Fracture Detection

Abstract

The most significant component of the skeletal and muscular system, whose function is vital to human existence, are the bones. Breaking a bone might occur from a specific hit or from a violent rearward movement. In this study, bone fracture detection was performed using convolutional neural network (CNN) based models, Faster R-CNN and RetinaNet, as well as a transformer-based model, DETR (Detection Transformer). A detailed investigation was conducted using different backbone networks for each model. This study's primary contributions are a methodical assessment of the performance variations between CNN and transformer designs. Models trained on an open-source dataset consisting of 5145 images were tested on 750 test images. According to the results, the RetinaNet/ResNet101 model exhibited superior performance with a 0.901 mAP50 ratio compared to other models. The obtained results show promising outcomes that the trained models could be utilized in computer-aided diagnosis (CAD) systems.

Keywords

• Bone fracture
• vision transformer
• DETR
• Faster R-CNN
• RetinaNet

References

1. REFERENCES
2. [1] Czermak E.D., Euler A., Franckenberg S., Finkenstaedt T., Villefort C., Dominic G., Guggenberger R. Evaluation of ultrashort echo-time (UTE) and fast-field-echo (FRACTURE) sequences for skull bone visualization and fracture detection – A postmortem study, Journal of Neuroradiology. 49 237-243, 2022
3. [2] Karanam S.R., Srinivas Y., Chakravarty S. A systematic review on approach and analysis of bone fracture classification, Materials Today: Proceedings. 80 2557-2562, 2023
4. [3] Caron R., Londono I., Seoud L., Villemure I. Segmentation of trabecular bone microdamage in Xray microCT images using a two-step deep learning method, Journal of the Mechanical Behavior of Biomedical Materials. 137 105540, 2023.
5. [4] Ozdemir C., Dogan Y. Advancing brain tumor classification through MTAP model: an innovative approach in medical diagnostics, Medical and Biological Engineering and Computing. 1-12, 2024
6. [5] Ozdemir C. Classification of brain tumors from MR images using a new CNN architecture." Traitement du Signal. 40(2) 611-618, 2023.
7. [6] Guan B., Yao J., Wang S., Zhang G., Zhang Y., Wang X., Wang M. Automatic detection and localization of thighbone fractures in X-ray based on improved deep learning method, Computer Vision and Image Understanding. 216 103345, 2022.
8. [7] O'Shea K., Nash R. An introduction to convolutional neural networks, arXiv preprint arXiv:1511.08458, 2015.

9. [8] Ozdemir C., Dogan Y., Kaya Y. RGB-Angle-Wheel: A new data augmentation method for deep learning models. Knowledge-Based Systems. 291 111615, 2024
10. [9] Ren S., He K., Girshick R., Sun J. Faster r-cnn: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence. 39 1137-1149, 2017.
11. [10] Lin T.Y., Goyal P., Girshick R., He K., Dollár P. Focal loss for dense object detection, IEEE Transactions on Pattern Analysis and Machine Intelligence. 42 318-327, 2020.
12. [11] Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., Dehghani M., Minderer M., Heigold G., Gelly S., Uszkoreit J., Houlsby N. An image is worth 16x16 words: Transformers for image recognition at scale, International Conference on Learning Representations. 2021.
13. [12] Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser L., Polosukhin, I. Attention is all you need, Advances in neural information processing systems 30(NIPS 2017). 30, 2017.
14. [13] Carion N., Massa F., Synnaeve G., Usunier N., Kirillov A., Zagoruyko S. End-to-end object detection with transformers, European Conference on Computer Vision. 12346 213-229, 2020.
15. [14] Warin K., Limprasert W., Suebnukarn S., Inglam S., Jantana P., Vicharueang S. Assessment of deep convolutional neural network models for mandibular fracture detection in panoramic radiographs, International Journal of Oral and Maxillofacial Surgery. 51 1488-1494, 2022.
16. [15] Huang G., Liu Z., Maaten L.V.D., Weinberger K.Q. Densely Connected Convolutional Networks, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2261-2269, 2017.
17. [16] Kim D.Y., Park E., Ku K., Hwang S.J., Hwang K.T., Lee C.H., Yoon G.H. Application of stacked autoencoder for identification of bone fracture, Journal of the Mechanical Behavior of Biomedical Materials. 146 106077, 2023.
18. [17] Tao B., Yu X., Wang W., Wang H., Chen X., Wang F., Wu Y. A deep learning-based automatic segmentation of zygomatic bones from cone-beam computed tomography images, Journal of Dentistry. 135 104582, 2023.
19. [18] Ahmed K.D., Hawezi R. Detection of bone fracture based on machine learning techniques, Measurement: Sensors. 27 100723, 2023.
20. [19] Du H., Wang H., Yang C., Kabalata L., Li H., Qiang C. Hand bone extraction and segmentation based on a convolutional neural network, Biomedical Signal Processing and Control. 89 105788, 2024.
21. [20] Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. 9351 234-241, 2015.
22. [21] Bochkovskiy A., Wang C.Y., Liao H.Y.M. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020.
23. [22] Zheng B., Wang H., Xu J., Tu P., Joskowicz L., Chen X. Two-Stage Structure-Focused Contrastive Learning for Automatic Identification and Localization of Complex Pelvic Fractures, IEEE Transactions on Medical Imaging. 42 2751-2762, 2023.
24. [23] Roboflow 100. Bone fracture dataset, Roboflow Universe. 2023.
25. [24] Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
26. [25] He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition, Proceedings of the IEEE conference on computer vision and pattern recognition. 770-778.
27. [26] Han S., Xiao X., Song B., Guan T., Zhang Y., Lyu M. Automatic borehole fracture detection and characterization with tailored Faster R-CNN and simplified Hough transform, Engineering Applications of Artificial Intelligence. 126 107024, 2023.
28. [27] Lyu H., Qiu F., An L., Stow D., Lewison R., Bohnett E. Deer survey from drone thermal imagery using enhanced faster R-CNN based on ResNets and FPN, Ecological Informatics. 79 102383, 2024.
29. [28] Tang Y., Chen Y., Sharifuzzaman S.A.S.M., Li T. An automatic fine-grained violence detection system for animation based on modified faster R-CNN, Expert Systems with Applications. 237 121691, 2024.
30. [29] Girshick R. Fast r-cnn, Proceedings of the IEEE international conference on computer vision. 1440-1448, 2015.
31. [30] Yang W., Xiao Y., Shen H., Wang Z. Generalized weld bead region of interest localization and improved faster R-CNN for weld defect recognition, Measurement. 222 113619, 2023.
32. [31] Cheng J., Wang R., Lin A., Jiang D., Wang Y. A feature enhanced RetinaNet-based for instance-level ship recognition, Engineering Applications of Artificial Intelligence. 126 107133, 2023.
33. [32] Lin T.Y., Dollár P., Girshick R., He K., Hariharan B., Belongie S. Feature pyramid networks for object detection, Proceedings of the IEEE conference on computer vision and pattern recognition. 2117-2125, 2017.
34. [33] Tong L., Xue S., Chen X., Fang R. Artificial intelligence-based detection of posterior tibial slope on X-ray images of unicompartmental knee arthroplasty patients, Journal of Radiation Research and Applied Sciences. 16 100615, 2023.
35. [34] Chen Y., Zhang C., Chen B., Huang Y., Sun Y., Wang C., Fu X., Dai Y., Qin F., Peng Y., Gao Y. Accurate leukocyte detection based on deformable-DETR and multi-level feature fusion for aiding diagnosis of blood diseases, Computers in Biology and Medicine. 170 107917, 2024.
36. [35] Zheng H., Wang G., Xiao D., Liu H., Hu X. FTA-DETR: An efficient and precise fire detection framework based on an end-to-end architecture applicable to embedded platforms, Expert Systems with Applications. 248 123394, 2024.
37. [36] Ma Y., Luo Y. Bone fracture detection through the two-stage system of Crack-Sensitive Convolutional Neural Network, Informatics in Medicine Unlocked. 22 100452, 2021.
38. [37] Guan B., Zhang G., Yao J., Wang X., Wang M. Arm fracture detection in X-rays based on improved deep convolutional neural network, Computers and Electrical Engineering. 81 106530, 2020.
39. [38] Guan B., Yao J., Zhang G., Wang X. Thigh fracture detection using deep learning method based on new dilated convolutional feature pyramid network, Pattern Recognition Letters. 125 521-526, 2019.
40. [39] Zou J., Arshad M.R. Detection of whole body bone fractures based on improved YOLOv7, Biomedical Signal Processing and Control. 91 105995, 2024.