Performance comparison of visual transformer based models for shoulder implant classification

Öz

Total shoulder arthroplasty (TSA) is a surgical procedure addressing severe pain and restricted shoulder joint movement. During TSA surgery, X-ray images guide the selection of the prosthetic implant suitable for the patient from a variety of models produced by different manufacturers. However, prostheses may wear or loosen over time, thus requiring periodic evaluation and replacement. Currently, the process involves taking new X-ray images from patients, resulting in variability in expert opinions on implant types. Therefore, there is a need for highly accurate automated diagnostic systems to help recognize unknown implants. In this study, we present a performance comparison of vision transformer (ViT) based models for automatic shoulder implant classification from X-ray images. Fine-tuning of pre-trained ViT models on a publicly available shoulder X-ray dataset showed high success in terms of accuracy, precision, sensitivity, and F-measure metrics. The Swin-B model yielded the highest results with 93.84\% accuracy, 88.15\% precision, and 85.52\% recall. These results showed that ViT based models can help improve treatment planning by providing reliable identification of shoulder implant manufacturers and model information and time efficiency, especially for specialists.

Anahtar Kelimeler

• Total shoulder arthroplasty
• Shoulder implants
• X-ray
• Classification
• Vision transformer

Omuz implantı sınıflandırmasında görü dönüştürücü tabanlı modellerin performans karşılaştırması

Öz

Total omuz artroplastisi (TSA), şiddetli ağrı ve kısıtlı omuz eklemi hareketini ele alan cerrahi bir prosedürdür. TSA ameliyatı sırasında röntgen görüntüleri, farklı üreticiler tarafından üretilen çeşitli modeller arasından hastaya uygun protez implantın seçimine rehberlik etmektedir. Bununla birlikte, protezler zamanla aşınabilir veya gevşeyebilir, bu nedenle periyodik değerlendirme ve değiştirme gerektirmektedir. Halihazırda bu süreç, hastalardan yeni röntgen görüntülerinin alınmasını gerektirmekte ve implant tiplerine ilişkin uzman görüşlerinde değişkenliğe neden olmaktadır. Bu nedenle, bilinmeyen implantları tanımaya yardımcı olacak yüksek doğrulukta otomatik teşhis sistemlerine ihtiyaç vardır. Bu çalışmada, X-ray görüntülerinden otomatik omuz implantı sınıflandırması için görü dönüştürücü (ViT) tabanlı modellerin performans karşılaştırması sunulmaktadır. Önceden eğitilmiş ViT modellerinin herkese açık bir veri kümesi üzerinde ince ayarı doğruluk, hassasiyet, duyarlılık ve F-ölçümü metriklerinde yüksek başarı göstermiştir. Swin-B modeli %93.84 doğruluk, %88.15 kesinlik ve %85.52 duyarlılık ile en yüksek sonuçları vermiştir. Bu sonuçlar, ViT tabanlı modellerin omuz implantı üreticilerinin ve model bilgilerinin güvenilir bir şekilde tanımlanmasını ve özellikle uzmanlar için zaman verimliliği sağlayarak tedavi planlamasının iyileştirilmesine yardımcı olabileceğini göstermiştir.

Anahtar Kelimeler

• Total omuz artroplastisi
• Omuz implantları
• X-ray
• Sınıflandırma
• Görü dönüştürücü

Kaynakça

1. R. H. Cofield, Total shoulder arthroplasty with the Neer prosthesis. JBJS, 66(6), 899-906, 1984. https://doi.org/10.2106/00004623-198466060-00010
2. J. Sanchez-Sotelo, Total shoulder arthroplasty. The open orthopaedics journal, 5, 106, 2011. https://doi.org/10.2174/1874325001105010106
3. C. Sukjamsri, The effect of implant misalignment on shoulder replacement outcomes (Doctoral dissertation, Imperial College London), 2015. https://doi.org/10.25560/28581
4. E. Sivari, M. S. Güzel, E. Bostanci and A. Mishra, A Novel Hybrid Machine Learning Based System to Classify Shoulder Implant Manufacturers. In Healthcare (Vol. 10, No. 3, p. 580), MDPI, 2022. https://doi.org/10.3390/healthcare10030580
5. D. P. Sahoo, M. Rout, P. K. Mallick and S. R. Samanta, Comparative Analysis of Medical Images using Transfer Learning Based Deep Learning Models. In 2022 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC) (pp. 1-8), IEEE, 2022. https://doi.org/10.1109/ASSIC55218.2022.10088373
6. B. Sistaninejhad, H. Rasi and P. Nayeri, A Review Paper about Deep Learning for Medical Image Analysis. Computational and Mathematical Methods in Medicine, 2023. https://doi.org/10.1155/2023/7091301
7. G. Urban, S. Porhemmat, M. Stark, B. Feeley, K. Okada and P. Baldi, Classifying shoulder implants in X-ray images using deep learning. Computational and structural biotechnology journal, 18, 967-972, 2020. https://doi.org/10.1016/j.csbj.2020.04.005
8. P. H. Yi, T. K. Kim, J. Wei, X. Li, G. D. Hager, H. I. Sair and J. Fritz, Automated detection and classification of shoulder arthroplasty models using deep learning. Skeletal radiology, 49, 1623-1632, 2020. https://doi.org/10.1007/s00256-020-03463-3

9. A. Yılmaz, Shoulder implant manufacturer detection by using deep learning: Proposed channel selection layer. Coatings, 11(3), 346, 2021. https://doi.org/10.3390/coatings11030346
10. H. Sultan, M. Owais, C. Park, T. Mahmood, A. Haider and K.R. Park, Artificial intelligence-based recognition of different types of shoulder implants in X-ray scans based on dense residual ensemble-network for personalized medicine. J. Pers. Med, 11, 482, 2021. https://doi.org/10.3390/jpm11060482
11. E. Efeoğlu and T. U. N. A. Gürkan, Radyografi Görüntüleri Ve Sınıflandırma Algoritmaları Kullanılarak Omuz Protezlerinin Üreticilerinin Belirlenmesi. Kırklareli Üniversitesi Mühendislik ve Fen Bilimleri Dergisi, 7(1), 57-73, 2021. https://doi.org/10.34186/klujes.906660
12. A. Karaci, Detection and classification of shoulder implants from X-ray images: YOLO and pretrained convolution neural network based approach. J. Fac. Eng. Archit. Gazi Univ, 37, 283-294, 2022. https://doi.org/10.17341/gazimmfd.888202
13. F. Shamshad, S. Khan, S. W. Zamir, M. H. Khan, M. Hayat, F. S. Khan and H. Fu, Transformers in medical imaging: A survey. Medical Image Analysis, 102802, 2023. https://doi.org/10.1016/j.media.2023.102802
14. A. Vaswani, P. Ramachandran, A. Srinivas, N. Parmar, B. Hechtman and J. Shlens, Scaling local self-attention for parameter efficient visual backbones. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12894-12904), 2021. https://doi.org/10.48550/arXiv .2103.12731
15. J. Devlin, M. W. Chang, K. Lee and K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805, 2018. https://doi.org/10.48550/ arXiv.1810.04805
16. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner and N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. https://doi.org/10.48550/ arXiv.2010.11929
17. H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles and H. Jégou, Training data-efficient image transformers & distillation through attention. In International conference on machine learning (pp. 10347-10357), PMLR, 2021.
18. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang and B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 10012-10022), 2021.
19. S. Tummala, J. Kim and S. Kadry, BreaST-Net: Multi-class classification of breast cancer from histopathological images using ensemble of swin transformers. Mathematics, 10(21), 4109, 2022. https://doi.org/10.3390/math10214109
20. S. Ayas, Multiclass skin lesion classification in dermoscopic images using swin transformer model. Neural Computing and Applications, 35(9), 6713-6722, 2023. https://doi.org/10.1007/s00521-022-08053-z
21. A. Alotaibi, T. Alafifi, F. Alkhilaiwi, Y. Alatawi, H. Althobaiti, A. Alrefaei and T. Nguyen, ViT-DeiT: An Ensemble Model for Breast Cancer Histopathological Images Classification. In 2023 1st International Conference on Advanced Innovations in Smart Cities (ICAISC) (pp. 1-6). IEEE, 2023. https://doi.org/10.1109/ICAISC56366.2023.10085467
22. S. Regmi, A. Subedi, U. Bagci and D. Jha, Vision Transformer for Efficient Chest X-ray and Gastrointestinal Image Classification. arXiv preprint arXiv:2304.11529, 2023.
23. D. M. Powers, Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061, 2020. https://doi.org/10.48550/arXiv.2010.16061
24. K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778), 2016. https://doi.org/ 10.48550/arXiv.1512.03385
25. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818-2826), 2016. https://doi.org/10.48550/ arXiv.1512.00567
26. A. Howard, M. Sandler, G. Chu, L. C. Chen, B. Chen, M. Tan and H. Adam, Searching for mobilenetv3. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 1314-1324), 2019. https://doi.org/10.48550/arXiv.1905.02244