Capsule Networks Based Automatic Measurement System for Scoliosis

Abstract

Scoliosis is a disease that deforms the general spine structureas a result of curvature of the spine. Although there are various methods in the diagnosis and treatment of the Scoliosis, the main purpose is to reduce the curvature angle, which is calledas Cobbangle, within the framework of reducing the level of Scoliosis. Cobbangle measurement is essentially performed manually on the spine x-rays by the expert. However, automating this process with an Artificial Intelligence approach such as deeplearning will provide great convenience and precision for both patient and the expert. Based on the explanations, the current status of the literature in terms of Scoliosis and deeplearning-focused studies was firstly discussed in this study, and then the Cobbangle measurements were automated with a Capsule Network (CapsNet)-based solution. As a result of the comparison of the CapsNet solution with the ConvNet, BoostNet, RFR and ResNet-50 models, it was determined that the CapsNet model gave the best findings

Keywords

• Scoliosis
• Spinal curvature
• Cobb angle
• Capsule networks
• CapsNet
• Deep learning

Skolyoz için Kapsül Ağları Tabanlı Otomatik Ölçüm Sistemi

Abstract

Skolyoz, omurganın eğrilmesi ile birlikte omurga genel yapısını deforme eden bir hastalıktır. Skolyoz tanı ve tedavi aşamasında çeşitli yöntemler olmakla birlikte, temel amaç Cobb açısı adı verilen eğrilik açısını azaltarak Skolyoz seviyesini düşürme çerçevesinde şekillenmektedir. Cobb açısı ölçümü esasında uzman tarafından, omurga röntgen filmleri üzerinde manuel olarak gerçekleştirilmektedir. Ancak bu sürecin derin öğrenme gibi bir Yapay Zeka yaklaşımıyla otomatikleştirilmesi hem hasta hem de uzman açısından büyük kolaylık ve kesinlik sağlayacaktır. Açıklamalardan hareketle bu çalışmada, öncelikli olarak Skolyoz ve derin öğrenme odaklı çalışmalar açısından literatürün güncel durumu ele alınmış, ardından Kapsül Ağları (CapsNet) tabanlı bir çözüm ile Cobb açısı ölçümlerinin otomatik bir hale getirilmesi sağlanmıştır. CapsNet çözümünün, ConvNet, BoostNet, RFR ve ResNet-50 modelleri ile karşılaştırılması neticesinde en iyi bulguları CapsNet modelinin verdiği tespit edilmiştir.

Keywords

• Skolyoz
• Omurga eğriliği
• Cobb açısı
• Kapsül ağları
• CapsNet
• Derin öğrenme

References

1. [1] J. Jin, “Screening for scoliosis in adolescents,” JAMA: The Journal of the American Medical Association, vol. 319, no. 2, pp. 202–202, 2018.
2. [2] D. Sevimli, M. Sanrı and M. Altuğ, “The effect of corrective exercises on the treatment of a scoliosis patient: Acase report,” International Congress on Education (ERPA-2016), 2016, pp. 40–42.
3. [3] M. Aebi, “The adult scoliosis, ”European Spine Journal, vol. 14, no. 10, pp. 925-948, 2005.
4. [4] M. T. Hresko, “Idiopathic scoliosis in adolescents,” New England Journal of Medicine, vol. 368, no. 9, pp. 834-841, 2013.
5. [5] Y. U. İbişoğlu ve F. A. Çaliş, “İzmir ili Bornova ilçesi ilköğretim kurumlarında okuyan 12-14 yaş grubu cocuklarda skolyoz prevalansı,” Türkiye Fiziksel Tıp ve Rehabilitasyon Dergisi, c. 58, s. 2, ss. 109–113, 2012.
6. [6] C. Varol, “Skolyozlu olgularda egzersizin solunum fonksiyonlarına ve yaşam kalitesine etkisi,” Yüksek lisans tezi, Kardiyopulmoner Fizyoterapi ve Rehabilitasyon, Marmara Üniversitesi, İstanbul, Türkiye, 2019.
7. [7] A. C. Kittleson and L. W. Lim, “Measurement of scoliosis,” American Journal of Roentgenology, vol. 108, no. 4, pp. 775–777, 1970.
8. [8] H. G. Yılmaz, “İdiyopatik skolyozda egzersiz reçeteleme,” Türkiye Fiziksel Tıp ve Rehabilitasyon Dergisi, c. 60, s. özel sayı 2, ss. S31–S35, 2014.

9. [9] M. Bakator and D. Radosav, “Deeplearning and medical diagnosis: A review of literature,” Multimodal Technologies and Interaction, vol. 2, no. 3, pp. 47, 2018.
10. [10] O. Deperlioglu, “Classification of segmented phonocardiograms by convolutional neural networks,” BRAIN: Broad Research in Artificial Intelligence and Neuroscience, vol. 10, no. 2, pp. 5-13, 2019.
11. [11] D. Ravi, C. Wong, F. Deligianni, M. Berthelot, J. Andreu-Perez, B. Lo and G. Z. Yang, “Deeplearning for healthin formatics,” IEEE Journal of Biomedical and Health Informatics, vol. 21, no. 1, pp. 4-21, 2016.
12. [12] G. E. Guraksin, S. Barin, E. Ozgul and F. Kaya, “COVID-19 diagnosis using deeplearning,” Düzce University Journal of Science & Technology, vol. 9, no. 3, pp. 8-23, 2021.
13. [13] J. Hemanth, U. Kose, O. Deperlioglu and V. H. C. de Albuquerque, “An augmented reality-supported mobile application for diagnosis of heart diseases,” The Journal of Supercomputing, vol. 76, no. 2, pp. 1242-1267, 2020.
14. [14] A. B. Levine, C. Schlosser, J. Grewal, R. Coope, S. J. Jones and S. Yip, “Rise of the machines: Advances in deeplearning for cancer diagnosis,” Trends in Cancer, vol. 5, no. 3, pp. 157-169, 2019.
15. [15] A. Helwan, G. El-Fakhri, H. Sasani and D. Uzun Ozsahin, “Deep networks in identifying CT brain hemorrhage,” Journal of Intelligent & Fuzzy Systems, vol. 35, no. 2, pp. 2215-2228, 2018.
16. [16] J. Hemanth, O. Deperlioglu and U. Kose, “An enhanced diabetic retinopathy detection and classification approach using deep convolutional neural network,” Neural Computing and Applications, vol. 32, no. 3, pp. 707-721, 2020.
17. [17] U. M. Prakash, K. Kottursamy, K. Cengiz, U. Kose and B. T. Hung, “4x-Expert systems for early predication of osteoporosis using multi-model algorithms,” Measurement, vol. 180, no. 8, pp. 109543, 2021.
18. [18] J. Alzubi, B. Bharathikannan, S. Tanwar, R. Manikandan, A. Khanna and C. Thaventhiran, “Boosted neural network ensemble classification for lung cancer disease diagnosis,” Applied Soft Computing, vol. 80, no. 7, pp. 579-591, 2019.
19. [19] M. I. Razzak, S. Naz and A. Zaib, “Deeplearning form edical image processing: Overview, challenges and the future,” in Classification in BioApps, 1st ed., vol. 26, Cham, Germany: Springer, 2018, pp. 323–350.
20. [20] S. Zhou, H. G. Kevin and S. Dinggang, Deep Learning for Medical Image Analysis, 1st ed., vol. 81, London, UK: Elsevier Academic Press, 2017, pp. 11–24.
21. [21] A. Esteva, K. Chou, S. Yeung, N. Naik, A. Madani, A. Mottaghi, Y. Liu, E. Topol, J. Dean andR. Socher, “Deeplearning-enabled medical computervision,” Digital Medicine, vol. 4, no. 1, pp. 1–9, 2021.
22. [22] I. Castiglioni, L. Rundo, M. Codari, G. DiLeo, C. Salvatore, M. Interlenghi, F. Gallivanone, A. Cozzi, N. C. D’amico and F. Sardanelli, “AI applications to medical images: From machine learning to deep learning,” Physica Medica, vol. 83, no. 1, 9–24, 2021.
23. [23] E. Xi, S. Bing and Y. Jin, “Capsule network performance on complex data,” 2017, arXiv:1712.03480.
24. [24] P. Afshar, A. Mohammadi and K. N. Plataniotis, “Brain tumor type classification via capsule networks,” in 2018 25th IEEE International Conference on Image Processing (ICIP), 2018, pp. 3129–3133.
25. [25] Wikipedia. (2006, November 23). Omurga / Vertebral column. (Author: Arcadian) [Online]. Available: https://tr.wikipedia.org/wiki/Omurga
26. [26] Wikipedia. (2017, November 5). Thoracic vertebrae / A thoracic spine X-ray of a 57-year-old male. (Author: Who is johngalt). [Online]. Available: https://en.wikipedia.org/wiki/Thoracic_vertebrae
27. [27] S. Bilgiç ve Ö. Erşen, “Adolesanidiyopatik skolyoz konservatif tedavisi,” Türkiye Klinikleri Journal of Orthopaedics and Traumatology Special Topics, c. 10, s. 2, ss. 118–123, 2017.
28. [28] A. A. Süzen, Z. Yıldız, K. Kayaalp, O. Ceylanve E. Arabacı, “Skolyoz hastalığının tespiti için taşınabilir cihaz tasarımı,” Avrupa Bilim ve Teknoloji Dergisi, c. 15, s. 1, ss. 483–490, 2019.
29. [29] P. Patias, E. Stylianidis, M. Pateraki, Y. Chrysanthou, C. Contozis and T. Zavitsanakis, “3D digital photogrammetric reconstructions for scoliosis screening,” in ISPRS Commission Symposium on Image Engineering and Vision Metrology, 2006, pp. 1214–1218.
30. [30] Molinaspine. (2020, December 26). Adolescent Scoliosis. Domingo Molina IV, M.D. Spine Surgeon [Online]. Available: https://molinaspine.com/spine-deformity
31. [31] M. A. Asher and D. Burton, “Adolescent idiopathic scoliosis: natural history and long term treatment effects,” Scoliosis, vol. 1, no. 2, pp. 1–10, 2006.
32. [32] Ö. Aydıngöz, “Adolesand omurga sorunları,” İstanbul Üniversitesi Cerrahpaşa Tıp Fakültesi Sürekli Tıp Eğitimi Etkinlikleri Adolesan Sağlığı Sempozyumu, İstanbul, Türkiye, 2005, ss. 125–133.
33. [33] S. Langensiepen, O. Semler, R. Sobottke, O. Fricke, J. Franklin, E. Schönau and P. Eysel, “Measuring procedures to determine the cobbangle in idiopathic scoliosis: a systematic review,” European Spine Journal, vol. 22, no. 11, pp. 2360–2371, 2013.
34. [34] R. T. Morrissy, G. S. Goldsmith, E. C. Hall, D. Kehl, and G. H. Cowie, “Measurement of the cobbangle on radiographs of patients who have scoliosis. Evaluation of intrinsicerror,” The Journal of Bone and Joint Surgery, vol. 72, no. 3, 320–327, 1990.
35. [35] T. A. Sardjono, M. H. Wilkinson, A. G. Veldhuizen, P. M. vanOoijen, K. E. Purnama and G. J. Verkerke, “Automatic cobbangle determination from radiographic images,” Spine, vol. 38, no. 20, pp. E1256–E1262, 2013.
36. [36] M. H. Horng, C. P. Kuok, M. J. Fu, C. J. Lin and Y. N. Sun, “Cobbangle measurement of spine fromx-ray images using convolutional neural network,” Computational and Mathematical Methods in Medicine, vol. 2019, no. 1, pp. 1–18, 2019.
37. [37] H. Kim, H. S. Kim, E. S. Moon, C. S. Yoon, T. S. Chung, H. T. Song, J. S. Suh, Y. H. Lee and S. Kim, “Scoliosis imaging: What radiologists should know—erratum,” Radiographics, vol. 35, no. 4, pp. 1316–1335, 2015.
38. [38] Y. Pan, Q. Chen, T. Chen, H. Wang, X. Zhu, Z. Fang and Y. Lu, “Evaluation of a computer-aided method form easuring the cobbangle on chest x-rays,” European Spine Journal, vol. 28, no. 12, pp. 3035–3043, 2019.
39. [39] J. Zhang, H. Li, L. Lv and Y. Zhang, “Computer-aided cobb measurement based on automatic detection of vertebral slope using deep neural network,” International Journal of Biomedical Imaging, vol. 2017, no. 1, pp. 1–6, 2017.
40. [40] R. Choi, K. Watanabe, H. Jinguji, N. Fujita, Y. Ogura, S. Demura, T. Kotani, K. Wada, M. Miyazaki, H. Shigematsu, and Y. Aoki, “CNN-based spine and cobbangle estimator using moire images,” IIEEJ Transactions on Image Electronicsand Visual Computing, vol. 5, no. 2, pp. 135–144, 2017.
41. [41] K. Zhang, N. Xu, G. Yang and J. Wu, “An automated cobbangle estimation method using convolutional neural network with area limitation,” inMICCAI 2019:Medical Image Computing and Computer Assisted Intervention, 1st ed., vol. 11769, Cham, Germany: Springer, 2019, pp. 775–783.
42. [42] W. E. Thong, H. Labelle, J. Shen, S. Parent and S. Kadoury, “Stackedauto-encoders for classification of 3d spine models in adolescentidio pathic scoliosis,” in Recent Advances in Computational Methods and Clinical Applications for Spine Imaging, 1st ed., vol. 20, Cham, Germany: Springer, 2018, pp. 13–25.
43. [43] Z. Tan, K. Yang, Y. Sun, B. Wu, H. Tao, Y. Hu and J. Zhang, “An automatic scoliosis diagnosis and measurement system based on deeplearning,” in 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2018, pp. 439–443.
44. [44] N. Sabri, H. N. A. Hamed, Z. Ibrahim and K. Ibrahim, “2D photogrammetry image of scoliosislenke typ eclassification using deeplearning,” in IEEE 9th International Conference on System Engineering and Technology, 2019, pp. 437–440.
45. [45] H. Anitha, “Lenke’s scoliosis classification using image processing techniques,” International Journal of Advanced Research in Computer and Communication Engineering, vol. 3, no. 9, pp. 56–71, 2014.
46. [46] L. Ramirez, N. G. Durdle, V. J. Raso and D. L. Hill, “A support vector machines classifierto assess the severity of idiopathic scoliosis from surface topography,” IEEE Transactionson Information Technology in Biomedicine, vol. 10, no. 1, pp. 84–91, 2006. [47] N. Mezghani, P. Phan, H. Labelle, C. E. Aubin and J. D. Gui, “Computer-aidedlenke classification of scoliotic spines,” World Academy of Science Engineering and Technology, vol. 53, no. 1, pp. 722–725, 2009.
47. [48] J. Zhang, E. Lou and X. Shi, “A computer-aided cobbangle measurement method and its reliability,”Journal of Spinal Disorders and Techniques, vol. 23, no. 6, pp. 383–387, 2010.
48. [49] O. Al Okashi, H. Du and H. Al-Assam, “Automatic spine curvature estimation from X-ray images of a mouse model,” ComputerMethods and Programs in Biomedicine, vol. 140, no. 1, 175–184, 2017.
49. [50] B. Samuvel, V. Thomas and M. G. Mini, “A mask based segmentation algorithm for automatic measurement of cobbangle from scoliosis x-ray image,” in International Conference on Advances in Computing and Communications, 2012, pp. 110–113.
50. [51] B. A. Kusuma, H. A. Nugroho and S. Wibirama, “Spinal curvature determination from scoliosisx-ray image using sum of squared difference template matching,” in 2nd International Conference on Science and Technology-Computer (ICST ‘16), 2016, pp. 29–34.
51. [52] N. B. Toan, T. Q. D. Khoa and V. Van Toi, “Computer-aid cobb measurement of scoliosis using deformable model with fuzzy spatial relations,” in 4th International Conference on Biomedical Engineering in Vietnam, 2013, pp. 350–353.
52. [53] V. Giannoglou and E. Stylianidis, “Review of advances in cobbangle calculation and image-based modeling techniques for spinal deformities,” ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 3, no. 5, pp. 12–19, 2016.
53. [54] J. Yang, K. Zhang, H. Fan, Z. Huang, Y. Xiang, J. Yang, L. He, L. Zhang, Y. Yang, R. Li, Y. Zhu, C. Chen, F. Liu, H. Yang, Y. Deng, W. Tan, N. Deng, X. Yu, X. Xuan, X. Xie, X. Liu and H. Lin, “Development and validation of deeplearning algorithms for scoliosis screening using backimages,” Communications Biology, vol. 2, no. 1, pp. 1-8, 2019.
54. [55] K. C. Kim, H. S. Yun, S. Kim and J. K. Seo, “Automation of spine curve assessment in frontal radiographs using deep learning of vertebral-tiltvector,” IEEE Access, vol. 8, no. 1, pp. 84618-84630, 2020.
55. [56] C. Vergari, W. Skalli and L. Gajny, “A convolutional neural network todetect scoliosis treatment in radiographs,” International Journal of Computer Assisted Radiology and Surgery, vol. 15, no. 6, pp. 1069-1074, 2020.
56. [57] Z. He, Y. Wang, X. Qin, R. Yin, Y. Qiu, K. He and Z. Zhu, “Classification of neurofibromatosis‐related dystrophicornondystrophic scoliosis based on image features using bilateral CNN,” Medical Physics, vol. 48, no. 4, pp. 1571-1583, 2021.
57. [58] T. Ungi, H. Greer, K. R. Sunderland, V. Wu, Z. M. Baum, C. Schlenger, M. Oetgen, K. Cleary, S. R. Aylward and G. Fichtinger, “Automatic spine ultrasound segmentation for scoliosis visualization and measurement,” IEEE Transactions on Biomedical Engineering, vol. 67, vol. 11, pp. 3234-3241, 2020.
58. [59] S. Sabour, N. Frosstand G. E. Hinton, “Dynamic routing between capsules,” 2017, arXiv:1710.09829.
59. [60] M. K. Patrick, A. F. Adekoya, A. A. Mighty and B. Y. Edward, “Capsule networks–a survey,” Journal of King Saud University-Computer and Information Sciences, 2019, doi: 10.1016/j.jksuci.2019.09.014.
60. [61] D. De. (2017, November 1). What is a Caps Netor Capsule Network? [Online]. Available: https://medium.com/hackernoon/what-is-a-capsnet-or-capsule-network-2bfbe48769cc [62] H. Wu, C. Bailey, P. Rasoulinejad and S. Li, “Automatic land mark estimation for adolescent idiopathic scoliosis assessment using boostnet,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2017, pp. 127–135.
61. [63] R. H. Alharbi, M. B. Alshaye, M. M. Alkanhal, N. M.Alharbi, M. A. Alzahrani and O. A. Alrehaili, “Deeplearning based algorithm for automatic scoliosis angle measurement,” in 3rd International Conference on Computer Applications & Information Security (ICCAIS), 2020, pp. 1–5.