Investigation of Usability of Artificial Intelligence Semantic Video Processing Methods in Medicine

Abstract

Aim: The goal of this study is to produce user-friendly software for healthcare professionals with various approaches such as detection, identification, classification, and tracking of polyps contained in endoscopic images utilizing appropriate video/image processing techniques and CNN architecture. Material and Method: There were 345 photos in total in the study. These photographs are images depicting anatomical milestones, clinical findings, or gastrointestinal procedures in the digestive tract that have been documented and validated by medical specialists (skilled endoscopists). Each class has hundreds of images. The photos were downloaded from https://datasets.simula.no/kvasir, which is a free source for educational and research purposes. In the modeling phase, CNN and the Max-Margin object detection technique (MMOD), one of the deep neural network designs in the Dlib package, were employed. The data set was separated as 80% training and 20% test dataset using the simple cross-validation method (hold-out). Precision, recall, F1-score, average precision (AP), mean average precision (mAP), ideal localization recall precision (oLRP), mean optimal LRP (moLRP), and intersection over union (IoU) were used to evaluate model performance. Results: When the previously described steps were performed on the open-access video image dataset of endoscopic polyps in the current study, all performance metrics examined in the training dataset received a value of 1, whereas, in the test dataset precision, sensitivity, F1-score, AP, mAP, oLRP, and moLRP were 98%, 90%, 94%, 89%, 89%, 48%, and 48% respectively. Conclusion: The proposed approach was found to make accurate predictions in the diagnosis of gastrointestinal polyps based on the values of the calculated performance criteria.

Keywords

• Object recognition
• deep learning
• decision support system
• gastrointestinal polyps
• convolutional neural networks

Yapay Zekâya Dayalı Anlamsal Video İşleme Yöntemlerinin Tıpta Kullanılabilirliğinin Araştırılması

Öz

Giriş: Bu çalışmada endoskopik görüntülerde yer alan poliplerin tespiti, tanımlanması, sınıflandırılması ve takibi için uygun video/görüntü işleme teknikleri ve CNN mimarisi kullanılarak sağlık profesyonelleri için kullanıcı dostu bir yazılımın geliştirilerek sunulması amaçlanmıştır. Material ve Methot: Çalışmada yer alan veri seti 345 görüntü içermekte olup görüntüler anatomik olarak bilinen dönüm noktaları, patolojik bulgular veya sindirim sistemindeki gastrointestinal prosedürler gibi her sınıf için yüzlerce görüntüden oluşmakta ve çeşitli tıp doktorları (deneyimli endoskopistler) tarafından açıklanmış ve doğrulanmıştır. Görseller araştırmalarda ve eğitimlerde kullanılmak amacıyla açık kaynak olan https://datasets.simula.no/kvasir adresinden alınmıştır. Modelleme esnasında Dlib kütüphanesinde yer alan derin sinir ağı mimarilerinden olan CNN ve Max-Margin nesne algılama yöntemi (MMOD) kullanılarak modellemeler yapılmıştır. Veri seti basit çapraz geçerlilik yöntemi (hold-out) kullanılarak %80’i eğitim, %20’si test veri seti olacak şekilde ayrıştırılmıştır. Model performansının değerlendirilmesinde ise kesinlik, duyarlılık, F1-skor, ortalama kesinlik (average precision, AP), ortalama kesinlik değerlerinin ortalaması (mean average precision, mAP), kesiştirilmiş bölgeler ölçütleri (intersection over union, IoU), en uygun konumlandırma kesinliği ve duyarlılığı (optimal localization recall precision, oLRP), ortalama en uygun LRP (Mean Optimal LRP, moLRP) kullanılmıştır. Bulgular: Mevcut çalışmada endoskopik poliplerin açık erişimli video görüntü veri kümesi üzerinde daha önce açıklanan adımlar gerçekleştirildiğinde, eğitim veri kümesinde incelenen tüm performans metrikleri 1 değerini alırken, test veri kümesinde kesinlik, duyarlılık, F1-skoru , AP, mAP, oLRP ve moLRP sırasıyla %98, %90, %94, %89, %89, %48 ve %48 idi. Sonuç: Çalışmada sonucunda elde edilen performans metriklerine ait değerler dikkate alındığında, önerilen sistemin gastrointestinal poliplerin tanısında başarılı tahmin sonuçları verdiği belirlenmiştir.

Kaynakça

1. 1. Yagin FH, Cicek IB, Kucukakcali Z. Classification of stroke with gradient boosting tree using smote-based oversampling method. Medicine. 2021;10(4):1510-5.
2. 2. Yagin FH, Kucukakcali Z, Cicek IB, Bag HG. The Effects of Variable Selection and Dimension Reduction Methods on the Classification Model in the Small Round Blue Cell Tumor Dataset. Middle Black Sea Journal of Health Science. 2021;7(3):390-6.
3. 3. Cicek İB, İlhami S, Yagin FH, Colak C. Development of a Python-Based Classification Web Interface for Independent Datasets. Balkan Journal of Electrical and Computer Engineering.10(1):91-6.
4. 4. Yagin FH, Yağin B, Arslan AK, Colak C. Comparison of Performances of Associative Classification Methods for Cervical Cancer Prediction: Observational Study. Turkiye Klinikleri Journal of Biostatistics. 2021;13(3).
5. 5. Cicek İB, Kucukakcali Z, Yagin FH. Detection of risk factors of PCOS patients with Local Interpretable Model-agnostic Explanations (LIME) Method that an explainable artificial intelligence model. The Journal of Cognitive Systems.6(2):59-63.
6. 6. Yilmaz R, Yagin FH. Early Detection of Coronary Heart Disease Based on Machine Learning Methods. Medical Records.4(1):1-6.
7. 7. Perçın İ, Yagin FH, Güldoğan E, Yoloğlu S, editors. ARM: An Interactive Web Software for Association Rules Mining and an Application in Medicine. 2019 International Artificial Intelligence and Data Processing Symposium (IDAP); 2019: IEEE.
8. 8. Perçın İ, Yagin FH, Arslan AK, Çolak C, editors. An Interactive Web Tool for Classification Problems Based on Machine Learning Algorithms Using Java Programming Language: Data Classification Software. 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT); 2019: IEEE.

9. 9. Cicek İpB, Kucukakcali Z, Çolak C. Associative classification approach can predict prostate cancer based on the extracted association rules. The Journal of Cognitive Systems.5(2):51-4.
10. 10. Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning: MIT press Cambridge; 2016.
11. 11. Dheir IM, Mettleq ASA, Elsharif AA, Abu-Naser SS. Classifying nuts types using convolutional neural network. 2020.
12. 12. Kucukakcali ZT, Cicek İpB, Güldoğan E. Performance evaluation of the deep learning models in the classification of heart attack and determination of related factors. The Journal of Cognitive Systems. 2020;5(2):99-103.
13. 13. Yagin FH, Güldoğan E, Ucuzal H, Çolak C. A Computer-Assisted Diagnosis Tool for Classifying COVID-19 based on Chest X-Ray Images. Konuralp Medical Journal.13(S1):438-45.
14. 14. Pogorelov K, Randel KR, Griwodz C, Eskeland SL, de Lange T, Johansen D, et al., editors. Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection. Proceedings of the 8th ACM on Multimedia Systems Conference; 2017.
15. 15. Yu J, Liu G. Knowledge-based deep belief network for machining roughness prediction and knowledge discovery. Comput Ind. 2020;121(7):103262.
16. 16. Deng L, Yu D. Deep learning: methods and applications. Foundations trends in signal processing. 2014;7(3–4):197-387.
17. 17. Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, et al. Deep learning: a primer for radiologists. Radiographics. 2017;37(7):2113-31.
18. 18. Minaee S, Boykov Y, Porikli F, Plaza A, Kehtarnavaz N, Terzopoulos D. Image segmentation using deep learning: A survey. arXiv preprint arXiv:05566. 2020.
19. 19. İnce Ö, Senel I, Yılmaz F. Image Processing and Analysis in Health: Advantages, Challenges, Threats and Examples. Archives of Health Science and Research. 2020;7(1):66-74.
20. 20. LeCun Y. LeNet-5, convolutional neural networks 2015 [cited 2020 27.04.2020]. Available from: http://yann.lecun.com/exdb/lenet/.
21. 21. Li Z, Dong M, Wen S, Hu X, Zhou P, Zeng Z. CLU-CNNs: Object detection for medical images. Neurocomputing. 2019;350(6):53-9.
22. 22. Billah M, Waheed S. Minimum redundancy maximum relevance (mRMR) based feature selection from endoscopic images for automatic gastrointestinal polyp detection. Multimed Tools Appl. 2020;79:1-11.