Detection of Microscopic Urine Image Contents with Mask R-CNN

Yıl 2023, , 1180 - 1189, 30.10.2023

Yunus Emre Yörük , Hamdi Melih Saraoğlu , Ömer Faruk Özer

https://doi.org/10.35414/akufemubid.1278080

Öz

Urinary particles in microscopic images provide important information about the body when they
analyse carefully and correctly. Based on the importance of urinalysis for human health, artificial
intelligence applications were made using deep learning image processing technique in order to detect
microscopic urine contents. Most of the studies in the literature have generally focused on semantic
segmentation. Unlike the others, in this study, the urinary contents of red blood cells, white blood cells,
epithelium, crystals, bacteria and yeast in microscopic urine images were determined using Mask R
CNN, which can perform instance segmentation. In object detection with Mask R-CNN, two types of
boundaries are drawn as mask and bounding box. The performance of the system is examined for both
boundary types. From a total of 1154 patterns in 100 images used for the test, 808 with masks and 843
with bounding boxes were correctly identified (IoU=0.5). The best detection occurred for white and red
blood cells. Epithelium has also been successfully identified according to bounding boxes, but there
were problems creating masks. Bacteria detection success rate is low because bacteria are so small.
Most of the crystals and yeast were correctly detected. In addition, mAP, a frequently used evaluation
metric for object detection, was also calculated. Calculated mAP values are 0.7842 and 0.8343 for masks
and bounding boxes respectively. Mask R-CNN can be used in urine analysis systems if it is well
optimized and trained with images of more urine contents.

Anahtar Kelimeler

Mask R-CNN, Urinalysis, Deep Learning, Instance Segmentation

Proje Numarası

220N334 – 121N231

Mask R-CNN ile Mikroskobik İdrar Görüntüsü İçeriklerinin Tespiti

Öz

Mikroskobik idrar içerikleri doğru ve dikkatli bir şekilde analiz edildiğinde vücut hakkında önemli bilgiler
verir. İdrar tahlilinin insan sağlığı için önemi nedeniyle mikroskobik idrar içeriklerinin tespit edilmesi
amacıyla derin öğrenme görüntü işleme tekniği kullanılarak yapay zeka uygulamaları yapılmıştır.
Literatürde yer alan çalışmaların çoğunda genel olarak semantik segmentasyon üzerine yoğunlaşılmıştır.
Bu çalışmada ise piksel düzeyinde segmentasyon yapabilen Mask R-CNN modeli ile mikroskobik idrar
görüntülerindeki alyuvar, akyuvar, epitel, kristal, bakteri ve mantar içerikleri konum ve nesne türü
bilgisiyle birlikte tespit edilmiştir. Mask R-CNN ile tespit edilen nesnelere maske ve çerçeve olmak üzere
iki tip sınır çizilmektedir. Sistemin performansı her iki sınır tipi için ayrı ayrı incelenmiştir. Test için
kullanılan 100 görüntüdeki toplam 1154 örüntüden maskelere göre 808 ve çerçevelere göre 843 nesne
doğru şekilde tespit edilmiştir (IoU=0,5). En iyi tespit oranı akyuvarlar ve alyuvarlar için gerçekleşmiştir.
Epiteller çerçevelere göre hesaplamada başarılı bir şekilde tespit edilmiştir fakat düzgün maske
oluşturulamamıştır. Bakteriler diğerlerine göre çok küçük olduğu için doğru tespit oranı düşük kalmıştır.
Kristallerin ve mantarların çoğu doğru şekilde tespit edilmiştir. Ayrıca, nesne tespitinde sıklıkla kullanılan
değerlendirme metriği mAP de hesaplanmıştır. Sistem için hesaplanan mAP değerleri maskelere göre
0,7842 ve çerçevelere göre 0,8343 olmuştur. Mask R-CNN sistemi iyi bir şekilde optimize edilip daha
fazla idrar içeriğine ait görüntülerle eğitilmesi durumunda idrar analiz sistemlerinde kullanılabilir.

Anahtar Kelimeler

Derin Öğrenme, İdrar Analizi, Piksel Düzeyinde Segmentasyon, Mask R-CNN

Destekleyen Kurum

TÜBİTAK

Proje Numarası

220N334 – 121N231

Teşekkür

TÜBİTAK 1071 – Uluslararası Araştırma Fonlarından Yararlanma Kapasitesinin ve Uluslararası Ar-Ge İşbirliklerine Katılımın Arttırılmasına Yönelik Destek Programı kapsamında desteklenmiştir. Bu çalışmada kullanılan idrar görüntülerinin temini konusunda Bezmialem Vakıf Üniversitesi Hastanesi destek vermiştir.

Kaynakça

• Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L., 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition (pp. 248-255). IEEE.
• Flach, P., & Kull, M., 2015. Precision-recall-gain curves: PR analysis done right. Advances in Neural Information Processing Systems, 28(1), 838-846.
• Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., & Garcia-Rodriguez, J., 2017. A review on deep learning techniques applied to semantic segmentation. arXiv preprint arXiv:1704.06857.
• Girshick, R., Donahue, J., Darrell, T., & Malik, J., 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 580-587). IEEE.
• Girshick, R., 2015. Fast r-cnn. In IEEE International Conference on Computer Vision (pp. 1440-1448). IEEE.
• Greenspan, H., Van Ginneken, B., & Summers, R. M., 2016. Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE Transactions on Medical Imaging, 35(5), 1153-1159.
• He, K., Zhang, X., Ren, S., & Sun, J., 2016. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (pp. 770-778). IEEE.
• He, K., Gkioxari, G., Dollár, P., & Girshick, R., 2017. Mask r-cnn. IEEE International Conference on Computer Vision (pp. 2961-2969). IEEE.
• Hu, X., Zhang, J., & Zhang, X., 2010. Evaluation of the Sysmex UF-1000i urine analyzer as a screening test to reduce the need for urine cultures for urinary tract infection. Laboratory Medicine, 41(6), 349-352.
• İnce, H., İmamoğlu, S. E., & İmamoğlu, S. Z., 2021. Yapay zeka uygulamalarının karar verme üzerine etkileri: Kavramsal bir çalışma. International Review of Economics and Management, 9(1), 50-63.
• Kouri, T., Fogazzi, G., Gant, V., Hallander, H., Hofmann, W., & Guder, W. G., 2000. European urinalysis guidelines. Scandinavian journal of clinical and laboratory investigation, 60(sup231), 1-96.
• Li, Y., Huang, H., Xie, Q., Yao, L., & Chen, Q., 2018. Research on a surface defect detection algorithm based on MobileNet-SSD. Applied Sciences, 8(9), 1678.
• Li, X., Li, M., Wu, Y., Zhou, X., Hao, F., & Liu, X., 2020. An accurate classification method based on multi-focus videos and deep learning for urinary red blood cell. Conference on Artificial Intelligence and Healthcare (pp. 67-71). ACM Digital Library.
• Liang, Y., Kang, R., Lian, C., & Mao, Y., 2018. An end-to-end system for automatic urinary particle recognition with convolutional neural network. Journal of Medical Systems, 42(9), 1-14.
• Lin, T. Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S., 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2117-2125). IEEE.
• Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C., 2016. Ssd: Single shot multibox detector. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14 (pp. 21-37). Springer International Publishing.
• Pande, B., Padamwar, K., Bhattacharya, S., Roshan, S., & Bhamare, M., 2022. A Review of Image Annotation Tools for Object Detection. In 2022 International Conference on Applied Artificial Intelligence and Computing (pp. 976-982). IEEE.
• Perazella, M. A., 2015. The urine sediment as a biomarker of kidney disease. American Journal of Kidney Diseases, 66(5), 748-755.
• Primas, S. R., 2018. The AutoScope: an automated point-of-care urinalysis system. Unpublished Doctoral Dissertation, Massachusetts Institute of Technology, USA.
• Rahman, M. A., & Wang, Y., 2016. Optimizing intersection-over-union in deep neural networks for image segmentation. In International symposium on visual computing (pp. 234-244). Springer, Cham.
• Redmon, J., Divvala, S., Girshick, R., & Farhadi, A., 2016. You only look once: Unified, real-time object detection. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 779-788). IEEE.
• Ren, S., He, K., Girshick, R., & Sun, J., 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 28, 91-99.
• Schwenke, C., & Schering, A. G., 2014. True positives, true negatives, false positives, false negatives. Wiley StatsRef: Statistics Reference Online.
• Simerville, J. A., Maxted, W. C., & Pahira, J. J., 2005. Urinalysis: a comprehensive review. American family physician, 71(6), 1153-1162.
• Strasinger, S. K., & Di Lorenzo, M. S., 2014. Urinalysis and body fluids. FA Davis.
• Suhail, K., & Brindha, D., 2021. A review on various methods for recognition of urine particles using digital microscopic images of urine sediments. Biomedical Signal Processing and Control, 68, 102806.
• Wang, Q., Bi, S., Sun, M., Wang, Y., Wang, D., & Yang, S., 2019. Deep learning approach to peripheral leukocyte recognition. PloS One, 14(6), e0218808.
• Yang, S., Fang, B., Tang, W., Wu, X., Qian, J., & Yang, W., 2017. Faster R-CNN based microscopic cell detection. In 2017 International Conference on Security Pattern Analysis and Cybernetics (pp. 345-350). IEEE.
• Zaman, Z., Fogazzi, G. B., Garigali, G., Croci, M. D., Bayer, G., & Kránicz, T., 2010. Urine sediment analysis: Analytical and diagnostic performance of sediMAX®-a new automated microscopy image-based urine sediment analyser. Clinica Chimica Acta, 411(3-4), 147-154.
• Zeb, B., Khan, A., Khan, Y., Masood, M. F., Tahir, I., & Asad, M., 2020. Towards the Selection of the Best Machine Learning Techniques and Methods for Urinalysis. In Proceedings of the 2020 12th International Conference on Machine Learning and Computing (pp. 127-133). ACM Digital Library.
• Zhang, X., Chen, G., Saruta, K., & Terata, Y., 2018. Detection and classification of RBCs and WBCs in urine analysis with deep network. In ACHI 2018: The Eleventh International Conference on Advances in Computer-Human Interactions (pp. 194-198). IARIA.
• Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q., 2020. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1), 43-76.
• https://pdf.medicalexpo.com/pdf/roche/compendium-urinalysis-urine-test-strips-microscopy/71020-136212.html, (30.03.2023)
• https://github.com/matterport/Mask_RCNN, (30.03.2023)
• https://www.robots.ox.ac.uk/~vgg/software/via/, (30.03.2023)
• https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173, (30.03.2023)