Detection of human sperm cells using deep learning-based object detection methods

Year 2024, Volume: 30 Issue: 4, 482 - 493, 30.08.2024

Mecit Yüzkat , Hamza Osman İlhan , Nizamettin Aydın

Abstract

Infertility has become a significant health issue worldwide in the last 50 years. This issue, with varying rates across different regions of the world, affects approximately one out of every ten couples on average globally. Diagnosis of male-related infertility is conducted by evaluating sperm quality. When investigating sperm quality, factors such as sperm count, motility, and morphological structure are assessed. Detection of sperm before the analysis of sperm motility and count is an important step. In this study, autonomous sperm detection was carried out using deep learning methods, namely Faster R-CNN and YOLOv3, on a newly generated and unique semen video dataset. This distinct dataset, created within the scope of this study, includes semen videos from 10 patients obtained with the assistance of a mobile phone under a microscope. Videos contain label information that classifies objects as sperm and non-sperm. Labeled videos prepared for analysis were evaluated under two scenarios: patient-focused and patient-independent. In the first scenario, eight labeled videos were combined to train and test Faster R-CNN and YOLOv3 models in three different ratios. In the second scenario, each trained model was tested with two videos that had never been part of the training process. In this second scenario, detection performances were evaluated using videos that had not been involved in training. The study achieved sperm detection results of approximately 96% in individual videos using the YOLOv3 model and an average mAP of 84.5%. When compared against two significant criteria, object detection accuracy and training times, the YOLOv3 method was observed to be more successful than the Faster R-CNN method.

Keywords

Infertility, Video analysis, Sperm detection, YOLOv3, Faster R-CNN

Derin öğrenme temelli nesne tespit yöntemleri kullanılarak insan sperm hücrelerinin tespiti

Abstract

Kısırlık son 50 yılda dünya çapında önemli bir sağlık problemi haline gelmiştir. Dünyanın farklı bölgelerine göre oranı değişen bu problem ortalama olarak dünyadaki her on çiften birini etkilemektedir. Erkek kaynaklı kısırlık teşhisi, sperm kalitesi değerlendirilerek yapılır. Sperm kalitesi araştırılırken sperm sayısı, hareketi ve morfolojik yapısı değerlendirilir. Sperm hareketi ve sayımının analizi öncesinde sperm tespiti önemli bir adımdır. Bu çalışmada, oluşturulan özgün yeni semen video veri kümesi üzerinde Faster R-CNN ve YOLOv3 derin öğrenme yöntemleri kullanılarak otonom sperm tespiti gerçekleştirilmiştir. Çalışma kapsamında oluşturulan bu özgün veri kümesi, mikroskop altında semen örneklerinin cep telefonu yardımı ile elde edilen 10 hastaya ait semen videolarını ve içeriğindeki nesnelerin sperm ve sperm olmayan şeklindeki etiket bilgisini içermektedir. Analiz için hazırlanmış etiketli videolar hasta odaklı ve hasta bağımsız olmak üzere iki senaryo ile değerlendirilmiştir. İlk senaryomuzda etiketli sekiz video birleştirilerek Faster R-CNN ve YOLOv3 modelleri 3 farklı oranda oluşturulmuş veriler ile eğitilmiş ve test edilmiştir. İkinci senaryoda ise eğitilmiş her bir modelimiz daha önce eğitime hiç katılmamış iki video ile test edilmiştir. İkinci senaryomuzda, eğitimine hiç katılmamış videolar kullanılarak algılama performansları değerlendirilmiştir. Yapılan çalışmada YOLOv3 modeli ile bireysel videolarda %96, ortalama da ise %84,5 gibi mAP sperm tespit sonuçları elde edilmiştir. Sonuçlar nesne tespitinin doğruluğu ve eğitim süreleri gibi iki önemli kriter ile karşılaştırıldığında YOLOv3 yöntemi Faster R-CNN yönteminden daha başarılı olduğu gözlemlenmiştir.

Keywords

Kısırlık, Video analizi, Sperm tespiti, YOLOv3, Faster R-CNN

References

• [1] World Health Organization. World Health Statistics. Geneva, Switzerland, World Health Organization, 2010.
• [2] Carlsen E, Giwercman A, Keiding N, Skakkebæk NE. “Evidence for decreasing quality of semen during past 50 years”. British Medical Journal, 305, 609-613, 1992.
• [3] Petraglia F, Serour GI, Chapron C. “The changing prevalence of infertility”. International Journal of Gynecology & Obstetrics, 123, 4-8, 2013.
• [4] Agarwal A, Mulgund A, Hamada A, Chyatte MR. “A unique view on male infertility around the globe”. Reproductive Biology and Endocrinology, 13(1), 1-9, 2015.
• [5] Eze UA, Okonofua FE. “High prevalence of male infertility in Africa: are Mycotoxins to blame?”. African Journal of Reproductive Health, 19(3), 9-17, 2015.
• [6] Sharlip ID, Jarow JP, Belker AM, Lipshultz, LI, Sigman M, Thomas AJ, Sadovsky R. “Best practice policies for male infertility”. Fertility and Sterility, 77(5), 873-882, 2022.
• [7] Thonneau P, Marchand S, Tallec A, Ferial ML, Ducot B, Lansac J, Spira A. “Incidence and main causes of infertility in a resident population (1 850 000) of three French regions (1988–1989)”. Human Reproduction, 6(6), 811-816, 1991.
• [8] Fainberg J, Kashanian JA. “Recent advances in understanding and managing male infertility”. F1000Research, 8, 1-8, 2019.
• [9] Alahmar AT. “Role of oxidative stress in male infertility: an updated review”. Journal of Human Reproductive Sciences, 12(1), 4-18, 2019.
• [10] Sinclair, S. “Male infertility: nutritional and environmental considerations”. Alternative Medicine Review: A Journal of Clinical Therapeutic, 5(1), 28-38, 2000.
• [11] Olayemi FO. “Review on some causes of male infertility”. African Journal of Biotechnology, 9(20), 1-12, 2010.
• [12] Macomber D, Sanders, MB. “The spermatozoa count”. New England Journal of Medicine, 200(19), 981-984, 1929.
• [13] World Health Organisation. Who Laboratory Manual for the Examination of Human Semen and Sperm-Cervical Mucus Interaction. Cambridge, UK, Cambridge University Press, 1999
• [14] Makler A. “The improved ten-micrometer chamber for rapid sperm count and motility evaluation”. Fertility and Sterility, 33, 337-338, 1980.
• [15] Smith JT, Mayer DT. “Evaluation of sperm concentration by the hemacytometer method: Comparison of four counting fluids”. Fertility and Sterility, 6(3), 271-275, 1955.
• [16] Centola, GM. “Semen assessment”. Urologic Clinics, 41(1), 163-167, 2014.
• [17] Yániz J, Alquézar-Baeta C, et al. “Expanding the limits of computer-assisted sperm analysis through the development of open software”. Biology, 9(8), 1-16, 2020.
• [18] Urbano LF. Robust Automatic Multi-Sperm Tracking in Time-Lapse Images. PhD Dissertation, Drexel University, Philadelphia, USA, 2014.
• [19] Amann RP, Waberski D. “Computer-assisted sperm analysis (CASA): Capabilities and potential developments”. Theriogenology, 81(1), 5-17, 2014.
• [20] Engel KM, Grunewald S, Schiller J, Paasch U. “Automated semen analysis by SQA Vision® versus the manual approach-A prospective double‐blind study”. Andrologia, 51(1), 1-10, 2019.
• [21] Menkveld R, Stander FS, Kotze TJV, Kruger TF, Zyl JAV. “The evaluation of morphological characteristics of human spermatozoa according to stricter criteria”. Human Reproduction, 5(5), 586-592, 1990.
• [22] Yeung CH, Nıeschlag E. “Performance and comparison of CASA systems equipped with different phase‐contrast optics”. Journal of Andrology, 14(3), 222-228, 1993.
• [23] Padilla R, Netto SL, Da Silva EA. “A survey on performance metrics for object-detection algorithms”. IEEE In 2020 International Conference on Systems, Signals and Image Processing (IWSSIP). Niteroi, Brazil, 1-3 July 2020.
• [24] Girshick R, Donahue J, Darrell T, Malik J. “Rich feature hierarchies for accurate object detection and semantic segmentation”. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Ohio, USA, 23-28 June 2014.
• [25] Girshick R. “Fast r-cnn”. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7-13 December 2015.
• [26] Ren S, He K, Girshick R, Sun J. “Faster r-cnn: Towards real-time object detection with region proposal networks”. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137-1149, 2016.
• [27] He K, Gkioxari G, Dollár P, Girshick R. “Mask r-cnn”. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22-29 October 2017.
• [28] Redmon J, Divvala S, Girshick R, Farhadi A. “You only look once: Unified, real-time object detection”. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Nevada, USA, 26 June-1 July 2016.
• [29] Redmon J, Farhadi A. “YOLO9000: better, faster, stronger”. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, 21-26 July 2017.
• [30] Redmon J, Farhadi A. “Yolov3: an incremental improvement”. ArXiv, 2018. https://arxiv.org/pdf/1804.02767
• [31] Bochkovskiy A, Wang CY, Liao HYM. “Yolov4: optimal speed and accuracy of object detection”. ArXiv, 2020. https://arxiv.org/pdf/2004.10934
• [32] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC. “SSD: single shot multibox detector”. In European Conference on Computer Vision, Amsterdam, Netherlands, 11-14 October 2016.
• [33] Lin TY, Goyal P, Girshick R, He K, Dollár P. “Focal loss for dense object detection”. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22-29 October 2017.
• [34] Zou Z, Shi Z, Guo Y, Ye J. “Object detection in 20 years: A survey”. ArXiv, 2019. https://arxiv.org/pdf/1905.05055
• [35] Benjdira B, Khursheed T, Koubaa A, Ammar A, Ouni K. “Car detection using unmanned aerial vehicles: Comparison between faster r-cnn and yolov3”. In 2019 1st International Conference on Unmanned Vehicle Systems-Oman (UVS), Muscat, Oman, 5-7 February 2019.
• [36] Jekhor. “Aerial-Cars-Dataset”. https://github.com/jekhor/aerial-cars-dataset. (20.08 2022).
• [37] Adarsh P, Rathi P, Kumar M. “YOLO v3-Tiny: Object Detection and Recognition using one stage improved model”. In 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 6-7 March 2020.
• [38] Baygin M, Tuncer I, Dogan S, Barua PD, Tuncer T, Cheong KH, Acharya UR. “Automated facial expression recognition using exemplar hybrid deep feature generation technique”. Soft Computing, 27(13), 8721-8737, 2023.
• [39] Muezzinoglu T, Baygin N, et al. “PatchResNet: multiple patch division–based deep feature fusion framework for brain tumor classification using MRI images”. Journal of Digital Imaging, 36(3), 973-987, 2023.
• [40] Sut SK, Koc M, et al. “Automated adrenal gland disease classes using patch-based center symmetric local binary pattern technique with CT images”. Journal of Digital Imaging, 36(3), 879-892, 2023.
• [41] Cai Z, Vasconcelos N. “Cascade r-cnn: Delving into high quality object detection”. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Utah, USA, 18-22 June 2018.
• [42] Lin TY, Maire M, et al. “Microsoft coco: Common objects in context”. Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, 6-12 September 2014.
• [43] Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A. “The pascal visual object classes (voc) challenge”. International Journal of Computer Vision, 88(2), 303-338, 2010.
• [44] Lammers J, Chtourou S, Reignier A, Loubersac S, Barrière P, Fréour T. “Comparison of two automated sperm analyzers using 2 different detection methods versus manual semen assessment”. Journal of Gynecology Obstetrics and Human Reproduction, 50(8), 1-7, 2021.
• [45] Urbano LF, Masson P, VerMilyea M, Kam M. “Automatic tracking and motility analysis of human sperm in time-lapse images”. IEEE Transactions on Medical Imaging, 36(3), 792-801, 2016.
• [46] Ilhan HO, Yuzkat M, Aydin N. “Sperm Motility Analysis by using Recursive Kalman Filters with the smartphone based data acquisition and reporting approach”. Expert Systems with Applications, 186, 1-12, 2021.
• [47] Hidayatullah P, Mengko TLER, Munir R. “A survey on multisperm tracking for sperm motility measurement”. International Journal of Machine Learning and Computing, 7(5), 144-151, 2017.
• [48] Broekhuijse MLWJ, Šoštarić E, Feitsma H, Gadella, BM. “Additional value of computer assisted semen analysis (CASA) compared to conventional motility assessments in pig artificial insemination”. Theriogenology, 76(8), 1473-1486, 2011.
• [49] Hidayatullah P, Wang X, et al. “DeepSperm: A robust and real-time bull sperm-cell detection in densely populated semen videos”. Computer Methods and Programs in Biomedicine, 209, 1-12, 2021.
• [50] Aggarwal M, Nair VS, Sun T. “Using deep learning to streamline intracytoplasmic sperm injection in cancer Patients”.http://web.stanford.edu/~manav/CS%20231N%20Final%20Paper.pdf (10.08.2022).
• [51] Valiuškaitė V, Raudonis V, Maskeliūnas R, Damaševičius R, Krilavičius T. “Deep learning based evaluation of spermatozoid motility for artificial insemination”. Sensors, 21(1), 1-14, 2020.
• [52] Rahimzadeh M, Attar A. “Sperm detection and tracking in phase-contrast microscopy image sequences using deep learning and modified csr-dcf”. ArXiv, 2020. https://arxiv.org/pdf/2002.04034
• [53] Hicks SA, Andersen JM, et al. “Machine learning-based analysis of sperm videos and participant data for male fertility prediction”. Scientific Reports, 9(1), 1-10, 2019.
• [54] Haugen TB, Hicks SA, et al. “Visem: A multimodal video dataset of human spermatozoa”. In Proceedings of the 10th ACM Multimedia Systems Conference, Massachusetts, USA, 18-21 June 2019.
• [55] Thambawita V, Halvorsen P, Hammer H, Riegler M, Haugen TB. “Extracting temporal features into a spatial domain using autoencoders for sperm video analysis”. ArXiv, 2019. https://arxiv.org/pdf/1911.03100
• [56] Zou S, Li C, et al. “TOD-CNN: An effective convolutional neural network for tiny object detection in sperm videos”. Computers in Biology and Medicine, 146, 1-12, 2022.
• [57] Chen A, Li C, et al. “SVIA dataset: A new dataset of microscopic videos and images for computer-aided sperm analysis”. Biocybernetics and Biomedical Engineering, 42(1), 204-214, 2022.
• [58] Gümüşçü A, Tenekeci ME. “Estimation of active sperm count in spermiogram using motion detection methods”. Journal of the Faculty of Engineering and Architecture of Gazi University, 34(3), 1274-1280, 2019.
• [59] Ravanfar MR, Moradi MH. “Low contrast sperm detection and tracking by watershed algorithm and particle filter”. In 2011 18th Iranian Conference of Biomedical Engineering (ICBME), Teheran, Iran, 20-21 December 2011.
• [60] Abbiramy VS, Shanthi V, Allidurai C. “Spermatozoa detection, counting and tracking in video streams to detect asthenozoospermia”. In 2010 International Conference on Signal and Image Processing, Chennai, India, 17-19 December 2010.
• [61] Nurhadiyatna A, Latifah AL, Fryantoni D, Wirahman T, Wijayanti R, Muttaqien FH. “Comparison and implementation of motion detection methods for sperm detection and tracking”. In 2014 International Symposium on Micro-NanoMechatronics and Human Science (MHS), Surabaya, Indonesia, 8-10 November 2014.
• [62] Mifune A, Ezaki Y, Saito D, Uto K, Ikeuchi M. “Image-Based Cell Sorting System Using Light-Actuated Microvalve Array”. In 2022 IEEE 35th International Conference on Micro Electro Mechanical Systems Conference (MEMS), Sapporo, Japan, 23-27 January 2022.
• [63] Sasaki H, Nakata M, Yamamoto M, Takeshima T, Yumura Y, Hamagami T. “Investigation about control of false positive rate for automatic sperm detection in assisted reproductive technology”. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan, 7-10 October 2018.
• [64] Hung J, Carpenter A. “Applying faster R-CNN for object detection on malaria images”. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Hawaii, USA, 21-26 July 2017.
• [65] D. Tzutalin. “LabelImg is a Graphical Image Annotation Tool and Label Object Bounding Boxes in Images”. https://github.com/tzutalin/labelImg (20.05.2022).
• [66] Nguyen TH, Kandel M, Shakir HM, Best-Popescu C, Arikkath J, Do MN, Popescu G. “Halo-free phase contrast microscopy”. Scientific Reports, 7(1), 1-10, 2017.