Comparing artificial intelligence based diagnosis with expert results in SARS-COV-2 RT-qPCR

Abstract

Objectives: Reverse transcription and real-time polymerase chain reaction (RT-qPCR) based on the SARS-CoV-2 viral RNA demonstration is the gold standard in diagnosis. Data files obtained from PCR devices should be analysed by a specialist physician and results should be transferred to Laboratory Information Management System (LIMS). CAtenA Smart PCR (Ventura, Ankara, Türkiye) program is a local bioinformatics software that assess PCR data files with artificial intelligence, submits to expert approval and transfers the approved results to LIMS. The aim of this study is to investigate its accuracy and matching success rate with expert analysis.

Methods: A total of 9400 RT-qPCR test results studied in Ankara Provincial Health Directorate Public Health Molecular Diagnosis Laboratory were compared with respect to expert evaluation and CAtenA results.

Results: It was determined that the preliminary evaluation results of the CAtenA matched 86% of the negative and 90% of the positive results provided by expert analysis. 987 tests which CAtenA determined as inconclusive and suggested repeating PCR were found either negative or positive by expert analysis. A significant difference between positive and negative matching success rates and artificial intelligence (AI) based software overall accuracy was found and associated with the missed tests of the AI.

Conclusions: As a result, it was suggested there is a low risk of confirming false positive results without expert analysis and test repetitions would cause losing time along with extra test costs. It was agreed that the PCR analysis used in CAtenA should be improved particularly in terms of test repetitions.

Keywords

• SARS-CoV-2
• RT-qPCR
• expert analysis
• CAtenA
• artificial intelligence

References

1. 1. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Eng J Med 2020;382:727-33.
2. 2. World Health Organization. WHO Director-General’s opening remarks at the media briefing on COVID-19, 11 March 2020. (online). Available from: https://www.who.int/ director-general/speeches/detail/who-directorgeneral-s-opening-remarks-at-the-media-briefing-oncovid-19---11-march-2020) (cited 14 April 2022).
3. 3. World Health Organization, Tracking SARS-CoV-2 variants (online). Available from: https://www.who.int/en/activities/tracking-sars-cov-2-variants/ (cited 14 April 2022).
4. 4. Worl Health Organization. Laboratory testing of 2019 novel coronavirus (2019-nCoV) in suspected human cases: interim guidance, 17 January 2020. (online). Available from: https://www.who.int/publications/i/item/laboratory-testingof-2019-novel-coronavirus-(-2019-ncov)-in-suspectedhuman-cases-interim-guidance-17-january-2020). (cited 14 April 2022).
5. 5. Sule, WF, Oluwayelu, DO. Real-time RT-PCR for COVID-19 diagnosis: challenges and prospects. Pan Afr Med J 2020;35(Suppl 2):121.
6. 6. The Turkish Ministry of Health, COVID-19 information page (online). Available from: https://covid19.saglik.gov.tr/TR 68720/covid-19-yetkilendirilmis-tani-laboratuvarlari-listesi.html. (cited 14 April 2022).
7. 7. Petruzzi G, De Virgilio A, Pichi B, Mazzola F, Zocchi J, Mercante G, et al. COVID-19: nasal and oropharyngeal swab. Head Neck 2020;42:1303-4.
8. 8. Sreepadmanabh M, Sahu AK, Chande A. COVID-19: advances in diagnostic tools, treatment strategies, and vaccine development. J Biosci 2020;45:148.

9. 9. Tasdelen A, Sen B. A hybrid CNN-LSTM model for pre-miRNA classification. Sci Rep 2021;11:14125.
10. 10. Peiffer-Smadja, N, Dellière S, Rodriguez C, Birgand G, Lescure F. X, Fourati S, et al. Machine learning in the clinical microbiology laboratory: has the time come for routine practice? Clin Microbiol Infect 2020;26:1300-9.
11. 11. Smith KP, Wang H, Durant TJS, Mathison BA, Sharpeh SE, Kirby JE, et al. Application of artificial intelligence in clinical microbiology diagnostic testing. Clin Microbiol Newsletter 2020;42:61-70.
12. 12. Asada K, Kaneko S, Takasawa K, Machino H, Takahashi S, Shinkai N, et al. Integrated analysis of whole genome and epigenome data using machine learning technology: toward the establishment of precision oncology. Front Oncol 2021;11:666937.
13. 13. van Oosten LN, Klein CD. Machine learning in mass spectrometry: a MALDI-TOF MS approach to phenotypic antibacterial screening. J Med Chem 2020;63:8849-56.
14. 14. Ventura. CAtenA Smart PCR. (online). Available from: https://ventura.com.tr/catena (cited 14 April 2022).
15. 15. Kuang J, Yan X, Genders AJ, Granata C, Bishop DJ. An overview of technical considerations when using quantitative real-time PCR analysis of gene expression in human exercise research. PloS One 2018;13:e0196438.