A Residual Neural Network with a Novel Orthogonal Regularization for Covid-19 Diagnosis using X-ray images

Abstract

Covid-19, solunum yollarını etkileyen ve küresel ölçekte ciddi sağlık sorunlarına yol açan viral bir enfeksiyondur. Hastalığın yüksek bulaşıcılığı nedeniyle erken teşhis ve doğru sınıflandırma büyük önem taşımaktadır. Bu çalışmada, X-ray görüntülerinden Covid-19 hastalığının tespit doğruluğunu artırmak için yeni bir ortogonal düzgünleştirme yöntemi önerilmektedir. ResNet110 kullanılarak test edilen önerilen düzgünleştirme yöntemiyle, geleneksel birimdik düzgünleştirme yaklaşımlarına göre sınıflandırma doğruluğu artırılmaktadır. Deneysel çalışmalarda, önerilen yöntem, farklı düzgünleştirme teknikleriyle karşılaştırılmış ve test doğruluk oranını %96,52 seviyesine çıkararak en yüksek sınıflama başarı oranına ulaşılmıştır. Bunun yanında önerilen yöntemin özellikle eğitim sürecinin ilerleyen aşamalarında modelin öğrenme eğrisini optimize ettiği ve test sınıflandırma doğruluğunu artırdığı gözlemlenmiştir. Ayrıca, Covid-19 tespiti için mevcut birimdik düzgünleştirme yöntemleriyle karşılaştırıldığında, önerilen yaklaşım ile test sınıflandırma başarımı doğruluk, f1score, duyarlılık, keskinlik ve özgünlük metriklerinde yaklaşık %1 oranında iyileştirme sağlanmıştır.

Keywords

• Derin Öğrenme
• Artık Ağlar
• Ortagonal Düzgünleştirme
• Covid-19

A Residual Neural Network with a Novel Orthogonal Regularization for Covid-19 Diagnosis using X-ray images

Abstract

Covid-19 is a viral infection that affects the respiratory tract and causes serious health problems on a global scale. Due to the high contagiousness of the disease, early detection and accurate classification are of great importance. In this study, a novel orthogonal regularization method is proposed to improve the detection accuracy of Covid-19 disease from X-ray images. The proposed regularization method, evaluated using ResNet110 using ResNet110, improves the classification accuracy compared to traditional Orthogonal regularization approaches. In the experimental studies, the proposed method is compared with various regularization techniques and the highest classification success rate is achieved by increasing the test accuracy rate to 96.52%. In addition, it is observed that the proposed method optimizes the learning curve of the model, especially in the later stages of the training process, and increasing the test accuracy. In addition, compared to the existing orthogonal regularization methods for Covid-19 detection, the proposed approach improved the test classification performance by approximately 1% in accuracy, F1-score, sensitivity, sharpness and specificity metrics.

Keywords

• Deep Learning
• Residual Network
• Orthogonal Regularization
• Covid-19

References

1. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A Novel Coronavirus from Patients with Pneumonia in China, 2019. N Engl J Med [Internet]. 2020 Feb 20;382(8):727–33. Available from: http://www.nejm.org/doi/10.1056/NEJMoa2001017
2. Lewis D. COVID-19 rarely spreads through surfaces. So why are we still deep cleaning? Nature [Internet]. 2021 Feb 4;590(7844):26–8. Available from: https://www.nature.com/articles/d41586-021-00251-4.
3. WHO. COVID-19 Epidemiological Update [Internet]. 2024. Available from: https://www.who.int/publications/m/item/covid-19-epidemiological-update---24-december-2024.
4. Tuncer T, Ozyurt F, Dogan S, Subasi A. A novel Covid-19 and pneumonia classification method based on F-transform. Chemom Intell Lab Syst [Internet]. 2021 Mar;210:104256. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0169743921000241.
5. Fang Y, Zhang H, Xie J, Lin M, Ying L, Pang P, et al. Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR. Radiology [Internet]. 2020 Aug;296(2):E115–7. Available from: http://pubs.rsna.org/doi/10.1148/radiol.2020200432
6. Xie X, Zhong Z, Zhao W, Zheng C, Wang F, Liu J. Chest CT for Typical Coronavirus Disease 2019 (COVID-19) Pneumonia: Relationship to Negative RT-PCR Testing. Radiology [Internet]. 2020 Aug;296(2):E41–5. Available from: http://pubs.rsna.org/doi/10.1148/radiol.2020200343
7. Quan H, Xu X, Zheng T, Li Z, Zhao M, Cui X. DenseCapsNet: Detection of COVID-19 from X-ray images using a capsule neural network. Comput Biol Med [Internet]. 2021 Jun;133:104399. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0010482521001931
8. Wong HYF, Lam HYS, Fong AHT, Leung ST, Chin TWY, Lo CSY, et al. Frequency and Distribution of Chest Radiographic Findings in Patients Positive for COVID-19. Radiology [Internet]. 2020 Aug;296(2):E72–8. Available from: http://pubs.rsna.org/doi/10.1148/radiol.2020201160

9. Narin A, Kaya C, Pamuk Z. Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks. Pattern Anal Appl [Internet]. 2021 Aug 9;24(3):1207–20. Available from: https://link.springer.com/10.1007/s10044-021-00984-y
10. Wang L, Wong A. COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images. 2020 Mar 22; Available from: http://arxiv.org/abs/2003.09871
11. Horry MJ, Chakraborty S, Paul M, Ulhaq A, Pradhan B, Saha M, et al. COVID-19 Detection Through Transfer Learning Using Multimodal Imaging Data. IEEE Access [Internet]. 2020;8:149808–24. Available from: https://ieeexplore.ieee.org/document/9167243/
12. Loey M, Manogaran G, Khalifa NEM. A deep transfer learning model with classical data augmentation and CGAN to detect COVID-19 from chest CT radiography digital images. Neural Comput Appl [Internet]. 2020 Oct 26; Available from: https://link.springer.com/10.1007/s00521-020-05437-x
13. Chowdhury MEH, Rahman T, Khandakar A, Mazhar R, Kadir MA, Mahbub Z Bin, et al. Can AI Help in Screening Viral and COVID-19 Pneumonia? IEEE Access [Internet]. 2020;8:132665–76. Available from: https://ieeexplore.ieee.org/document/9144185/
14. Rahman T, Khandakar A, Qiblawey Y, Tahir A, Kiranyaz S, Abul Kashem S Bin, et al. Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images. Comput Biol Med [Internet]. 2021 May;132:104319. Available from: https://linkinghub.elsevier.com/retrieve/pii/S001048252100113X
15. He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) [Internet]. IEEE; 2016. p. 770–8. Available from: http://ieeexplore.ieee.org/document/7780459/
16. Bishop CM. Neural networks for pattern recognition. Oxford; 1995.
17. Goodfellow IJ, Bengio Y, Courville A. Deep Learning. Cambridge, MA, USA: MIT Press; 2016.
18. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM [Internet]. 2017 May 24;60(6):84–90. Available from: https://dl.acm.org/doi/10.1145/3065386
19. Szu H, Scheff K. Gram-Schmidt Orthogonalization Neural Nets for. 2018.
20. Zhang L, Li D, Guo Q. Deep Learning From Spatio-Temporal Data Using Orthogonal Regularizaion Residual CNN for Air Prediction. IEEE Access [Internet]. 2020;8:66037–47. Available from: https://ieeexplore.ieee.org/document/9056826/
21. Bansal N, Chen X, Wang Z. Can We Gain More from Orthogonality Regularizations in Training Deep CNNs? 2018 Oct 22; Available from: http://arxiv.org/abs/1810.09102
22. Xie D, Xiong J, Pu S. All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) [Internet]. IEEE; 2017. p. 5075–84. Available from: http://ieeexplore.ieee.org/document/8100022/
23. Rodríguez P, Gonzàlez J, Cucurull G, Gonfaus JM, Roca X. Regularizing CNNs with Locally Constrained Decorrelations. 2016 Nov 7; Available from: http://arxiv.org/abs/1611.01967
24. Huang L, Liu X, Lang B, Yu A, Wang Y, Li B. Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds in deep neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2018.
25. Donoho DL. Compressed sensing. IEEE Trans Inf Theory [Internet]. 2006 Apr;52(4):1289–306. Available from: http://ieeexplore.ieee.org/document/1614066/
26. Lu C, Li H, Lin Z. Optimized projections for compressed sensing via direct mutual coherence minimization. Signal Processing [Internet]. 2018 Oct;151:45–55. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0165168418301464
27. Zhang Z, Ma W, Wu Y, Wang G. Self-Orthogonality Module: A Network Architecture Plug-in for Learning Orthogonal Filters. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) [Internet]. IEEE; 2020. p. 1044–8. Available from: https://ieeexplore.ieee.org/document/9093466/
28. Zhang L, Li D, Guo Q. Deep Learning From Spatio-Temporal Data Using Orthogonal Regularizaion Residual CNN for Air Prediction. IEEE Access. 2020;8:66037–47.
29. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
30. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations (ICLR 2015). Computational and Biological Learning Society; 2015. p. 1–14.
31. Fırıldak K, Çelik G, Talu MF. Derin Ağlar İçin Yeni Bir Birimdik Düzgünleştirme Yaklaşımı. Adıyaman Üniversitesi Mühendislik Bilim Derg [Internet]. 2024 Apr 30;11(22):18–34. Available from: http://dergipark.org.tr/tr/doi/10.54365/adyumbd.1390894
32. Griewank A, Walther A. Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM; 2008.
33. Powers DMW. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. 2020 Oct 10; Available from: http://arxiv.org/abs/2010.16061
34. Celik G. Detection of Covid-19 and other pneumonia cases from CT and X-ray chest images using deep learning based on feature reuse residual block and depthwise dilated convolutions neural network. Appl Soft Comput [Internet]. 2023 Jan;133:109906. Available from: https://linkinghub.elsevier.com/retrieve/pii/S1568494622009553
35. Dodia S, B. A, Mahesh PA. Recent advancements in deep learning based lung cancer detection: A systematic review. Eng Appl Artif Intell [Internet]. 2022 Nov;116:105490. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0952197622004808