Akciğer Bilgisayarlı Tomografi Görüntülerinde Yapay Zekâ Tabanlı Örneğe Duyarlı Semantik Lob Segmentasyonu

Öz

Dünya çapında sağlık krizine neden olan Koronavirüs hastalığı (COVID-19) tüm dünyayı etkisi altına almıştır. Ölümcül sonuçlara yol açan akut hipoksemik solunum yetmezliği en çok endişe verici komplikasyondur. COVID-19'un etkisini hafifletmek için tedaviden önce enfekte olan bölge analiz edilmelidir. Bu nedenle göğüs bilgisayarlı tomografi (BT), COVID-19'a ait şiddet düzeyini belirlemek için kullanılan yaygın bir yöntemdir. Ayrıca BT görüntülerinde COVID-19 içeren lob bölgelerinin sayısı radyologların bilateral, multifokal ve multilobar gibi bulguları teşhis etmesine yardımcı olur. Lob bölgeleri radyologlar tarafından manuel olarak ayırt edilir, ancak bu uzun çalışma saatleri nedeniyle yanlış teşhislere neden olur. Bu nedenle lob bölgelerini otomatik olarak çıkarabilen yeni araçlara olan ihtiyaç artmıştır. Evrişim sinir ağları (CNN'ler), BT görüntülerinde lob bölgelerini çıkaran, örneğe duyarlı anlamsal lob segmentasyon görevinde insan hatalarını en aza indirmek için otomatik bir yaklaşım sağlar. Bu makalede, örneğe duyarlı anlamsal lob bölütleme görevinde bir kıyaslama oluşturmak için VGG-16, VGG-19 ve ResNet-50 kullanan DeepLabV3+ gibi CNN tabanlı mimariler kullanıldı. Bölütleme sonuçlarını iyileştirmek için lob bölütlemesinden önce akciğeri çıkarmada görüntülere ön işleme uygulandı. Deneysel değerlendirmeler için akciğer ve lob bölgelerini piksel düzeyinde etiketli 9036 görüntü içeren büyük ölçekli bir veri seti oluşturulmuştur. ResNet-50 kullanan DeepLabV3+, lob bölütlemede sırasıyla zar benzerlik katsayısı (DSC) ve Jaccard benzerlik katsayısı (IOU) açısından en yüksek başarımı sırasıyla ile % 99.59 ve % 99.19 olarak gösterdi. Deneyler, yaklaşımımızın, örneğe duyarlı anlamsal lob bölütleme görevi için birkaç son teknoloji yöntemden daha iyi performans gösterdi. Ayrıca, göğüs BT görüntülerinde lob bölgelerinin bölütlenmesi için LobeChestApp adlı yeni bir masaüstü uygulaması geliştirilmiştir.

Anahtar Kelimeler

• Yapay Zekâ
• Derin Öğrenme
• Örneğe Duyarlı Anlamsal Lobe Bölütlemesi
• COVID-19
• Evrişimsel Sinir Ağları.
• Artificial Intelligence
• Deep Learning
• Instance-Aware Semantic Lobe Segmentation
• Convolutional Neural Network.

Artificial Intelligence Based Instance-Aware Semantic Lobe Segmentation on Chest Computed Tomography Images

Abstract

The coronavirus disease (COVID-19) has taken the entire world under its influence, causing a worldwide health crisis. The most concerning complication is acute hypoxemic respiratory failure that results in fatal consequences. To alleviate the effect of COVID-19, the infected region should be analyzed before the treatment. Thus, chest computed tomography (CT) is a popular method to determine the severity level of COVID-19. Besides, the number of lobe regions containing COVID-19 on CT images helps radiologists to diagnose the findings, such as bilateral, multifocal, and multilobar. Lobe regions can be distinguished manually by radiologists, but this may result in misdiagnosis due to human intervention. Therefore, in this study, a new tool has been developed that can automatically extract lobe regions using artificial intelligence-based instance-aware semantic lobe segmentation. Convolution neural networks (CNNs) offer automatic feature extraction in the instance-aware semantic lobe segmentation task that extracts the lobe regions on CT images. In this paper, CNN-based architectures, including DeepLabV3+ with VGG-16, VGG-19, and ResNet-50, were utilized to create a benchmark for the instance-aware semantic lobe segmentation task. For further improvement in segmentation results, images were preprocessed to detect the lung region prior to lobe segmentation. In the experimental evaluations, a large-scale dataset including 9036 images with pixel-level annotations for lung and lobe regions, has been created. DeepLabV3+ with ResNet-50 showed the highest performance in terms of dice similarity coefficient (DSC) and intersection over union (IOU) for lobe segmentation at 99.59 % and 99.19 %, respectively. The experiments demonstrated that our approach outperformed several state-of-the-art methods for the instance-aware semantic lobe segmentation task. Furthermore, a new desktop application called LobeChestApp was developed for the segmentation of lobe regions on chest CT images.

Keywords

• Artificial Intelligence
• Deep Learning
• Instance-Aware Semantic Lobe Segmentation
• Convolutional Neural Network.

Kaynakça

1. Abdulkareem, K. H., Mohammed, M. A., Salim, A., Arif, M., Geman, O., Gupta, D., & Khanna, A. (2021). Realizing an effective COVID-19 diagnosis system based on machine learning and IOT in smart hospital environment. IEEE Internet of Things Journal, 8(21), 15919-15928.
2. Akosman, Ş. A., Öktem, M., Moral, Ö. T., & Kılıç, V. (2021). Deep Learning-based Semantic Segmentation for Crack Detection on Marbles. Paper presented at the 2021 29th Signal Processing and Communications Applications Conference (SIU).
3. Aydın, S., Çaylı, Ö., Kılıç, V., & Onan, A. (2022). Sequence-to-Sequence Video Captioning with Residual Connected Gated Recurrent Units. J Avrupa Bilim ve Teknoloji Dergisi(35), 380-386.
4. Çaylı, Ö., Kılıç, V., Onan, A., & Wang, W. (2022). Auxiliary Classifier based Residual RNN for Image Captioning. Paper presented at the 2022 30th European Signal Processing Conference (EUSIPCO).
5. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2017). Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. J IEEE transactions on pattern analysis, 40(4), 834-848.
6. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. Paper presented at the Proceedings of the European conference on computer vision (ECCV).
7. Chen, X., Zhang, R., & Yan, P. (2019). Feature fusion encoder decoder network for automatic liver lesion segmentation. Paper presented at the 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019).
8. Cruz, A. A. (2007). Global surveillance, prevention and control of chronic respiratory diseases: a comprehensive approach: World Health Organization.

9. Das, S., Fime, A. A., Siddique, N., & Hashem, M. J. (2021). Estimation of road boundary for intelligent vehicles based on deepLabV3+ architecture. IEEE Access, 9, 121060-121075.
10. Davis, S. D., Brody, A. S., Emond, M. J., Brumback, L. C., & Rosenfeld, M. J. (2007). Endpoints for clinical trials in young children with cystic fibrosis. Proceedings of the American Thoracic Society, 4(4), 418-430.
11. Davis, S. D., Fordham, L. A., Brody, A. S., Noah, T. L., Retsch-Bogart, G. Z., Qaqish, B. F., . . . Leigh, M. W. (2007). Computed tomography reflects lower airway inflammation and tracks changes in early cystic fibrosis. J American journal of respiratory critical care medicine, 175(9), 943-950.
12. Doǧan, V., Isık, T., Kılıç, V., & Horzum, N. J. (2022). A field-deployable water quality monitoring with machine learning-based smartphone colorimetry. Analytical Methods, 14(35), 3458-3466.
13. Doğan, V., & Kılıç, V. (2021). Akıllı Telefon Kullanarak Yapay Zeka Tabanlı Farenjit Tespiti: Artificial Intelligence Based Pharyngitis Detection Using Smartphone. J Sağlık Bilimlerinde Yapay Zeka Dergisi ISSN: -, 1(2), 14-19.
14. Ferreira, F. T., Sousa, P., Galdran, A., Sousa, M. R., & Campilho, A. (2018). End-to-end supervised lung lobe segmentation. Paper presented at the 2018 International Joint Conference on Neural Networks (IJCNN).
15. Fetiler, B., Çaylı, Ö., Moral, Ö. T., Kılıç, V., & Onan, A. (2021). Video captioning based on multi-layer gated recurrent unit for smartphones. J Avrupa Bilim ve Teknoloji Dergisi(32), 221-226.
16. Geng, L., Zhang, S., Tong, J., & Xiao, Z. (2019). Lung segmentation method with dilated convolution based on VGG-16 network. J Computer Assisted Surgery, 24(sup2), 27-33.
17. Giri, B., Pandey, S., Shrestha, R., Pokharel, K., Ligler, F. S., & Neupane, B. B. J. A. (2021). Review of analytical performance of COVID-19 detection methods. Analytical bioanalytical chemistry, 413(1), 35-48.
18. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition.
19. He, K., Zhao, W., Xie, X., Ji, W., Liu, M., Tang, Z., Liu, J. (2021a). Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of COVID-19 in CT images. J Pattern recognition, 113, 107828.
20. He, K., Zhao, W., Xie, X., Ji, W., Liu, M., Tang, Z., . . . Liu, J. J. P. r. (2021b). Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of COVID-19 in CT images. 113, 107828.
21. Hofmanninger, J., Prayer, F., Pan, J., Röhrich, S., Prosch, H., & Langs, G. (2020). Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem. J European Radiology Experimental, 4(1), 1-13.
22. Imran, A.-A.-Z., Hatamizadeh, A., Ananth, S. P., Ding, X., Tajbakhsh, N., Terzopoulos, D. J. C. M. i. B., Visualization. (2020). Fast and automatic segmentation of pulmonary lobes from chest CT using a progressive dense V-network. 8(5), 509-518.
23. Keskin, R., Çaylı, Ö., Moral, Ö. T., Kılıç, V., & Onan, A. (2021). A benchmark for feature-injection architectures in image captioning. J Avrupa Bilim ve Teknoloji Dergisi(31), 461-468.
24. Keskin, R., Moral, Ö. T., Kılıç, V., & Onan, A. (2021). Multi-GRU based automated image captioning for smartphones. Paper presented at the 2021 29th Signal Processing and Communications Applications Conference (SIU).
25. Kilic, B., Dogan, V., Kilic, V., & Kahyaoglu, L. N. J. (2022). Colorimetric food spoilage monitoring with carbon dot and UV light reinforced fish gelatin films using a smartphone application. International Journal of Biological Macromoleculesnternational Journal of Biological Macromolecules, 209, 1562-1572.
26. Kılıç, V. (2021). Deep gated recurrent unit for smartphone-based image captioning. J Sakarya University Journal of Computer Information Sciences, 4(2), 181-191.
27. Liu, H., & Lang, B. J. (2019). Machine learning and deep learning methods for intrusion detection systems: A survey. applied sciences, 9(20), 4396.
28. Mercan, Ö. B., & Kılıç, V. (2020). Deep Learning based Colorimetric Classification of Glucose with Au-Ag nanoparticles using Smartphone. Paper presented at the 2020 Medical Technologies Congress (TIPTEKNO).
29. Mercan, Ö. B., Kılıç, V., & Şen, M. (2021). Machine learning-based colorimetric determination of glucose in artificial saliva with different reagents using a smartphone coupled μPAD. J Sensors Actuators B: Chemical, 329, 129037.
30. Müller, N. J. A. (1991). Clinical value of high-resolution CT in chronic diffuse lung disease. American journal of roentgenology, 157(6), 1163-1170.
31. Palaz, Z., Doğan, V., & Kılıç, V. (2021). Smartphone-based Multi-parametric Glucose Prediction using Recurrent Neural Networks. J Avrupa Bilim ve Teknoloji Dergisi(32), 1168-1174.
32. Palsson, B., Sveinsson, J. R., & Ulfarsson, M. O. J. (2022). Blind hyperspectral unmixing using autoencoders: A critical comparison. IEEE Journal of Selected Topics in Applied Earth Observations, 15, 1340-1372.
33. Sajid, N. J. M. (2020). Covid-19 patients lungs x ray images 10000.
34. Şen, M., Yüzer, E., Doğan, V., Avcı, İ., Ensarioğlu, K., Aykaç, A., . . . Kılıç, V. J. (2022). Colorimetric detection of H2O2 with Fe3O4@ Chi nanozyme modified µPADs using artificial intelligence. Microchimica Acta, 189(10), 1-11.
35. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. J arXiv preprint arXiv:.05645.
36. Simpson, S., Kay, F. U., Abbara, S., Bhalla, S., Chung, J. H., Chung, M., . . . Ko, J. P. J. (2020). Radiological Society of North America expert consensus statement on reporting chest CT findings related to COVID-19. Endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA. Journal of thoracic imaging.
37. Soomro, T. A., Zheng, L., Afifi, A. J., Ali, A., Yin, M., & Gao, J. J. (2022). Artificial intelligence (AI) for medical imaging to combat coronavirus disease (COVID-19): A detailed review with direction for future research. Artificial Intelligence Review, 55(2), 1409-1439.
38. Suzuki, K. (2017). Overview of deep learning in medical imaging. J Radiological physics technology, 10(3), 257-273.
39. Wang, W., Xu, Y., Gao, R., Lu, R., Han, K., Wu, G., & Tan, W. J. J. (2020). Detection of SARS-CoV-2 in different types of clinical specimens. 323(18), 1843-1844.
40. Xie, W., Jacobs, C., Charbonnier, J.-P., & Van Ginneken, B. J. I. t. o. m. i. (2020). Relational modeling for robust and efficient pulmonary lobe segmentation in CT scans. 39(8), 2664-2675.
41. Yang, X., He, X., Zhao, J., Zhang, Y., Zhang, S., & Xie, P. J. (2020a). COVID-CT-dataset: a CT scan dataset about COVID-19. arXiv preprint arXiv:.13865.
42. Yang, X., He, X., Zhao, J., Zhang, Y., Zhang, S., & Xie, P. J. a. p. a. (2020b). COVID-CT-dataset: a CT scan dataset about COVID-19.
43. Yüzer, E., Doğan, V., Kılıç, V., & Şen, M. J. (2022). Smartphone embedded deep learning approach for highly accurate and automated colorimetric lactate analysis in sweat. Sensors Actuators B: Chemical, 371, 132489.