Effect of Hybrid Activation and Loss Functions for Pneumonia Classification in Chest X-ray Images

Abstract

Pneumonia is one of the major infectious diseases leading to death worldwide and its early detection is crucial for successful treatment. Chest X-ray images are a frequently used method for the detection of pneumonia and often contain complex structures to make an accurate diagnosis. In this study, deep learning based models are used to classify normal and pneumonia labeled data in Chest X-ray images. As a result of the comparisons made on MobileNetV2, ResNet50, VGG19, Xception and ViT models, the VGG19 model achieved the highest success with an accuracy of 88.14%. In addition, the proposed hybrid activation function integrated into the VGG19 model performed the best with 91.67% accuracy and improved the classification success. Performance evaluations with the integration of different loss functions (MSE, MAE, Binary Cross-Entropy and the proposed loss function) also revealed that the Proposed Hybrid loss function achieved the highest performance with 92.63% accuracy. These findings show that hybrid activation and loss functions significantly improve classification accuracy in deep learning-based medical imaging applications.

Keywords

• Chest x-ray
• Deep learning
• Hybrid activation function
• Loss function
• Pneumonia

Ethical Statement

All data used in this study are taken from a publicly available and freely accessible dataset. Therefore, there is no requirement for an ethical declaration. The dataset is obtained from open sources provided for research purposes and does not contain any personal data.

Chest X-ray Görüntülerinde Pnömoni Sınıflandırması İçin Hibrit Aktivasyon ve Kayıp Fonksiyonlarının Etkisi

Abstract

Pnömoni, dünya çapında ölüme yol açan önemli enfeksiyon hastalıklarından biridir ve erken teşhis edilmesi, tedavi süreçlerinin başarısı için büyük önem taşımaktadır. Chest X-ray görüntüleri, pnömoninin tespiti için sıklıkla kullanılan bir yöntem olup, doğru teşhis koymak için genellikle karmaşık yapılar içerir. Bu çalışmada, Chest X-ray görüntülerindeki normal ve pnömoni etiketli verilerin sınıflandırılması amacıyla derin öğrenme tabanlı modeller kullanılmıştır. MobileNetV2, ResNet50, VGG19, Xception ve ViT modelleri üzerinde yapılan karşılaştırmalar sonucunda, VGG19 modeli %88.14 doğruluk oranı ile en yüksek başarıyı elde etmiştir. Ayrıca, VGG19 modeline entegre edilen önerilen hibrit aktivasyon fonksiyonu, %91.67 doğruluk ile en iyi performansı sergileyerek sınıflandırma başarısını artırmıştır. Farklı kayıp fonksiyonlarının (MSE, MAE, Binary Cross-Entropy ve önerilen kayıp fonksiyonu) entegrasyonu ile yapılan performans değerlendirmeleri de, Proposed Hybrid kayıp fonksiyonunun %92.63 doğruluk oranı ile en yüksek başarıyı sağladığını ortaya koymuştur. Bu bulgular, hibrit aktivasyon ve kayıp fonksiyonlarının derin öğrenme tabanlı tıbbi görüntüleme uygulamalarında sınıflandırma doğruluğunu önemli ölçüde iyileştirdiğini göstermektedir.

Keywords

• Göğüs röntgeni
• Derin öğrenme
• Hibrit aktivasyon fonksiyonu
• Kayıp fonksiyonu
• Zatürre

Ethical Statement

Bu çalışmada kullanılan tüm veriler, halka açık ve ücretsiz erişime sahip bir veri setinden alınmıştır. Dolayısıyla, etik beyan gereksinimi bulunmamaktadır. Veri seti, araştırma amacıyla sağlanan açık kaynaklardan temin edilmiştir ve herhangi bir kişisel veri içermemektedir.

References

1. [1] Ruuskanen, O., Lahti, E., Jennings, L. C., & Murdoch, D. R. (2011). Viral pneumonia. The Lancet, 377(9773), 1264-1275.
2. [2] Hammoudi, K., Benhabiles, H., Melkemi, M., Dornaika, F., Arganda-Carreras, I., Collard, D., & Scherpereel, A. (2021). Deep learning on chest X-ray images to detect and evaluate pneumonia cases at the era of COVID-19. Journal of medical systems, 45(7), 75.
3. [3] MacMahon, H. (2003). Digital chest radiography: practical issues. Journal of thoracic imaging, 18(3), 138-147.
4. [4] Rajpurkar, P. (2017). CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning. ArXiv abs/1711, 5225.
5. [5] Pan, S. J., & Yang, Q. (2009). A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10), 1345-1359.
6. [6] Khan, E., Rehman, M. Z. U., Ahmed, F., Alfouzan, F. A., Alzahrani, N. M., & Ahmad, J. (2022). Chest X-ray classification for the detection of COVID-19 using deep learning techniques. Sensors, 22(3), 1211.
7. [7] Shelke, A., Inamdar, M., Shah, V., Tiwari, A., Hussain, A., Chafekar, T., & Mehendale, N. (2021). Chest X-ray classification using deep learning for automated COVID-19 screening. SN computer science, 2(4), 300.
8. [8] Ibrahim, A. U., Ozsoz, M., Serte, S., Al-Turjman, F., & Yakoi, P. S. (2024). Pneumonia classification using deep learning from chest X-ray images during COVID-19. Cognitive computation, 16(4), 1589-1601.

9. [9] Abbas, A., Abdelsamea, M. M., & Gaber, M. M. (2021). Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. Applied Intelligence, 51, 854-864.
10. [10] Giełczyk, A., Marciniak, A., Tarczewska, M., & Lutowski, Z. (2022). Pre-processing methods in chest X-ray image classification. Plos one, 17(4), e0265949.
11. [11] Alshmrani, G. M. M., Ni, Q., Jiang, R., Pervaiz, H., & Elshennawy, N. M. (2023). A deep learning architecture for multi-class lung diseases classification using chest X-ray (CXR) images. Alexandria Engineering Journal, 64, 923-935.
12. [12] Asif, S., Wenhui, Y., Jin, H., & Jinhai, S. (2020, December). Classification of COVID-19 from chest X-ray images using deep convolutional neural network. In 2020 IEEE 6th international conference on computer and communications (ICCC) (pp. 426-433). IEEE.
13. [13] Zhao, J., Li, M., Shi, W., Miao, Y., Jiang, Z., & Ji, B. (2021, April). A deep learning method for classification of chest X-ray images. In Journal of Physics: Conference Series (Vol. 1848, No. 1, p. 012030). IOP Publishing.
14. [14] Okolo, G. I., Katsigiannis, S., & Ramzan, N. (2022). IEViT: An enhanced vision transformer architecture for chest X-ray image classification. Computer Methods and Programs in Biomedicine, 226, 107141.
15. [15] Kolonne, S., Fernando, C., Kumarasinghe, H., & Meedeniya, D. (2021, December). MobileNetV2 based chest X-rays classification. In 2021 International Conference on Decision Aid Sciences and Application (DASA) (pp. 57-61). IEEE.
16. [16] Souid, A., Sakli, N., & Sakli, H. (2021). Classification and predictions of lung diseases from chest x-rays using mobilenet v2. Applied Sciences, 11(6), 2751.
17. [17] Kaggle, Available at: https://www.kaggle.com/datasets/alifrahman/chestxraydataset/data, Accessed: 02 December 2024.
18. [18] Dosovitskiy, A., Springenberg, J. T., Riedmiller, M., & Brox, T. (2014). Discriminative unsupervised feature learning with convolutional neural networks. Advances in neural information processing systems, 27.
19. [19] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4510-4520).
20. [20] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
21. [21] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
22. [22] Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251-1258).
23. [23] Vaswani, A. (2017). Attention is all you need. Advances in Neural Information Processing Systems.
24. [24] Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2021, July). Training data-efficient image transformers & distillation through attention. In International conference on machine learning (pp. 10347-10357). PMLR.
25. [25] Akalın, F. (2024). Survival Classification in Heart Failure Patients by Neural Network-Based Crocodile and Egyptian Plover (CEP) Optimization Algorithm. Arabian Journal for Science and Engineering, 49(3), 3897-3914.
26. [26] Nair, V., & Hinton, G. E. (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10) (pp. 807-814).
27. [27] Hochreiter, S. (1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(02), 107-116.
28. [28] Sun, K., Yu, J., Zhang, L., & Dong, Z. (2020). A convolutional neural network model based on improved softplus activation function. In International Conference on Applications and Techniques in Cyber Intelligence ATCI 2019: Applications and Techniques in Cyber Intelligence 7 (pp. 1326-1335). Springer International Publishing.
29. [29] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
30. [30] Mao, A., Mohri, M., & Zhong, Y. (2023, July). Cross-entropy loss functions: Theoretical analysis and applications. In International conference on Machine learning (pp. 23803-23828). PMLR.
31. [31] Hadsell, R., Chopra, S., & LeCun, Y. (2006, June). Dimensionality reduction by learning an invariant mapping. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06) (Vol. 2, pp. 1735-1742). IEEE.
32. [32] Akalın, F. (2024). Değiştirilmiş Yapay Arı Kolonisi Optimizasyon Algoritması ve İstatistiksel Modelleme ile Sentetik Veri Üretimi. Journal of the Institute of Science & Technology/Iğdır Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 14(4).
33. [33] Ibrahim, A. U., Ozsoz, M., Serte, S., Al-Turjman, F., & Yakoi, P. S. (2024). Pneumonia classification using deep learning from chest X-ray images during COVID-19. Cognitive computation, 16(4), 1589-1601.
34. [34] Saraiva, A. A., Santos, D. B. S., Costa, N. J. C., Sousa, J. V. M., Ferreira, N. M. F., Valente, A., & Soares, S. (2019, February). Models of learning to classify X-ray images for the detection of pneumonia using neural networks. In Bioimaging (pp. 76-83).
35. [35] Gülgün, O. D., & Erol, H. (2020). Classification performance comparisons of deep learning models in pneumonia diagnosis using chest x-ray images. Turkish Journal of Engineering, 4(3), 129-141.