ANALYSIS OF DIFFERENT POOLING FUNCTIONS ON A CONVOLUTION NEURAL NETWORK BASED MODEL

Year 2024, , 266 - 276, 30.08.2024

Halit Çetiner , Sedat Metlek

https://doi.org/10.46519/ij3dptdi.1484354

Abstract

The common denominator of deep learning models used in many different fields today is the pooling functions used in their internal architecture. These functions not only directly affect the performance of the study, but also directly affect the training time. For this reason, it is extremely important to measure the performance of different pooling functions and share their success values. In this study, the performances of commonly used soft pooling, max pooling, spatial pyramid pooling and average pooling functions were measured on a dataset used as benchmarking in the literature. For this purpose, a new CNN based architecture was developed. Accuracy, F1 score, precision, recall and categorical cross entropy metrics used in many studies in the literature were used to measure the performance of the developed architecture. As a result of the performance metrics obtained, 97.79, 92.50, 91.60 and 89.09 values from best to worst for accuracy were obtained from soft pooling, max pooling, spatial pyramid pooling and average pooling functions, respectively. In the light of these results, the pooling functions used in this study have provided a better conceptual and comparative understanding of the impact of a CNN-based model.

Keywords

Pooling, Artificial Intelligence, Convolution Neural Network, Classification.

References

• 1. Çetiner, H. and Metlek, S., “DenseUNet+: A novel hybrid segmentation approach based on multi-modality images for brain tumor segmentation,”, J. King Saud Univ. - Comput. Inf. Sci., Vol. 35, Issue 8, Pages 101663, 2023.
• 2. Metlek, S., “CellSegUNet: an improved deep segmentation model for the cell segmentation based on UNet++ and residual UNet models,”, Neural Comput. Appl., Vol. 36, Issue 11, Pages 5799–5825, 2024.
• 3. Çetiner, H., “Citrus disease detection and classification using based on convolution deep neural network,”, Microprocess. Microsyst., Vol. 95, Issue 104687, Pages 1–10, 2022.
• 4. Nirthika, R., Manivannan, S., Ramanan, A., and Wang, R., “Pooling in convolutional neural networks for medical image analysis: a survey and an empirical study,”, Neural Comput. Appl., Vol. 34, Issue 7, Pages 5321–5347, 2022.
• 5. Jena, B., Saxena, S., Nayak, G. K., Saba, L., Sharma, N., and Suri, J. S., “Artificial intelligence-based hybrid deep learning models for image classification: The first narrative review,”, Comput. Biol. Med., Vol. 137, Pages 104803, 2021.
• 6. Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., and Sanner, S., “Online continual learning in image classification: An empirical survey,”, Neurocomputing, Vol. 469, Pages. 28–51, 2022.
• 7. Schmarje, L., Santarossa, M., Schröder, S.-M., and Koch, R., “A survey on semi-, self-and unsupervised learning for image classification,”, IEEE Access, Vol. 9, Pages. 82146–82168, 2021.
• 8. Zafar, A., Aamir, M., Nawi, N. M., Arshad, A., Riaz, S., Alruban, A., Dutta, A. K., and Almotairi, S. “A Comparison of Pooling Methods for Convolutional Neural Networks,”, Applied Sciences, Vol. 12, Issue 17, 2022.
• 9. Zhao, R., Song, W., Zhang, W., Xing, T., Lin, J, and Srivastava, M., “Accelerating binarized convolutional neural networks with software-programmable FPGAs,”, in Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Pages 15–24, 2017.
• 10. Yildirim, O., Baloglu, U. B., Tan, R.-S., Ciaccio, E. J. and Acharya, U. R., “A new approach for arrhythmia classification using deep coded features and LSTM networks,”, Comput. Methods Programs Biomed., Vol. 176, Pages 121–133, 2019.
• 11. Cai, H., Gan, C., Wang, T., Zhang, Z., and Han, S., “Once-for-all: Train one network and specialize it for efficient deployment,”, arXiv Prepr. arXiv1908.09791, 2019.
• 12. Murray, N. and Perronnin, F., “Generalized max pooling,”, in Proceedings of the IEEE conference on computer vision and pattern recognition, Pages 2473–2480, 2014.
• 13. Roy, P., Ghosh, S., Bhattacharya, S., and Pal, U., “Effects of degradations on deep neural network architectures,” arXiv Prepr. arXiv1807.10108, 2018.
• 14. Özdemir, C., “Avg-topk: A new pooling method for convolutional neural networks,”, Expert Syst. Appl., Vol. 223, Pages 119892, 2023.
• 15. Dogo, E. M., Afolabi, O. J., Nwulu, N. I., Twala, B. and Aigbavboa, C. O., “A Comparative Analysis of Gradient Descent-Based Optimization Algorithms on Convolutional Neural Networks,”, in 2018 International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS), Pages 92–99, 2018.
• 16. Sikandar, S., Mahum, R. and Alsalman, A., “A Novel Hybrid Approach for a Content-Based Image Retrieval Using Feature Fusion,”, Applied Sciences, Vol. 13, Issue 7, 2023.
• 17. Prabavathi, M. V and Sakthi, M., “Poisson Wavelet Quantized Piecewise Regressive Distributed Coding for Image Compression and Transmission,”, Tuijin Jishu/Journal Propuls. Technol., Vol. 44, Issue 6, 2023.
• 18. Praveenkumar, G. D. and Nagaraj, R., “Regularized Anisotropic Filtered Tanimoto Indexive Deep Multilayer Perceptive Neural Network learning for effective image classification,”, Neurosci. Informatics, Vol. 2, Issue 2, Pages 100063, 2022.
• 19. Mohamed, E. A., Gaber, T., Karam, O. and Rashed, E. A., “A Novel CNN pooling layer for breast cancer segmentation and classification from thermograms,”, PLoS One, Vol. 17, Issue 10, Pages e0276523, 2022.
• 20. Vigneron, V., Maaref, H. and Syed, T. Q., “A New Pooling Approach Based on Zeckendorf’s Theorem for Texture Transfer Information,”, Entropy, Vol. 23, Issue 3. 2021.
• 21. Sharma, T., Verma, N. K. and Masood, S. “Mixed fuzzy pooling in convolutional neural networks for image classification,”, Multimed. Tools Appl., Vol. 82, Issue 6, Pages 8405–8421, 2023.
• 22. Bhattacharjee, K., Pant, M., Zhang, Y.-D. and Satapathy, S. C., “Multiple Instance Learning with Genetic Pooling for medical data analysis,”, Pattern Recognit. Lett., Vol. 133, Pages 247–255, 2020.
• 23. Boureau, Y.-L., Ponce, J., and LeCun, Y., “A theoretical analysis of feature pooling in visual recognition,”, in Proceedings of the 27th international conference on machine learning (ICML-10), Pages 111–118, 2010.
• 24. Singh, P., Chaudhury, S., and Panigrahi, B. K., “Hybrid MPSO-CNN: Multi-level Particle Swarm optimized hyperparameters of Convolutional Neural Network,”, Swarm Evol. Comput., Vol. 63, Pages 100863, 2021.
• 25. He, Z., Shao, H., Zhong, X., and Zhao, X., “Ensemble transfer CNNs driven by multi-channel signals for fault diagnosis of rotating machinery cross working conditions,”, Knowledge-Based Syst., Vol. 207, Pages 106396, 2020.
• 26. Riesenhuber, M. and Poggio, T., “Hierarchical models of object recognition in cortex,”, Nat. Neurosci., Vol. 2, Issue 11, Pages 1019–1025, 1999.
• 27. Stergiou, A., Poppe, R., and Kalliatakis, G., “Refining activation downsampling with SoftPool,”, in Proceedings of the IEEE/CVF international conference on computer vision, Pages 10357–10366, 2021.
• 28. Zeiler, M. D. and Fergus, R., “Visualizing and understanding convolutional networks,”, in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, Springer, Pages 818–833, 2014.
• 29. Girshick, R., Donahue, J., Darrell, T., and Malik, J., “Rich feature hierarchies for accurate object detection and semantic segmentation,”, in Proceedings of the IEEE conference on computer vision and pattern recognition, Pages 580–587, 2014.
• 30. Mumuni, A. and Mumuni, F., “CNN Architectures for Geometric Transformation-Invariant Feature Representation in Computer Vision: A Review,”, SN Comput. Sci., Vol. 2, Issue 5, Pages 340, 2021.
• 31. Cao, Z., Xu, X., Hu, B. and Zhou, M., “Rapid Detection of Blind Roads and Crosswalks by Using a Lightweight Semantic Segmentation Network,”, IEEE Trans. Intell. Transp. Syst., Vol. 22, Issue 10, Pages 6188–6197, 2021.
• 32. Ghosh, P., Azam, S., Jonkman, M., Karim, A., Shamrat, F. J. M., Ignatious, E., and De Boer, F. , “Efficient prediction of cardiovascular disease using machine learning algorithms with relief and LASSO feature selection techniques,”, IEEE Access, Vol. 9, Pages 19304–19326, 2021.
• 33. Gupta, A., Kumar, R., Arora, H. S., and Raman, B., “MIFH: A machine intelligence framework for heart disease diagnosis,”, IEEE access, Vol. 8, Pages 14659–14674, 2019.
• 34. Wang, L., Zhou, W., Chang, Q., Chen, J., and Zhou, X., “Deep ensemble detection of congestive heart failure using short-term RR intervals,”, IEEE Access, Vol. 7, Pages 69559–69574, 2019.
• 35. Miao, F., Cai, Y. P., Zhang, Y. X., Fan, X. M., and Li, Y., “Predictive modeling of hospital mortality for patients with heart failure by using an improved random survival forest,”, IEEE Access, Vol. 6, Pages 7244–7253, 2018.
• 36. Mohan, S., Thirumalai, C., and Srivastava, G., “Effective heart disease prediction using hybrid machine learning techniques,”, IEEE access, Vol. 7, Pages 81542–81554, 2019.
• 37. Çelebi, S. B. and Emiroğlu, B. G., “A Novel Deep Dense Block-Based Model for Detecting Alzheimer’s Disease,”, Applied Sciences, Vol. 13, Issue 15, 2023.
• 38. Zhang, W., Li, C., Peng, G., Chen, Y., and Zhang, Z., “A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load,”, Mech. Syst. Signal Process., Vol. 100, Pages 439–453, 2018.
• 39. Metlek, S. and Çetiner, H., “ResUNet+: A New Convolutional and Attention Block-Based Approach for Brain Tumor Segmentation,” IEEE Access, Vol. 11, Pages. 69884–69902, 2023.
• 40.Metlek, S. and Çetiner, H., “Inception SH: A New CNN Model Based on Inception Module for Classifying Scene Images,”, Mühendislik Bilim. ve Tasarım Dergisi, Vol. 12, Issue 2, Pages 328–344, 2024.