Training of Multilayer Artificial Neural Network with Snow Ablation Optimizer Algorithm

Year 2024, Volume: 39 Issue: 2, 391 - 406, 11.07.2024

Feyza Nur Özdemir , Ahmet Özkış

https://doi.org/10.21605/cukurovaumfd.1514409

Abstract

The snow ablation optimizer (SAO) is a new metaheuristic algorithm proposed in 2023, inspired by snow melting. In this study, a hybrid model was developed with the SAO algorithm to update the artificial neural network weights. The developed hybrid model was compared with hybrid models created with gray wolf, reptile search, cuckoo and sine cosine algorithms on five different data sets named aggregation, balance, liver, pathbased and wine. Four different metrics called sensitivity, specificity, precision and f1-score were used to measure the success of the models. The success ranking of the models for each data set and the average success ranking for all data sets are given. When the results are examined, it is seen that the SAO model ranks 2nd in the wine data set and 1st in all other data sets for all metrics. Regarding average success rank, the SAO model achieved the best result with a value of 1.2 in all metrics. In addition, convergence graphs of the mean square error values of the hybrid models in the training phase were drawn and it was observed that the SAO hybrid model had a faster convergence performance than the compared models in all other data sets except Wine. Finally, the effect of the number of particles in the population on the success of the hybrid SAO model was analyzed and it was observed that the success increased when the number of individuals was 100.

Keywords

Snow ablation optimizer algorithm, Artificial neural networks, Machine learning, Supervised learning, Classification

Kar Erime Optimizasyonu Algoritması ile Çok Katmanlı Yapay Sinir Ağının Eğitimi

Abstract

Kar erime optimizasyonu (snow ablation optimizer, SAO) algoritması, karın erimesinden ilham alınarak 2023 yılında önerilen yeni bir metasezgisel algoritmadır. Bu çalışmada, yapay sinir ağının ağırlıklarının güncellenmesi amacıyla SAO algoritması ile hibrit bir model geliştirilmiştir. Geliştirilen hibrit model aggregation, balance, liver, pathbased ve wine adlı beş farklı veri seti üzerinde gri kurt, sürüngen arama, guguk kuşu ve sinüs kosinüs algoritmaları ile oluşturulan hibrit modeller ile karşılaştırılmıştır. Modellerin başarısını ölçmek için duyarlılık, özgüllük, kesinlik ve f1-puanı adı verilen dört farklı metrik kullanılmıştır. Modellerin her veri seti için başarı sıralaması ve tüm veri setleri için ortalama başarı sıralaması verilmiştir. Sonuçlar incelendiğinde, SAO modelinin tüm metrikler için wine veri setinde 2., diğer tüm veri setlerinde 1. olduğu görülmektedir. Ortalama başarı sırası açısından ise SAO modeli tüm metriklerde 1.2 değeri ile en iyi sonucu elde etmiştir. Ayrıca hibrit modellerin, eğitim aşamasındaki ortalama karesel hata değerlerinin yakınsama grafikleri çizdirilmiş ve SAO hibrit modelinin wine hariç diğer tüm veri setlerinde karşılaştırılan modellerden daha hızlı bir yakınsama performansına sahip olduğu gözlenmiştir. Son olarak popülasyondaki parçacık sayısının hibrit SAO modelinin başarısına etkisi analiz edilmiş ve birey sayısının 100 olması durumunda başarının arttığı gözlenmiştir.

Keywords

Kar erime optimizasyonu algoritması, Yapay sinir ağları, Makine öğrenmesi, Danışmanlı öğrenme, Sınıflandırma

References

• Turing, A.M., 2009. Computing Machinery and Intelligence. Springer.
• 2. Öztemel, E., 2003. Yapay Sinir Ağlari. Papatya Yayincilik.
• 3. Rosenblatt, F., 1958. The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain. Psychological review, 65(6), 386.
• 4. Rumelhart, D.E., Hinton, G.E., Williams, R.J., 1986. Learning Representations by Back-Propagating Errors. Nature, 323(6088), 533-536.
• 5. Ciregan, D., Meier, U., Schmidhuber, J., 2012. Multi-Column Deep Neural Networks for Image Classification. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 3642-3649.
• 6. Krizhevsky, A., Sutskever, I., Hinton, G.E., 2012. Imagenet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, 25, 1-9.
• 7. Liu, X., Zeng, S., Namaiti, A., Xin, R., 2023. Comparison between Three Convolutional Neural Networks for Local Climate Zone Classification Using Google Earth Images: A Case Study of the Fujian Delta in China. Ecological Indicators, 148, 110086.
• 8. Bas, E., Egrioglu, E., Cansu, T., 2024. Robust Training of Median Dendritic Artificial Neural Networks for Time Series Forecasting. Expert Systems with Applications, 238, 122080.
• 9. Dalal, A.-A., AlRassas, A.M., Al-qaness, M.A., Cai, Z., Aseeri, A.O., Abd Elaziz, M., Ewees, A.A., 2023. Tlia: Time-Series Forecasting Model Using Long Short-Term Memory Integrated with Artificial Neural Networks for Volatile Energy Markets. Applied Energy, 343, 121230.
• 10. Egrioglu, E., Baş, E., Chen, M.-Y., 2022. Recurrent Dendritic Neuron Model Artificial Neural Network for Time Series Forecasting. Information Sciences, 607, 572-584.
• 11. Ergun, U., Tayfun, D., 2020. Jaya Algoritması Ile Optimize Edilmiş Yapay Sinir Ağlarını Kullanarak Türkiye’de Elektrik Enerjisi Tüketiminin Tahmini. Gazi University Journal of Science Part C: Design Technology, 8(3), 511-528.
• 12. Jayasimha, S., Lingaraju, K., Raju, H., 2022. Prediction of Surface Finish in Extrusion Honing Process by Regression Analysis and Artificial Neural Networks. Applications in Engineering Science, 10, 100105.
• 13. Selim, A., Shuvo, S.N.A., Moniruzzaman, M., Islam, M., Shah, S., Ohiduzzaman, M., 2023. Predictive Models for Dissolved Oxygen in an Urban Lake by Regression Analysis and Artificial Neural Network. 1-11.
• 14. Tam, V.W., Butera, A., Le, K.N., Da Silva, L.C., Evangelista, A.C., 2022. A Prediction Model for Compressive Strength of Co2 Concrete Using Regression Analysis and Artificial Neural Networks. Construction Building Materials, 324, 126689.
• 15. Ghiassi, M., Burnley, C., 2010. Measuring Effectiveness of a Dynamic Artificial Neural Network Algorithm for Classification Problems. Expert Systems with Applications, 37(4), 3118-3128.
• 16. Xu, B., Su, J., Dale, D., Watson, M., 2000. Cotton Color Grading with a Neural Network. Textile Research Journal, 70(5), 430-436.
• 17. Yaman, S., Karakaya, B., Köküm, M., 2024. A Neural Network Approach for Classification of Fault-Slip Data in Geoscience. Ain Shams Engineering Journal, 15(1), 102325.
• 18. Elangasinghe, M., Singhal, N., Dirks, K., Salmond, J., Samarasinghe, S., 2014. Complex Time Series Analysis of Pm10 and Pm2. 5 for a Coastal Site Using Artificial Neural Network Modelling and K-Means Clustering. Atmospheric Environment, 94, 106-116.
• 19. Erilli, N.A., Yolcu, U., Eğrioğlu, E., Aladağ, Ç.H., Öner, Y., 2011. Determining the Most Proper Number of Cluster in Fuzzy Clustering by Using Artificial Neural Networks. Expert Systems with Applications, 38(3), 2248-2252.
• 20. Nithya, A., Appathurai, A., Venkatadri, N., Ramji, D., Palagan, C.A., 2020. Kidney Disease Detection and Segmentation Using Artificial Neural Network and Multi-Kernel K-Means Clustering for Ultrasound Images. Measurement, 149, 106952.
• 21. Jiadong, Q., Ohl, J.P., Tran, T.-T., 2024. Predicting Clay Compressibility for Foundation Design with High Reliability and Safety: A Geotechnical Engineering Perspective Using Artificial Neural Network and Five Metaheuristic Algorithms. Reliability Engineering System Safety, 243, 109827.
• 22. Alameer, Z., Abd Elaziz, M., Ewees, A.A., Ye, H., Jianhua, Z., 2019. Forecasting Gold Price Fluctuations Using Improved Multilayer Perceptron Neural Network and Whale Optimization Algorithm. Resources Policy, 61, 250-260.
• 23. Du, W., Zhang, Q., Chen, Y., Ye, Z., 2021. An Urban Short-Term Traffic Flow Prediction Model Based on Wavelet Neural Network with Improved Whale Optimization Algorithm. Sustainable Cities Society, 69, 102858.
• 24. Ouladbrahim, A., Belaidi, I., Khatir, S., Magagnini, E., Capozucca, R., Wahab, M.A., 2022. Experimental Crack Identification of Api X70 Steel Pipeline Using Improved Artificial Neural Networks Based on Whale Optimization Algorithm. Mechanics of Materials, 166, 104200.
• 25. Deng, L., Liu, S., 2023. Snow Ablation Optimizer: A Novel Metaheuristic Technique for Numerical Optimization and Engineering Design. Expert Systems with Applications, 225, 120069.
• 26. Karakoyun, M., Özkış, A., 2021. Transfer Fonksiyonları Kullanarak İkili Güve-Alev Optimizasyonu Algoritmalarının Geliştirilmesi ve Performanslarının Karşılaştırılması. Necmettin Erbakan Üniversitesi Fen ve Mühendislik Bilimleri Dergisi, 3(2), 1-10.
• 27. Özkış, A., Karakoyun, M., 2023. A Binary Enhanced Moth Flame Optimization Algorithm for Uncapacitated Facility Location Problems. Pamukkale University Journal of Engineering Sciences, 29(7), 737-751.
• 28. Çelik, İ., Yıldız, C., Şekkeli, M., 2018. Rüzgâr Enerji Santrali Kurulumunda Rüzgâr Türbinlerinin Mikro Yerleşimi Için Bir Optimizasyon Modeli. Gazi University Journal of Science Part C: Design Technology, 6(4), 898-908.
• 29. Kong, M., Tian, P., Kao, Y., 2008. A New Ant Colony Optimization Algorithm for the Multidimensional Knapsack Problem. Computers Operations Research, 35(8), 2672-2683.
• 30. Irmak, B., 2022. Yapay Sinir Ağlarının Eğitimi Için Kelebek Optimizasyonu Algoritmasının Iyileştirilmesi. Yüksek Lisans Tezi, Necmettin Erbakan Üniversitesi, Fen Bilimleri Enstitüsü, Konya, Türkiye, 75.
• 31. Wolpert, D.H., Macready, W.G., 1997. No Free Lunch Theorems for Optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67-82.
• 32. Singh, P., Pandey, V.K., Chakraborty, S., Dash, K.K., Singh, R., Béla, K., 2023. Ultrasound-Assisted Extraction of Phytochemicals from Green Coconut Shell: Optimization by Integrated Artificial Neural Network and Particle Swarm Technique. Heliyon, 9(12).
• 33. Bendine, K., Pereira, J.L.J., Gomes, G.F., 2023. Energy Harvesting Enhancement of Nonuniform Functionally Graded Piezoelectric Beam Using Artificial Neural Networks and Lichtenberg Algorithm. Structures, 105271.
• 34. Yang, S., Tian, X., Zhang, Q., Jiang, J., Dong, P., Tan, J., Meng, Y., Liu, P., Bai, H., Song, J., 2023. Microorganism Inspired Hydrogels: Optimization by Response Surface Methodology and Genetic Algorithm Based on Artificial Neural Network. European Polymer Journal, 201, 112497.
• 35. Wang, C., He, Q., Li, Z., Yu, J., Bello, I.T., Zheng, K., Han, M., Ni, M., 2024. A Novel in-Tube Reformer for Solid Oxide Fuel Cell for Performance Improvement and Efficient Thermal Management: A Numerical Study Based on Artificial Neural Network and Genetic Algorithm. Applied Energy, 357, 122030.
• 36. Cinar, A.C., Natarajan, N., 2022. An Artificial Neural Network Optimized by Grey Wolf Optimizer for Prediction of Hourly Wind Speed in Tamil Nadu, India. Intelligent Systems with Applications, 16, 200138.
• 37. Bernard, J., Popescu, E., Graf, S., 2022. Improving Online Education through Automatic Learning Style Identification Using a Multi-Step Architecture with Ant Colony System and Artificial Neural Networks. Applied Soft Computing, 131, 109779.
• 38. Zhang, H., Nguyen, H., Bui, X.-N., Nguyen-Thoi, T., Bui, T.-T., Nguyen, N., Vu, D.-A., Mahesh, V., Moayedi, H., 2020. Developing a Novel Artificial Intelligence Model to Estimate the Capital Cost of Mining Projects Using Deep Neural Network-Based Ant Colony Optimization Algorithm. Resources Policy, 66, 101604.
• 39. Mirjalili, S., 2015. How Effective Is the Grey Wolf Optimizer in Training Multi-Layer Perceptrons. Applied Intelligence, 43, 150-161.
• 40. Gülcü, Ş., 2022. Training of the Feed Forward Artificial Neural Networks Using Dragonfly Algorithm. Applied Soft Computing, 124, 109023.
• 41. Turkoglu, B., Kaya, E., 2020. Training Multi-Layer Perceptron with Artificial Algae Algorithm. Engineering Science Technology, an International Journal, 23(6), 1342-1350.
• 42. Qaddoura, R., Faris, H., Aljarah, I., Castillo, P.A., 2021. Evocluster: An Open-Source Nature-Inspired Optimization Clustering Framework. SN Computer Science, 2, 1-12.
• 43. Ataseven, B., 2013. Yapay Sinir Ağlari Ile Öngörü Modellemesi. Öneri Dergisi, 10(39), 101-115.
• 44. Yang, X.-S., Deb, S., 2009. Cuckoo Search Via Lévy Flights. 2009 World Congress on Nature & Biologically inspired computing (NaBIC), Coimbatore, India, 210-214.
• 45. Pu, Y., Song, J., Wu, M., Xu, X., Wu, W., 2023. Node Location Using Cuckoo Search Algorithm with Grouping and Drift Strategy for Wsn. Physical Communication, 59, 102088.
• 46. Mirjalili, S., Mirjalili, S.M., Lewis, A., 2014. Grey Wolf Optimizer. Advances in Engineering Software, 69, 46-61.
• 47. Abualigah, L., Abd Elaziz, M., Sumari, P., Geem, Z.W., Gandomi, A.H., 2022. Reptile Search Algorithm (RSA): A Nature-Inspired Meta-Heuristic Optimizer. Expert Systems with Applications, 191, 116158.
• 48. Mirjalili, S., 2016. Sca: A Sine Cosine Algorithm for Solving Optimization Problems. Knowledge-Based Systems, 96, 120-133.