Optimization of hyper parameters of artificial neural networks for prediction of elastic modulus of normal and high strength concrete

Abstract

In this study, while modeling the concrete elasticity modulus with Artificial Neural Networks (ANN), the optimal determination of the parameters of ANNs was carried out with the help of meta-heuristic algorithms. The hyperparameters of ANNs are the number of hidden layers, the number of neurons in hidden layers, and the activation functions in hidden layers. ANNs have been successfully used in classification and regression problems. But determining hyperparameters is a time-consuming process. Therefore, in this study, hyperparameters were determined using meta-heuristic algorithms. Whale Optimization Algorithm, Ant Lion Optimizer and Particle Swarm Optimization algorithms were used because they are successful in solving many engineering problems. The elastic modulus of normal and high strength concrete was estimated using ANN, whose hyperparameters were determined. The results obtained were compared with previous studies in literature. The proposed method outperformed the previous methods by showing better or equal results in most experiments. In the training process, for high strength concrete, it was more successful in 44.9%, equal in 34.8% and less successful in 20.3%. Overall, it performed equal to or better than the previous methods in 79.7% of the training process and 76.4% in the testing process. For normal strength concrete, the proposed method performed better or equal in 59.6% of the training process and 69.2% of the testing process, proving its effectiveness in both cases. As a result, better modeling results were obtained than in previous studies. As a result of modeling with different datasets, the R^2\ value was found to be the highest 0.98. It has been shown that better results can be obtained from ANN used without tuning the hyperparameter.

Keywords

• Artificial neural network
• Elastic modulus
• Optimization
• Meta-Heuristic

Normal ve yüksek dayanımlı betonların elastisite modülünün tahmini için yapay sinir ağlarının hiper parametrelerinin optimizasyonu

Öz

Bu çalışmada, beton elastisite modülü Yapay Sinir Ağları (YSA) kullanılarak modellenmiştir. YSA'ın yapısal parametrelerinin optimum olarak belirlenmesi ise meta-sezgisel algoritmalar yardımıyla gerçekleştirilmiştir. YSA'ların hiper parametreleri; gizli katman sayısı, gizli katmanlardaki nöron sayıları ve gizli katmanlarda kullanılan aktivasyon fonksiyonlarıdır. YSA, sınıflandırma ve regresyon problemlerinde başarılı sonuçlar elde edebilen bir yöntemdir. Ancak hiper parametrelerinin belirlenmesi zaman alıcıdır. Bu nedenle bu çalışmada meta-sezgisel algoritmalar kullanılarak hiper parametreler belirlenmiştir. Birçok mühendislik probleminin çözümünde Balina Optimizasyon Algoritması, Karınca Aslanı Optimizasyonu ve Parçacık Sürü Optimizasyon algoritmaları başarılı sonuçlar elde edebildikleri için bu çalışmada tercih edilmişlerdir. Normal ve yüksek dayanımlı betonların elastisite modülü, hiper parametreleri belirlenen YSA kullanılarak tahmin edilmiş ve elde edilen sonuçlar literatürdeki önceki çalışmalarla karşılaştırılmıştır. Önerilen yöntem, çoğu deneyde daha iyi veya eşit sonuçlar göstererek önceki yöntemlerden daha iyi performans göstermiştir. Eğitim aşamasında, yüksek dayanımlı beton için %44.9'nda daha başarılı, %34.8'nde aynı başarıyı göstermiş ve %20,3'nde ise daha az başarılı olmuştur. Genel olarak, eğitim aşamasının %79.7'sinde önceki yöntemlere eşit veya daha iyi, test aşamasında ise %76.4 başarı göstermiştir. Normal dayanımlı beton için amaçlanan yöntem, eğitim aşamasının %59.6'sında ve test aşamasının %69,2'sinde daha iyi veya aynı performansı göstermiş ve her iki durumda da etkinliğini kanıtlamıştır. Sonuç olarak önceki çalışmalara göre daha iyi modelleme sonuçları elde edilmiştir. Farklı veri kümeleri ile yapılan modelleme sonucunda R^2 değeri en yüksek 0.98 olarak bulunmuştur. Hiper parametreler bulunmadan kullanılan YSA'dan daha iyi sonuçlar elde edilebileceği gösterilmiştir.

Anahtar Kelimeler

• Yapay sinir ağı
• Elastisite modülü
• Optimizasyon
• Meta-Sezgisel

References

1. [1] Demir F. “Prediction of elastic modulus of normal and high strength concrete by artificial neural networks”. Construction and building Materials, 22(7), 1428-1435, 2008.
2. [2] Farooq F, Nasir Amin M, Khan K, Rehan Sadiq M, Javed M F, Aslam F, Alyousef R. “A comparative study of random forest and genetic engineering programming for the prediction of compressive strength of high strength concrete (HSC)”. Applied Sciences, 10(20), 7330, 2020.
3. [3] Abellán García J, Fernandez Gomez J, Torres Castellanos N. “Properties prediction of environmentally friendly ultra-high-performance concrete using artificial neural networks,” European Journal of Environmental and Civil Engineering, 26(6), 2319-2343, 2022.
4. [4] Abdulla NA. “Using the artificial neural network to predict the axial strength and strain of concrete-filled plastic tube”. Journal of Soft Computing in Civil Engineering, 4(2), 63-84, 2020.
5. [5] Al-Gburi SN, Akpinar P, Helwan A. “Machine learning in concrete's strength prediction”. Computers and Concrete, 29(6), 433, 2022.
6. [6] Zeng Z, Zhu Z, Yao W, Wang Z, Wang C, Wei Y, Wei Z, Guan X. “Accurate prediction of concrete compressive strength based on explainable features using deep learning”. Construction and Building Materials, 329, 127082, 2022.
7. [7] BKA MAR, Ngamkhanong C, Wu Y, Kaewunruen S. “Recycled aggregates concrete compressive strength prediction using artificial neural networks (ANNs)”. Infrastructures, 6, 1-20, 2021.
8. [8] Khotimah BK, Agustina F, Puspitarini OR, Husni H, Anamisa DR, Prayugo N, Putri AMS. “Random search hyperparameter optimization for BPNN to forecasting cattle population”. 1st TMIC 2023, Madura Island, Indonesia, 29 November 2023.

9. [9] Du J, Ma H, Sun D, Pan P. “Data driven strength and strain enhancement model for FRP confined concrete using Bayesian optimization”. Structures, 41, 1345-1358, 2022.
10. [10] Han J, Gondro C, Reid K, Steibel JP. “Heuristic hyperparameter optimization of deep learning models for genomic prediction”. G3, 11(7), jkab032, 2021.
11. [11] Liu Z, Gu X, Yang H, Wang L, Chen Y, Wang D. “Novel YOLOv3 model with structure and hyperparameter optimization for detection of pavement concealed cracks in GPR images”. IEEE Transactions on Intelligent Transportation Systems, 23(11), 22258-22268, 2022.
12. [12] Newcomer MW, Hunt RJ. “NWTOPT–A hyperparameter optimization approach for selection of environmental model solver settings”. Environmental Modelling Software, 147, 105250, 2022.
13. [13] Peng Y, Gong D, Deng C, Li H, Cai H, Zhang H. “An automatic hyperparameter optimization DNN model for precipitation prediction”. Applied Intelligence, 52(3), 2703-2719, 2022.
14. [14] Qu Z, Xu J, Wang Z, Chi R, Liu H. “Prediction of electricity generation from a combined cycle power plant based on a stacking ensemble and its hyperparameter optimization with a grid-search method”. Energy, 227, 120309, 2021.
15. [15] Şenel B, Şenel FA. “Novel neural network optimization approach for modeling scattering and noise parameters of microwave transistor”. International Journal of Numerical Modelling: Electronic Networks, Devices and Fields, 35(1), e2930, 2022.
16. [16] Tsai CW, Fang ZY. “An effective hyperparameter optimization algorithm for DNN to predict passengers at a metro station”. ACM Transactions on Internet Technology (TOIT), 21(2), 1-24, 2021.
17. [17] Wei Y, Chen Z, Zhao C, Tu Y, Chen X, Yang R. “A BiLSTM hybrid model for ship roll multi-step forecasting based on decomposition and hyperparameter optimization”. Ocean Engineering, 242, 110138, 2021.
18. [18] Aydın ZE, Erdem B İ, Cıcek ZIE. “Prediction bike-sharing demand with gradient boosting methods”. Pamukkale Üniversitesi Mühendislik Bilimleri Dergisi, 29(8), 824-832, 2023.
19. [19] Haseli B, Nouri G, Mardi M, Adili E, Bahari M. “Prediction of strength parameters of concrete containing different additives using optimized neural network algorithm”. Numerical Methods in Civil Engineering, 7(4), 12-23, 2023.
20. [20] Franchini G. “GreenNAS: A green approach to the hyperparameters tuning in deep learning”. Mathematics, 12(6), 850, 2024.
21. [21] Mirjalili S, Lewis A. “The whale optimization algorithm”. Advances in Engineering Software, 95, 51-67, 2016.
22. [22] Mirjalili S. “The ant lion optimizer”. Advances in Engineering Software, 83, 80-98, 2015.
23. [23] Kennedy J, Eberhart R. “Particle swarm optimization”. In Proceedings of ICNN'95-International Conference on Neural Networks, Perth, Australia, 27 November-1 December 1995.
24. [24] Gesoǧlu M, Güneyisi E, Özturan T. “Effects of end conditions on compressive strength and static elastic modulus of very high strength concrete”. Cement and Concrete Research, 32(10), 1545-1550, 2002.
25. [25] Ozturan T. An İnvestigation of Concrete Abrasion as two Phase Material. Ph D Thesis, Faculty of Civil Engineering, Istanbul Technical University, Istanbul, Türkiye, 1984.
26. [26] Turan M, Iren M. “Strain stress relationship of concrete”. Journal of Engineering and Architecture, 12(1), 76-81, 1997.
27. [27] Wee TH, Chin M S, Mansur MA. “Stress-strain relationship of high-strength concrete in compression”. Journal of Materials in Civil Engineering, 8(2), 70-76, 1996.
28. [28] Şenel FA, Gökçe F, Yüksel AS, Yiğit T. “A novel hybrid PSO-GWO algorithm for optimization problems”. Engineering with Computers, 35(4), 1359-1373, 2019.