Investigation of the effect of different performance metrics on evaporation modeling

Year 2023, , 472 - 486, 15.04.2023

Volkan Yılmaz

https://doi.org/10.28948/ngumuh.1206278

Abstract

In this study, monthly total open surface evaporation was modeled using Anamur meteorological station data with Artificial Bee Colony (ABC) optimization algorithm. Coefficient of Determination (R2), Nash–Sutcliffe Efficiency (NSE) coefficient, Mean Squared Error (MSE) and Percent Bias (PBIAS) metrics were used in the studies. With the help of these metrics, R2 and NSE maximization and MSE MSE/R2 MSE/+NSE and PBIAS minimization were applied and how the selected performance metrics affected the result was investigated. Among the objective functions used, it has been seen that the models obtained by R2 maximization have underestimation/overestimation problem and PBIAS minimization produces very unsuccessful models, and it has been concluded that the most successful models are obtained with MSE/+NSE objective function. The fact that the MSE/+NSE metric, which has not been applied before in the literature in this field, has been shown to achieve successful results is accepted as the main output of the current study and it is thought that this situation constitutes the innovative part of the study.

Keywords

Artificial bee colony, Evaporation, Optimization, Performance metrics

Farklı performans metriklerinin buharlaşma modellemesi üzerindeki etkisinin incelenmesi

Abstract

Bu çalışmada Yapay Arı Kolonisi (YAK) optimizasyon algoritması ile Anamur meteoroloji istasyonu verileri kullanarak aylık toplam açık yüzey buharlaşması modellenmiştir. Çalışmalarda Determinasyon Katsayısı (R2), Nash–Sutcliffe Etkinlik katsayısı (NSE), Ortalama Karesel Hata (Mean Squared Error, MSE) ve Yanlılık Yüzdesi (Percent Bias, PBIAS) metrikleri kullanılarak; R2 ve NSE maksimizasyonu ile MSE, MSE/R2, MSE/+NSE ve PBIAS minimizasyonu uygulanarak seçilen performans metriklerinin sonucu ne derecede etkilediği araştırılmıştır. Çalışmalar sonucunda öncelikle YAK algoritmasıyla başarılı buharlaşma modellerinin oluşturulabileceği görülmüş olup, seçilen performans metriklerinin sonucu önemli derecede etkilediği çıktısı elde edilmiştir. Kullanılan amaç fonksiyonları içerisinden R2 maksimizasyonu ile elde edilen modellerde düşük tahmin/yüksek tahmin probleminin meydana geldiği ve PBIAS minimizasyonunun ise oldukça başarısız modeller ürettiği görülmüş olup en başarılı modellerin MSE/+NSE amaç fonksiyonu ile elde edildiği sonucuna ulaşılmıştır. Bu alanda literatürde daha önce uygulanmamış olan MSE/+NSE metriğinin başarılı sonuçlar elde ettiğinin gösterilmiş olması mevcut çalışmanın ana çıktısı olarak kabul edilmekte ve bu durumun çalışmanın yenilikçi kısmını oluşturduğu düşünülmektedir.

Keywords

Buharlaşma, Optimizasyon, Performans metrikleri, Yapay arı kolonisi

References

• F. Ellsäßer, A. Röll, C. Stiegler, Hendrayanto and D. Hölscher, Introducing QWaterModel, a QGIS plugin for predicting evapotranspiration from land surface temperatures. Environmental Modelling & Software, 130,2020.https://doi.org/10.1016/j.envsoft.2020.104739.
• V. P. Singh and C.-Y. Xu, Evaluatıon and generalızatıon of 13 mass-transfer equatıons for determınıng free water evaporatıon. Hydrol Process, 11, 311–323, 1997. https://doi.org/ 10.1002/(SICI)1099-1085(19970315)11:3.
• S. N. Qasem, S. Samadianfard, S. Kheshtgar, S. Jarhan, O. Kisi, S. Shamshirband and K. W. Chau, Modeling Monthly Pan Evaporation Using Wavelet Support Vector Regression and Wavelet Artificial Neural Networks in Arid and Humid Climates. Engineering Applications of Computational Fluid Mechanics, 13, 177–187, 2019. https://doi.org/10.1080/ 19942060.2018.1564702.
• G. Tezel and M. Buyukyildiz, Monthly Evaporation Forecasting Using Artificial Neural Networks and Support Vector Machines. Theor Appl Climatol, 124, 69-80, 2016. https://doi.org/10.1007/s00704-015-1392-3.
• Ö. Terzi, Daily Pan Evaporation Estimation Using Gene Expression Programming and Adaptive Neural-Based Fuzzy Inference System. Neural Comput Appl, 23, 1035–1044, 2013. https://doi.org/10.1007/S00521-012-1027-X/FIGURES/9.
• J. E. Cahoon, T. A. Costello and J. A. Ferguson, Estimating Pan Evaporation Using Limited Meteorological Observations. Agric For Meteorol, 55, 191-190, 1991. https://doi.org/10.1016/0168-1923(91)90061-T.
• S. Kim, J. Shiri, O. Kisi and V. P. Singh, Estimating Daily Pan Evaporation Using Different Data-Driven Methods and Lag-Time Patterns. Water Resources Management, 27, 2267–2286, 2013. https://doi.org/ 10.1007/S11269-013-0287-2.
• P. B. Shirsath and A. K. Singh, A Comparative Study of Daily Pan Evaporation Estimation Using ANN, Regression and Climate Based Models. Water Resources Management, 24:8, 1571–1581, 2009. https://doi.org/10.1007/S11269-009-9514-2.
• G. F. Lin, H. Y. Lin and M. C. Wu, Development of a Support-Vector-Machine-Based Model for Daily Pan Evaporation Estimation. Hydrol Process, 27, 3115–3127, 2013. https://doi.org/10.1002/HYP.9428.
• R. M. Adnan, A. Malik, A. Kumar, K. S. Parmar and O. Kisi, Pan Evaporation Modeling by Three Different Neuro-Fuzzy Intelligent Systems Using Climatic Inputs. Arabian Journal of Geosciences, 12, 1-14, 2019. https://doi.org/10.1007/S12517-019-4781-6/TABLES/7.
• A. D. Kulkarni and G. S. Anaokar, Prediction of Evaporation Loss in Reservoir with Fuzzy Logic Approach. European Journal of Advances in Engineering and Technology, 3, 39-42, 2016. https://doi.org/10.13140/RG.2.2.24931.96804.
• A. Ashrafzadeh, M. A. Ghorbani, S. M. Biazar and Z. M. Yaseen, Evaporation Process Modelling over Northern Iran: Application of an Integrative Data-Intelligence Model with the Krill Herd Optimization Algorithm. Hydrological Sciences Journal, 64, 1843-1856, 2019. https://doi.org/10.1080/026 26667.2019.1676428.
• M. A. Ghorbani, R. Kazempour, K. W. Chau, S. Shamshirband and P. T. Ghazvinei, Forecasting Pan Evaporation with an Integrated Artificial Neural Network Quantum-Behaved Particle Swarm Optimization Model: A Case Study in Talesh, Northern Iran. Engineering Applications of Computational Fluid Mechanics, 12(1), 724-737, 2018. https://doi.org/10.1080/19942060.2018.1517052.
• A. R. Ghumman, Y. M. Ghazaw, A. Alodah, A.ur Rauf, M. Shafiquzzaman and H. Haider, Identification of Parameters of Evaporation Equations Using an Optimization Technique Based on Pan Evaporation. Water, 12, 2020. https://doi.org/10.3390/W12010228.
• S. Kim and H. S. Kim, Neural Networks and Genetic Algorithm Approach for Nonlinear Evaporation and Evapotranspiration Modeling. J Hydrol (Amst), 351, 299–317, 2008. https://doi.org/10.1016/ J.JHYDROL.2007.12.014.
• L. Wang, O. Kisi, M, Zounemat-Kermani and H. Li, Pan Evaporation Modeling Using Six Different Heuristic Computing Methods in Different Climates of China. J Hydrol (Amst), 544, 407–427, 2017.https://doi.org/10.1016/J.JHYDROL.2016.11.059.
• M. Bulmer, Principles of Statistics. Courier Corporation, 1979.
• B. G. Tabachnick, L. S. Fidell and J. B. Ullman, Using multivariate statistics (Seventh), 2019.
• D. Karaboga, An idea based on honey bee swarm for numerical optimization, Erciyes university, engineering faculty, computer engineering department. Technical report-tr06, 2005.
• D. Karaboga and B. Akay, A Comparative Study of Artificial Bee Colony Algorithm. Appl Math Comput, 213, 108-132, 2009. https://doi.org/10.1016/ J.AMC.2009.03.090.
• A. L. Bolaji, A. F. Bamigbola, L. B. Adewole, P. B. Shola, A. Afolorunso, A. A. Obayomi, D. R. Aremu and A. A. A. Almazroi, A Room-Oriented Artificial Bee Colony Algorithm for Optimizing the Patient Admission Scheduling Problem. Comput Biol Med, 148, 2022. https://doi.org/10.1016/ J.COMPBIOMED.2022.105850.
• Y. Cui, W. Hu and A. Rahmani, A Reinforcement Learning Based Artificial Bee Colony Algorithm with Application in Robot Path Planning. Expert Syst Appl, 203, 2022. https://doi.org/10.1016/ J.ESWA.2022.117389.
• V. Yilmaz, The Use of Band Similarity in Urban Water Demand Forecasting as a New Method. Water Supply, 22, 1004-1019, 2022. https://doi.org/10.2166/ WS.2021.221.
• M. W. Van Liew, T. L. Veith, D. D. Bosch and J. G. Arnold, Suitability of SWAT for the conservation effects assessment project: a comparison on USDA-ARS experimental watersheds, J. Hydrol. Eng., 12 (2), 173-189, 2007. https://doi.org/10.1061/(ASCE)1084-0699(2007)12:2(173).
• J. E. Nash and J. v. Sutcliffe, River Flow Forecasting through Conceptual Models Part I — A Discussion of Principles. J Hydrol (Amst), 10, 282-290, 1970. https://doi.org/10.1016/0022-1694(70)90255-6.
• D. N. Moriasi, J. G Arnold, M. W. Van Liew, R. L. Bingner, R. D. Harmel and T. L. Veith, Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations. Trans ASABE, 50, 885-900, 2007. https://doi.org/ 10.13031/2013.23153.
• H. V. Gupta, S. Sorooshian and P. O. Yapo, Status of automatic calibration for hydrologic models: Comparison with multilevel expert calibration. Journal of hydrologic engineering, 4(2), 135-143, 1999. https://doi.org/10.1061/(ASCE)10840699(1999)4:2(135).
• M. Buyukyildiz and S. Y. Kumcu, An Estimation of the Suspended Sediment Load Using Adaptive Network Based Fuzzy Inference System, Support Vector Machine and Artificial Neural Network Models. Water Resources Management, 31, 1343-1359, 2017. https://doi.org/10.1007/s11269-017-1581-1.
• M. K. Goyal, B. Bharti, J. Quilty, J. Adamowski and A. Pandey, Modeling of daily pan evaporation in sub tropical climates using ANN, LS-SVR, Fuzzy Logic, and ANFIS, Expert Systems with Applications, 41, 5267-5276, 2014. https://doi.org/10.1016/ J.ESWA.2014.02.047.
• A. Ashrafzadeh, A. Malik, V. Jothiprakash, M. A. Ghorbani and S. M. Biazar, Estimation of daily pan evaporation using neural networks and meta-heuristic approaches, ISH Journal of Hydraulic Engineering, 26:4, 421-429, 2020. https://doi.org/10.1080/ 09715010.2018.1498754.
• Z. M. Yaseen, A. M. Al-Juboori, U. Beyaztas, N. Al-Ansari, K. W. Chau, C. Qi, M. Ali, S. Q. Salih and S. Shahid, Prediction of evaporation in arid and semi-arid regions: a comparative study using different machine learning models. Engineering Applications of Computational Fluid Mechanics, 14(1), 70-89, 2020.https://doi.org/10.1080/19942060.2019.1680576.
• A. Malik, A. Kumar, S. Kim, M. H. Kashani, V. Karimi, A. Sharafati, M. A. Ghorbani, N. Al-Ansari, S. Q. Salih, Z. M. Yaseen and K. W. Chau, Modeling monthly pan evaporation process over the Indian central Himalayas: application of multiple learning artificial intelligence model. Engineering Applications of Computational Fluid Mechanics, 14(1), 323-338, 2020.https://doi.org/10.1080/19942060.2020.1715845.
• R. Moazenzadeh, B. Mohammadi, S. Shamshirband and K. W. Chau, Coupling a firefly algorithm with support vector regression to predict evaporation in northern Iran, Engineering Applications of Computational Fluid Mechanics, 12(1), 584-597, 2018.https://doi.org/10.1080/19942060.2018.1482476.
• M. Shimi, M. Najjarchi, K. Khalili, E. Hezavei and S. M. Mirhoseyni, Investigation of the accuracy of linear and nonlinear time series models in modeling and forecasting of pan evaporation in IRAN. Arabian Journal of Geosciences, 13, 1-16, 2020. https://doi.org/10.1007/s12517-019-5031-7.
• S. Yang, S. Liu, X, Li, Y. Zhong, X, He and C, Wu, The short-term forecasting of evaporation duct height (EDH) based on ARIMA model. Multimedia Tools and Applications, 76, 24903-24916, 2017. https://doi.org/10.1007/s11042-016-4143-2.
• L. Wu, G. Huang, J. Fan, X. Ma, H. Zhou and W. Zeng, Hybrid extreme learning machine with meta-heuristic algorithms for monthly pan evaporation prediction. Computers and electronics in agriculture, 168, 105-115, 2020. https://doi.org/10.1016/ j.compag.2019.105115.