KONTROLLÜ DENGESİZLİK SENARYOLARINDA TOPLULUK ÖĞRENME MODELLERİN SİSTEMATİK KARŞILAŞTIRMASI

Abstract

Bu çalışma, sınıf dengesizliğinin topluluk öğrenme algoritmaları üzerindeki etkisini kontrollü bir deneysel tasarım ile incelemeyi amaçlamaktadır. Çalışma kapsamında, Iris ve Wine veri setleri üzerinde dört farklı sınıf dağılımı senaryosu (orijinal, hafif, orta ve şiddetli dengesizlik) uygulanmış ve her senaryoda Random Forest, Gradient Boosting ve Bagging algoritmaları test edilmiştir. Değerlendirmelerde yalnızca doğruluk değil, aynı zamanda Macro-F1, Balanced Accuracy, G-Mean ve Cohen Kappa gibi çoklu performans metrikleri kullanılmıştır. Elde edilen bulgular, Gradient Boosting modelinin yüksek dengesizlik düzeylerinde ciddi performans kayıpları yaşadığını; buna karşılık Random Forest algoritmasının tüm senaryolarda kararlı ve güvenilir sonuçlar sunduğunu ortaya koymuştur. Bu yönüyle çalışma, sınıf dengesizliğine karşı dayanıklı model seçiminin ve çok boyutlu metriklerle yapılan değerlendirmelerin önemini vurgulamaktadır.

Keywords

• Sınıf dengesizliği
• topluluk öğrenme
• Random Forest
• performans metrikleri
• dengesiz veri setleri

A SYSTEMATIC COMPARISON OF ENSEMBLE MODELS UNDER CONTROLLED CLASS IMBALANCE SCENARIOS

Abstract

This study aims to investigate the impact of class imbalance on ensemble learning algorithms through a controlled experimental design. Four different class distribution scenarios (original, mild, moderate, and severe imbalance) were applied to the Iris and Wine datasets, and three ensemble models Random Forest, Gradient Boosting, and Bagging were evaluated in each scenario. Model performance was assessed using not only accuracy but also multiple metrics such as Macro-F1, Balanced Accuracy, G-Mean, and Cohen’s Kappa. The findings reveal that Gradient Boosting suffered significant performance degradation under high imbalance conditions, while Random Forest consistently delivered stable and reliable results across all scenarios. These results highlight the importance of selecting imbalance-resilient models and conducting evaluations using multiple performance indicators in imbalanced classification tasks.

Keywords

• Class imbalance
• ensemble learning
• Random Forest
• performance metrics
• imbalanced datasets

References

1. Ö. ÇELİK, A Research on Machine Learning Methods and Its Applications, Journal of Educational Technology and Online Learning 1 (2018) 25–40. https://doi.org/10.31681/jetol.457046.
2. S.B. Kotsiantis, I.D. Zaharakis, P.E. Pintelas, Machine learning: a review of classification and combining techniques, Artif Intell Rev 26 (2006) 159–190. https://doi.org/10.1007/s10462-007-9052-3.
3. S. Yadav, G.P. Bhole, Handling Imbalanced Dataset Classification in Machine Learning, in: 2020 IEEE Pune Section International Conference (PuneCon), IEEE, 2020: pp. 38–43. https://doi.org/10.1109/PuneCon50868.2020.9362471.
4. H. Kaur, H.S. Pannu, A.K. Malhi, A Systematic Review on Imbalanced Data Challenges in Machine Learning, ACM Comput Surv 52 (2020) 1–36. https://doi.org/10.1145/3343440.
5. Z.-H. Zhou, Ensemble Learning, in: Mach Learn, Springer Singapore, Singapore, 2021: pp. 181–210. https://doi.org/10.1007/978-981-15-1967-3_8.
6. J. Beemer, K. Spoon, L. He, J. Fan, R.A. Levine, Ensemble Learning for Estimating Individualized Treatment Effects in Student Success Studies, Int J Artif Intell Educ 28 (2018) 315–335. https://doi.org/10.1007/s40593-017-0148-x.
7. A. Mohammed, R. Kora, A comprehensive review on ensemble deep learning: Opportunities and challenges, Journal of King Saud University - Computer and Information Sciences 35 (2023) 757–774. https://doi.org/10.1016/j.jksuci.2023.01.014.
8. O. Saidani, M. Umer, A. Alshardan, N. Alturki, M. Nappi, I. Ashraf, Student academic success prediction in multimedia-supported virtual learning system using ensemble learning approach, Multimed Tools Appl 83 (2024) 87553–87578. https://doi.org/10.1007/s11042-024-18669-z.

9. M.R. Khalilpour Darzi, S.T.A. Niaki, M. Khedmati, Binary classification of imbalanced datasets: The case of CoIL challenge 2000, Expert Syst Appl 128 (2019) 169–186. https://doi.org/10.1016/j.eswa.2019.03.024.
10. W. Lee, K. Seo, Downsampling for Binary Classification with a Highly Imbalanced Dataset Using Active Learning, Big Data Research 28 (2022) 100314. https://doi.org/10.1016/j.bdr.2022.100314.
11. V. Kumar, G.S. Lalotra, P. Sasikala, D.S. Rajput, R. Kaluri, K. Lakshmanna, M. Shorfuzzaman, A. Alsufyani, M. Uddin, Addressing Binary Classification over Class Imbalanced Clinical Datasets Using Computationally Intelligent Techniques, Healthcare 10 (2022) 1293. https://doi.org/10.3390/healthcare10071293.
12. S. Cateni, V. Colla, M. Vannucci, A method for resampling imbalanced datasets in binary classification tasks for real-world problems, Neurocomputing 135 (2014) 32–41. https://doi.org/10.1016/j.neucom.2013.05.059.
13. P. Manikanta, D. Jayaprakash, D. Sharma, R. Bathla, Wine Quality Prediction Using Machine Learning Techniques, in: 2024 2nd International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT), IEEE, 2024: pp. 1015–1018. https://doi.org/10.1109/ICAICCIT64383.2024.10912091.
14. M. De Marsico, M. Nappi, D. Riccio, H. Wechsler, Mobile Iris Challenge Evaluation (MICHE)-I, biometric iris dataset and protocols, Pattern Recognit Lett 57 (2015) 17–23. https://doi.org/10.1016/j.patrec.2015.02.009.
15. G. Biau, E. Scornet, A random forest guided tour, TEST 25 (2016) 197–227. https://doi.org/10.1007/s11749-016-0481-7.
16. A. Parmar, R. Katariya, V. Patel, A Review on Random Forest: An Ensemble Classifier, in: 2019: pp. 758–763. https://doi.org/10.1007/978-3-030-03146-6_86.
17. Y. Zhang, A. Haghani, A gradient boosting method to improve travel time prediction, Transp Res Part C Emerg Technol 58 (2015) 308–324. https://doi.org/10.1016/j.trc.2015.02.019.
18. C. Bentéjac, A. Csörgő, G. Martínez-Muñoz, A comparative analysis of gradient boosting algorithms, Artif Intell Rev 54 (2021) 1937–1967. https://doi.org/10.1007/s10462-020-09896-5.
19. J.G. Dias, J.K. Vermunt, A bootstrap-based aggregate classifier for model-based clustering, Comput Stat 23 (2008) 643–659. https://doi.org/10.1007/s00180-007-0103-7.
20. T. Hothorn, B. Lausen, Double-bagging: combining classifiers by bootstrap aggregation, Pattern Recognit 36 (2003) 1303–1309. https://doi.org/10.1016/S0031-3203(02)00169-3.
21. C.W. Fisher, E.J.M. Lauria, C.C. Matheus, An Accuracy Metric, Journal of Data and Information Quality 1 (2009) 1–21. https://doi.org/10.1145/1659225.1659229.
22. M.C. Hinojosa Lee, J. Braet, J. Springael, Performance Metrics for Multilabel Emotion Classification: Comparing Micro, Macro, and Weighted F1-Scores, Applied Sciences 14 (2024) 9863. https://doi.org/10.3390/app14219863.
23. V. García, R.A. Mollineda, J.S. Sánchez, Index of Balanced Accuracy: A Performance Measure for Skewed Class Distributions, in: 2009: pp. 441–448. https://doi.org/10.1007/978-3-642-02172-5_57.
24. H.R. Sofaer, J.A. Hoeting, C.S. Jarnevich, The area under the precision‐recall curve as a performance metric for rare binary events, Methods Ecol Evol 10 (2019) 565–577. https://doi.org/10.1111/2041-210X.13140.
25. H. Guo, H. Liu, C. Wu, W. Zhi, Y. Xiao, W. She, Logistic discrimination based on G-mean and F-measure for imbalanced problem, Journal of Intelligent & Fuzzy Systems 31 (2016) 1155–1166. https://doi.org/10.3233/IFS-162150.
26. A. Figueroa, S. Ghosh, C. Aragon, Generalized Cohen’s Kappa: A Novel Inter-rater Reliability Metric for Non-mutually Exclusive Categories, in: 2023: pp. 19–34. https://doi.org/10.1007/978-3-031-35132-7_2.
27. A. Aggarwal, Z. Xu, O. Feyisetan, N. Teissier, On Log-Loss Scores and (No) Privacy, in: Proceedings of the Second Workshop on Privacy in NLP, Association for Computational Linguistics, Stroudsburg, PA, USA, 2020: pp. 1–6. https://doi.org/10.18653/v1/2020.privatenlp-1.1.
28. R. Singh, N.S. Mangat, Stratified Sampling, in: 1996: pp. 102–144. https://doi.org/10.1007/978-94-017-1404-4_5.
29. M. Moeini, Hyperparameter tuning of supervised bagging ensemble machine learning model using Bayesian optimization for estimating stormwater quality, Sustain Water Resour Manag 10 (2024) 83. https://doi.org/10.1007/s40899-024-01064-9.
30. R.I. Alkanhel, E.-S.M. El-Kenawy, M.M. Eid, L. Abualigah, M.A. Saeed, Optimizing IoT-driven smart grid stability prediction with dipper throated optimization algorithm for gradient boosting hyperparameters, Energy Reports 12 (2024) 305–320. https://doi.org/10.1016/j.egyr.2024.06.034.
31. Y. Rimal, N. Sharma, A. Alsadoon, The accuracy of machine learning models relies on hyperparameter tuning: student result classification using random forest, randomized search, grid search, bayesian, genetic, and optuna algorithms, Multimed Tools Appl 83 (2024) 74349–74364. https://doi.org/10.1007/s11042-024-18426-2.
32. O. Kramer, Scikit-Learn, in: 2016: pp. 45–53. https://doi.org/10.1007/978-3-319-33383-0_5.
33. H. Bin Mehare, J.P. Anilkumar, N.A. Usmani, The Python Programming Language, in: A Guide to Applied Machine Learning for Biologists, Springer International Publishing, Cham, 2023: pp. 27–60. https://doi.org/10.1007/978-3-031-22206-1_2.