PERFORMANCE COMPARISON OF SMOTE-BASED MACHINE LEARNING MODELS ON UNBALANCED DATASETS: A STUDY ON DATE AND PISTACHIO FRUITS

Öz

Creating balanced datasets is a significant challenge that substantially affects the performance of machine learning models in the classification of agricultural products. In this research, we tried to overcome this challenge by using an unbalanced dataset containing information on 7 date palm (Phoenix dactylifera L.) and 2 pistachio (Pistacia vera L.) cultivars. The aim of the study is to compare the classification performance of machine learning models on an unbalanced dataset and a balanced dataset using the SMOTE technique. Initially, classification was performed on the unbalanced dataset using machine learning approaches. Among the machine learning models applied on the unbalanced dataset, the Linear-SVM model showed the highest accuracy rate with an accuracy rate of 92,62%. In the data set extended by applying the SMOTE technique, the RBF-SVM model again showed the highest accuracy rate with 95,55% accuracy rate. In summary, our study highlights the difficulties in machine learning-based agricultural crop classification due to data unbalances. Utilizing the SMOTE technique for oversampling was effective in overcoming this obstacle and improving classification accuracy.

Anahtar Kelimeler

• Machine learning
• SMOTE
• fruit classification
• oversampling

SMOTE TABANLI MAKİNE ÖĞRENMESİ MODELLERİNİN DENGESİZ VERİ KÜMELERİ ÜZERİNDE PERFORMANS KARŞILAŞTIRMASI: HURMA VE ANTEP FISTIĞI MEYVELERİ ÜZERİNE BİR ÇALIŞMA

Öz

Dengeli veri kümeleri oluşturmak, tarımsal ürünlerin sınıflandırılmasında makine öğrenimi modellerinin performansını önemli ölçüde etkileyen önemli bir zorluktur. Yapılan bu araştırmada, 7 hurma (Phoenix dactylifera L.) ve 2 Antep fıstığı (Pistacia vera L.) çeşidine ait bilgileri içeren dengesiz bir veri kümesi kullanarak bu zorluğun üstesinden gelinmeye çalışılmıştır. Çalışmanın ana hedefi, makine öğrenmesi modellerinin dengesiz veri kümesi ve SMOTE tekniği ile dengelenmiş veri kümesi üzerindeki sınıflandırma başarısını karşılaştırmaktır. Başlangıç olarak, dengesiz veri kümesi üzerinde makine öğrenimi yaklaşımları kullanılarak sınıflandırma yapılmıştır. Dengesiz veri kümesinde uygulanan makine öğrenmesi modelleri içerisinde %92,62 doğruluk oranı ile en yüksek doğruluk oranını Linear-SVM modeli göstermiştir. SMOTE tekniği uygulanarak genişletilen veri kümesinde ise %95,55 doğruluk oranı ile en yüksek doğruluk oranını RBF-SVM modeli göstermiştir. Özetle, çalışmamız makine öğrenimi tabanlı tarımsal ürün sınıflandırmasında veri dengesizliklerinden kaynaklanan zorlukların altını çizmektedir. Aşırı örnekleme için SMOTE tekniğinden yararlanmak, bu engelin üstesinden gelmede ve sınıflandırma doğruluğunu artırmada etkili olmuştur.

Anahtar Kelimeler

• Makine öğrenmesi
• SMOTE
• meyve sınıflandırma
• aşırı örnekleme

Kaynakça

1. Aggarwal, S., & Kaur, D. (2013). Naive Bayes Classifier with Various Smoothing Techniques for Text Documents. International Journal of Computer Trends and Technology (IJCTT, 4(4). http://ijcttjournal.org/Volume4/issue-4/IJCTT-V4I4P187.pdf
2. Anugerah Simanjuntak, Rosni Lumbantoruan, Kartika Sianipar, Rut Gultom, Mario Simaremare, Samuel Situmeang, & Erwin Panggabean. (2024). Research and Analysis of IndoBERT Hyperparameter Tuning in Fake News Detection. Jurnal Nasional Teknik Elektro Dan Teknologi Informasi, 13(1), 60–67. https://doi.org/10.22146/jnteti.v13i1.8532
3. Aydın, B. S. Y. (2018). Siirt Yöresi Fıstık Yetiştiricilerinin Sulama Eğilimlerinin Belirlenmesi. Süleyman Demirel Üniversitesi Ziraat Fakültesi Dergisi, 119–127.
4. Bakan, Z., & Kanbay, F. (2024). Makine Öğrenmesi Yöntemleri ile Eğitim Başarısına Etki Eden Faktörlerin Modellenmesi. İstanbul Ticaret Üniversitesi Fen Bilimleri Dergisi, 23(45), 27–41. https://doi.org/10.55071/ticaretfbd.1442084
5. Belete, D. M., & Huchaiah, M. D. (2022). Grid search in hyperparameter optimization of machine learning models for prediction of HIV/AIDS test results. International Journal of Computers and Applications, 44(9), 875–886. https://doi.org/10.1080/1206212X.2021.1974663
6. Bhardwaj, P., Tiwari, P., Olejar, K., Parr, W., & Kulasiri, D. (2022). A machine learning application in wine quality prediction. Machine Learning with Applications, 8, 100261. https://doi.org/10.1016/j.mlwa.2022.100261
7. Boyd, C., Brown, G. C., Kleinig, T. J., Mayer, W., Dawson, J., Jenkinson, M., & Bezak, E. (2024). Hyperparameter selection for dataset‐constrained semantic segmentation: Practical machine learning optimization. Journal of Applied Clinical Medical Physics, 25(12). https://doi.org/10.1002/acm2.14542
8. Breiman, L. (2001). Random Forests. Machine Learning, 45(5–32). https://doi.org/https://doi.org/10.1023/A:1010933404324

9. Çağlar, A., Tomar, O., Vatansever, H., & Ekmekçi, E. (2017). Antepfıstığı (Pistacia vera L.) ve İnsan Sağlığı Üzerine Etkileri. Akademik Gıda, 436–447. https://doi.org/10.24323/akademik-gida.370408
10. Çelik, G. K. Ö. (2020). Yeniden Örnekleme Teknikleri Kullanarak SMS Verisi Üzerinde Metin Sınıflandırma Çalışması. Erciyes Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 36(3).
11. Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. https://doi.org/10.1613/jair.953
12. Chemchem, A., Alin, F., & Krajecki, M. (2019). Combining SMOTE Sampling and Machine Learning for Forecasting Wheat Yields in France. 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), 9–14. https://doi.org/10.1109/AIKE.2019.00010
13. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. https://doi.org/10.1007/BF00994018
14. Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21–27. https://doi.org/10.1109/TIT.1967.1053964
15. Cox, D. R. (1958). The Regression Analysis of Binary Sequences. Journal of the Royal Statistical Society. Series B (Methodological), 20(2), 215–242.
16. Divakar, S., Bhattacharjee, A., & Priyadarshini, R. (2021). Smote-DL: A Deep Learning Based Plant Disease Detection Method. 2021 6th International Conference for Convergence in Technology (I2CT), 1–6. https://doi.org/10.1109/I2CT51068.2021.9417920
17. Echegaray, N., Gullón, B., Pateiro, M., Amarowicz, R., Misihairabgwi, J. M., & Lorenzo, J. M. (2023). Date Fruit and Its By-products as Promising Source of Bioactive Components: A Review. Food Reviews International, 39(3), 1411–1432. https://doi.org/10.1080/87559129.2021.1934003
18. Ha, T. M., & Bunke, H. (1997). Off-line, handwritten numeral recognition by perturbation method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(5), 535–539. https://doi.org/10.1109/34.589216
19. Koklu, M., Kursun, R., Taspinar, Y. S., & Cinar, I. (2021). Classification of Date Fruits into Genetic Varieties Using Image Analysis. Mathematical Problems in Engineering, 2021, 1–13. https://doi.org/10.1155/2021/4793293
20. Menéndez, M. L., Pardo, J. A., Pardo, L., & Pardo, M. C. (1997). The Jensen-Shannon divergence. Journal of the Franklin Institute, 334(2), 307–318. https://doi.org/10.1016/S0016-0032(96)00063-4
21. Özdemir, A., Polat, K., & Alhudhaif, A. (2021). Classification of imbalanced hyperspectral images using SMOTE-based deep learning methods. Expert Systems with Applications, 178, 114986. https://doi.org/10.1016/j.eswa.2021.114986
22. Prasetio, D., & Harlili, D. (2016). Predicting football match results with logistic regression. 2016 International Conference On Advanced Informatics: Concepts, Theory And Application (ICAICTA), 1–5. https://doi.org/10.1109/ICAICTA.2016.7803111
23. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1(1), 81–106. https://doi.org/10.1007/BF00116251
24. Quinlan, J. R. (1996). Learning decision tree classifiers. ACM Computing Surveys, 28(1), 71–72. https://doi.org/10.1145/234313.234346
25. Rıdvan Saraçoğlu İlker Ali Özkan, M. K. (2021). Classification of Pistachio Species Using Improved K-NN Classifier. Journal of Nutrition and Internal Medicine, 23(1). https://doi.org/https://doi.org/10.23751/pn.v23i2.9686
26. Sağlam, F., & Cengiz, M. A. (2022). A novel SMOTE-based resampling technique trough noise detection and the boosting procedure. Expert Systems with Applications, 200, 117023. https://doi.org/10.1016/j.eswa.2022.117023
27. Sher, M., Minallah, N., Ahmad, T., & Khan, W. (2023). Hyperparameters analysis of long short-term memory architecture for crop classification. International Journal of Electrical and Computer Engineering (IJECE), 13(4), 4661. https://doi.org/10.11591/ijece.v13i4.pp4661-4670
28. Shubhangi, D. C., & Pratibha, A. K. (2021). Asthma, Alzheimer’s and Dementia Disease Detection based on Voice Recognition using Multi-Layer Perceptron Algorithm. 2021 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES), 1–7. https://doi.org/10.1109/ICSES52305.2021.9633923
29. Townsend, J. T. (1971). Theoretical analysis of an alphabetic confusion matrix. Perception & Psychophysics, 9(1), 40–50. https://doi.org/10.3758/BF03213026
30. Umer, M., Sadiq, S., Missen, M. M. S., Hameed, Z., Aslam, Z., Siddique, M. A., & NAPPI, M. (2021). Scientific papers citation analysis using textual features and SMOTE resampling techniques. Pattern Recognition Letters, 150, 250–257. https://doi.org/10.1016/j.patrec.2021.07.009
31. Varshney, T., Chug, A., & Singh, A. P. (2021). Deep Learning Models for Prediction of Tomato Powdery Mildew Disease. 2021 8th International Conference on Signal Processing and Integrated Networks (SPIN), 1036–1041. https://doi.org/10.1109/SPIN52536.2021.9566132
32. Wang, X., Ren, H., Ren, J., Song, W., Qiao, Y., Ren, Z., Zhao, Y., Linghu, L., Cui, Y., Zhao, Z., Chen, L., & Qiu, L. (2023). Machine learning-enabled risk prediction of chronic obstructive pulmonary disease with unbalanced data. Computer Methods and Programs in Biomedicine, 230, 107340. https://doi.org/10.1016/j.cmpb.2023.107340
33. Xiao, T., Liu, H., & Cheng, Y. (2019). Corn Disease Identification Based on improved GBDT Method. 2019 6th International Conference on Information Science and Control Engineering (ICISCE), 215–219. https://doi.org/10.1109/ICISCE48695.2019.00051
34. Yavaş, M., Güran, A., & Uysal, M. (2020). Covid-19 Veri Kümesinin SMOTE Tabanlı Örnekleme Yöntemi Uygulanarak Sınıflandırılması. European Journal of Science and Technology, 258–264. https://doi.org/10.31590/ejosat.779952
35. Yıldız, M. S. N. (2019). Hurma Ağacının (Phoenix dactylifera L.) İklim ve Toprak İstekleri. International Journal of Engineering, Design and Technology, 1(2), 64–70.