Öğrenci Başarısının Sınıflandırılmasında Farklı Makine Öğrenmesi Modellerinin Karşılaştırılması

Abstract

Eğitim-öğretim sürecinde öğrenci başarısının değerlendirilmesi ve sınıflandırılması, öğretim yöntemlerinin etkinliğini belirlemek açısından kritiktir. Öğrencilerin başarı durumlarını "geçti", "kaldı", "başarılı" ve "başarısız" gibi kategorilere ayırmak hem öğrencilerin akademik gelişimini izlemek hem de öğretim stratejilerini iyileştirmek için önemli bilgiler sağlar. Makine Öğrenmesi (MÖ) modellerinin bu sınıflandırmalarda kullanılması, büyük veri setlerinde daha doğru ve objektif değerlendirmeler yapılmasını sağlar. Bu sebeple bu araştırma, MÖ modellerinin öğrenci başarısını sınıflandırmadaki doğruluğunu incelemeyi amaçlamaktadır. MÖ, eğitimde büyük ve karmaşık veri setlerini analiz ederek daha doğru ve objektif değerlendirmeler sağlar. Bu araştırmada, Naive Bayes (NB), Destek Vektör Makineleri (DVM) ve Rastgele Orman (RO) makine öğrenmesi modellerinin sınıflandırma doğrulukları incelenmiştir. Bu araştırma, öğrencilerin başarı durumunu farklı makine öğrenmesi algoritmalarıyla tahmin ederek elde edilen performans ölçütlerini karşılaştırmakta ve model sonuçlarını incelemektedir. Bu yönüyle araştırma, betimsel tarama türünde ve uygulamalı bir çalışmadır. Etnik köken, ebeveyn eğitim düzeyi ve matematik başarısı gibi değişkenlerden oluşan 1000 örneklem içeren bir veri seti kullanılmıştır. Analizler SPSS ve R programı ile gerçekleştirilmiştir. Araştırma bulguları, en yüksek sınıflandırma doğruluğuna sahip modelin RO modeli olduğunu göstermiştir. MÖ modellerinin eğitimde kullanılması öğrenci başarısını, başarısızlık risklerini, öğretim yöntem ve materyallerinin etkinliğini tahmin ederek eğitim kalitesinin iyileştirilmesine katkı sağlayabilir.

Keywords

• Eğitim
• Makine öğrenmesi
• Sınıflandırma doğruluğu
• Rastgele orman

A Comparative Analysis of Different Machine Learning Models for Classifying Student Achievement

Abstract

In the context of teaching and learning, evaluating and classifying student achievement is critical for determining the effectiveness of instructional methods. Categorizing students’ academic performance into groups such as “passed,” “failed,” “successful,” and “unsuccessful” provides valuable insights for tracking academic progress and improving instructional strategies. The use of Machine Learning (ML) models in such classifications enables more accurate and objective evaluations, particularly when dealing with large datasets. Therefore, this study aims to examine the accuracy of various ML models in classifying student performance. ML offers enhanced precision and objectivity by analyzing large and complex educational datasets. In this study, the classification accuracies of three machine learning algorithms—Naive Bayes (NB), Support Vector Machines (SVM), and Random Forest (RF)—were evaluated. The research compares the performance metrics of these models in predicting students' academic success and examines the results in detail. As such, the study adopts a descriptive survey design and has an applied nature. A dataset comprising 1,000 samples and variables such as ethnicity, parental education level, and mathematics achievement was used. The analyses were conducted using SPSS and R software. The findings reveal that the Random Forest model achieved the highest classification accuracy. The integration of ML models in education can contribute to improving educational quality by predicting student success, identifying risk of failure, and evaluating the effectiveness of instructional methods and materials.

Keywords

• Education
• Machine learning
• Classification accuracy
• Random forest
• Classification models

References

1. Alamri, L., S. Almuslim, R., S. Alotibi, M., K. Alkadi, D., Ullah Khan, I., & Aslam, N. (2020). Predicting Student Academic Performance using Support Vector Machine and Random Forest [Paper Presentation]. 3rd International Conference on Education Technology Management. London, United Kingdom.
2. Aydın, S. (2007). Veri madenciliği ve Anadolu Üniversitesi uzaktan eğitim sisteminde bir uygulama. (Publication No. 220873) [Yayınlanmamış Doktora tezi]. Anadolu Üniversitesi, Eskişehir.
3. Aydın, S. ve Özkul, AE (2015). Veri madenciliği ve anadolu üniversitesi açık öğretim uygulaması bir uygulama [A data mining application and Anadolu University open education system: A case study]. Eğitim ve Öğretim Araştırmaları Dergisi, 4 (3), 36-44. http://www.jret.org/FileUpload/ks281142/File/05a.sinan_aydin.pdf
4. Breiman, L. (2001). Random forests. Machine learning, 45, 5-32. http://dx.doi.org/10.1023/A:1010933404324.
5. Brownlee, J. (2016). Machine learning mastery with python. Machine Learning Mastery Pty Ltd, 527, 100-120.
6. Buda, M., Maki, A., & Mazurowski, M. A. (2018). A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, 106, 249-259. https://doi.org/10.1016/j.neunet.2018.07.011
7. Cristianini, N., & Shawe-Taylor, J. (2000). An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press.
8. Cruz-Jesus, F., Castelli, M., Oliveira, T., Mendes, R., Nunes, C., Sa-Velho, M., & Rosa-Louro, A. (2020). Using artificial intelligence methods to assess academic achievement in public high schools of a European Union country. Heliyon, 6(6). https://doi.org/10.1016/j.heliyon.2020.e04081

9. Cohen, J. (1988). Statistical power analysis fort he behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
10. Corno, L., & Snow, R. E. (1986). Adapting teaching to individual differences among learners. In Handbook of research on teaching (pp. 605-629). Chicago, IL: Rand McNally.
11. Cortez, P., & Silva, A. M. G. (2008). Using data mining to predict secondary school student performance. https://hdl.handle.net/1822/8024
12. Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: A Wiley-Interscience Publication.
13. Geurts, P., Irrthum, A., & Wehenkel, L. (2009). Supervised Learning with Decision Tree-Based Methods in Computational and Systems Biology. Molecular BioSystems, 5, 1593-1605.
14. Gunawan, G., Hanes, H., & Catherine, C. (2021). C4. 5, K-nearest neighbor, naïve bayes, and random forest algorithms comparison to predict students' on time graduation. Indonesian Journal of Artificial Intelligence and Data Mining, 4(2), 62-71.
15. Golino, H. F., Gomes, C. M. A., & Andrade, D. (2014). Predicting academic achievement of high-school students using machine learning. Psychology, 5(18), 2046-2057. 10.4236/psych.2014.518207
16. Gray, G., McGuinness, C., & Owende, P. (2014). An application of classification models to predict learner progression in tertiary education. In 2014 IEEE international advance computing conference (IACC) (pp. 549-554). IEEE.
17. Gültepe, Y. (2019). Makine öğrenmesi algoritmaları ile hava kirliliği tahmini üzerine karşılaştırmalı bir değerlendirme [A comparative evaluation on air pollution forecasting with machine learning algorithms]. Avrupa Bilim ve Teknoloji Dergisi, (16), 8-15. https://doi.org/10.31590/ejosat.530347.
18. Hämäläinen, W., & Vinni, M. (2011). Classifiers for educational data mining. Handbook of Educational Data Mining, Chapman & Hall/CRC Data Mining and Knowledge Discovery Series, 57-71.
19. Hart, S. A. (2016). Precision education initiative: Moving toward personalized education. Mind Brain and Education, 10(4), 209–211. https://doi.org/10.1111/mbe.12109.
20. Hastie, T., Tibshirani, R., Friedman, J. H., & Friedman, J. H. (2009). The elements of statistical learning: data mining, inference, and prediction (Vol. 2, pp. 1-758). New York: springer.
21. Huang, T. M., Kecman, V., & Kopriva, I. (2006). Kernel based algorithms for mining huge data sets: Supervised, semi-supervised, and unsupervised learning. Heidelberg: Springer.
22. Hussain, S., Dahan, N. A., Ba-Alwib, F. M., & Ribata, N. (2018). Educational data mining and analysis of students’ academic performance using WEKA. Indonesian Journal of Electrical Engineering and Computer Science, 9(2), 447-459. http://doi.org/10.11591/ijeecs.v9.i2.pp447-459.
23. IBM. (2024, June 15). Makine Öğrenmesi. https://www.ibm.com/docs/tr/rpa/21.0?topic=classification-text-algorithms.
24. Iwendi, C., Bashir, A. K., Peshkar, A., Sujatha, R., Chatterjee, J. M., Pasupuleti, S., Mishra, R., Pillai, S., Jo, O. (2020). COVID-19 patient health prediction using boosted random forest algorithm. Frontiers in public health, 8, 357. https://doi.org/10.3389/fpubh.2020.00357.
25. Jayaprakash, S., Krishnan, S., & Jaiganesh, V. (2020). Predicting students academic performance using an improved random forest classifier [Paper Presentation]. International Conference On Emerging Smart Computing and Informatics (ESCI), 238-243, IEEE. Pune, India
26. Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415.
27. Karasar, N. (2012). Bilimsel araştırma yöntemi [Scientific research method]. Ankara: Nobel Akademik Yayıncılık.
28. Khan, M. Y., Qayoom, A., Nizami, M. S., Siddiqui, M. S., Wasi, S., & Raazi, S. M. K. U. R. (2021). Automated prediction of good dictionary examples (GDEX): A comprehensive experiment with distant supervision, machine learning, and word embedding‐based deep learning techniques. Complexity, 2021(1), 2553199. https://doi.org/10.1155/2021/2553199.
29. Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015). Machine learning applications in cancer prognosis and prediction. Computational and Structural Biotechnology Journal, 13, 8–17. https://doi.org/10.1016/j.csbj.2014.11.005.
30. Koyuncu, İ. (2018). Öğrencilerin PISA matematik başarılarının yordanmasında veri madenciliği yöntemlerinin karşılaştırılması. (Yayınlanmamış Doktora Tezi). Hacettepe Üniversitesi, Ankara.
31. Kucak, D., Juricic, V. & Dambic, G. (2018). Machine learning in education- a survey of current research trends. 29th DAAAM International Symposium, Vienna, Austria.
32. Kuch, D., Kearnes, M., & Gulson, K. (2020). The Promise of precision: Datafication in medicine, agriculture and education. Policy Studies, 41(5), 527-546. https://doi.org/10.1080/01442872.2020.1724384.
33. Kuroki, M. (2023). Integrating data science into an econometrics course with a Kaggle competition. The Journal of Economic Education, 54(4), 364-378. http://hdl.handle.net/10.1080/00220485.2023.2220695
34. Lin, W. Y., Hu, Y. H., & Tsai, C. F. (2011). Machine learning in financial crisis prediction: A Survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4), 421-436. 10.1109/TSMCC.2011.2170420.
35. Nurhachita, N. ve Negara E. S. (2021). A comparison between deep learning, naïve bayes and random forest for the application of data mining on the admission of new students. IAES International Journal of Artificial Intelligence (IJ-AI) 2(10), 324-331. http://doi.org/10.11591/ijai.v10.i2.pp324-331.
36. Nurpiana, A., & Wijayanto, A. W. (2022). Comparison of models for classification of learning achievement of middle school students in ındonesia in 2019 using the support vector machine algorithm, conditional inference trees, and random forest. Jurnal Matematika, Statistika dan Komputasi, 18(3), 447-455. https://doi.org/10.20956/j.v18i3.19208.
37. Özlüer Başer, B., Yangın, M. & Sarıdaş, E. S. (2021). Makine öğrenmesi teknikleriyle diyabet hastalığının sınıflandırılması [Classification of diabetes using machine learning techniques]. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 25 (1), 112-120. https://doi.org/10.19113/sdufenbed.842460.
38. Quaranta, L., Calefato, F., & Lanubile, F. (2021, May). KGTorrent: A dataset of python jupyter notebooks from kaggle. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR) (pp. 550-554). IEEE. https://doi.org/10.1109/MSR52588.2021.00072
39. Quinn, R.J., & Gray, G. (2020). Prediction of student academic performance using Moodle data from a Further Education setting, Irish Journal of Technology Enhanced Learning, 5(1), 17-29. https://doi.org/10.22554/ijtel.v5i1.57.
40. Rácz, A., Bajusz, D., & Héberger, K. (2019). Multi-level comparison of machine learning classifiers and their performance metrics. Molecules, 24(15), 2811 https://doi.org/10.3390/molecules24152811.
41. Rajendran, S., Chamundeswari, S., & Sinha, A. A. (2022). Predicting the academic performance of middle-and high-school students using machine learning algorithms. Social Sciences & Humanities Open, 6(1), 100357. https://doi.org/10.1016/j.ssaho.2022.100357
42. Rao, K. P., Sekhara, M. C., & Ramesh, B. (2016). Predicting learning behavior of students using classification techniques. International Journal of Computer Applications, 139(7), 15-19. https://www.ijcaonline.org/research/volume139/number7/rao-2016-ijca-909188.pdf.
43. Rebai, S., Yahia, F. B., & Essid, H. (2020). A graphically based machine learning approach to predict secondary schools performance in Tunisia. Socio-Economic Planning Sciences, 70, 100724. https://doi.org/10.1016/j.seps.2019.06.009
44. Rish, I. (2001). An empirical study of the naive Bayes classifier. In IJCAI 2001 Workshop On Empirical Methods in Artificial Intelligence, 3(22), 41-46. https://www.researchgate.net/publication/228845263.
45. Rodrigues, I., Parayil, A., Shetty, T., & Mirza, I. (2020). Use of linear discriminant analysis (LDA), k nearest neighbours (KNN), decision tree (CART), random forest (RF), gaussian naive bayes (NB), support vector machines (SVM) to predict admission for post-graduation courses [Paper Presentation]. International Conference on Recent Advances in Computational Techniques (IC-RACT).
46. Sarıcaoğlu, C. (2019). Sözcüksel analiz kullanarak kötü niyetli url'leri derin öğrenme teknikleri ile tespit etme (Publication No.655610) [Yayınlanmamış Yüksek Lisans Tezi]. Gazi Üniversitesi, Ankara.
47. Sembiring, S., Zarlis, M., Hartama, D., Ramliana, S., & Wani, E. (2011, April). Prediction of student academic performance by an application of data mining techniques International Conference on Management and Artificial Intelligence IPEDR (Vol. 6, pp. 110-114).
48. Shahiri, A. M., & Husain, W. (2015). A review on predicting student's performance using data mining techniques. Procedia Computer Science, 72, 414-422. https://doi.org/10.1016/j.procs.2015.12.157
49. Shalev-Shwartz, S., & Ben-David, S. (2014). Understanding machine learning: From theory to algorithms. Cambridge University Press.
50. Sullivan, G. M., & Feinn, R. (2012). Using effect size—or why the P value is not enough. Journal of graduate medical education, 4(3), 279-282. https://doi.org/10.4300/JGME-D-12-00156.1.
51. Şengür, D., & Tekin, A. (2013). Öğrencilerin mezuniyet notlarının veri madenciliği metotları ile tahmini [Prediction of students' graduation grades using data mining methods]. Bilişim Teknolojileri Dergisi, 6(3), 7-16. https://dergipark.org.tr/tr/download/article-file/75330.
52. Tosunoğlu, E., Yılmaz, R., Özeren, E., Sağlam, Z. (2021). Eğitimde makine öğrenmesi: Araştırmalardaki güncel eğilimler üzerine inceleme [Machine learning in education: A review of current trends in research]. Ahmet Keleşoğlu Eğitim Fakültesi Dergisi, 3(2), 178-199. https://dergipark.org.tr/en/download/article-file/1873550
53. Türkmenoglu, C., & Tantug, A. C. (2014, June). Sentiment analysis in Turkish media [Paper Presentation]. International Conference on Machine Learning (ICML). Beijing, China.
54. Ülker, M., & Civalek, O. (2002). Yapay sinir ağları ile eksenel yüklü kolonların burkulma analizi [Buckling analysis of axially loaded columns using artificial neural networks]. Turkish J. Eng. Env. Sci, 26, 117-125.
55. Vandamme, J. P., Meskens, N., & Superby, J. F. (2007). Predicting academic performance by data mining methods. Education Economics, 15(4), 405. https://doi.org/10.1080/09645290701409939.
56. Vapnik, V. (2013). The nature of statistical learning theory. Springer science & business media.
57. Vijayalakshmi, V., & Venkatachalapathy, K. (2019). Comparison of predicting student’s performance using machine learning algorithms. International Journal of Intelligent Systems and Applications, 10(12), 34. https://doi.org/10.5815/ijisa.2019.12.04.
58. Yağcı, M. (2022). Educational data mining: prediction of students' academic performance using machine learning algorithms. Smart Learning Environments, 9(1), 11. https://doi.org/10.1186/s40561-022-00192-z
59. Yang, S. J. H. (2019). Precision education: New challenges for AI in education [Paper Presentation]. 27th International Conference on Computers in Education (ICCE) (pp. XXVII-XXVIII). Kenting, Taiwan.
60. Yassein, N. A., Helali, R. G. M., & Mohomad, S. B. (2017). Predicting student academic performance in KSA using data mining techniques. Journal of Information Technology & Software Engineering, 7(5), 1-5. DOI:10.4172/2165-7866.1000213
61. Zhou, Y., Huang, C., Hu, Q., Zhu, J., & Tang, Y. (2018). Personalized learning full-path recommendation model based on LSTM neural networks. Information Sciences, 444, 135-152. https://doi.org/10.1016/j.ins.2018.02.053.
62. Zilyas, D. ve Yılmaz, A. (2023). Makine öğrenmesi yöntemleri ile eğitim başarısının tahmini modeli [A model for predicting academic success using machine learning methods]. DÜMF MD, 14(3), 437–447. https://doi.org/10.24012/dumf.1322273.
63. Wang, A. Y., Wang, D., Drozdal, J., Liu, X., Park, S., Oney, S., & Brooks, C. (2021, May). What makes a well-documented notebook? a case study of data scientists’ documentation practices in kaggle. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-7). https://hdl.handle.net/1721.1/146030
64. Williamson, B. (2019). Digital policy sociology: Software and science in data-intensive precision education. Critical Studies in Education, 62(2), 1-17. https://doi.org/10.1080/17508487.2019.1691030.
65. Wu, D., Jennings, C., Terpenny, J., Gao, R. X., & Kumara, S. (2017). A Comparative study on machine learning algorithms for smart manufacturing: Tool wear prediction using random forests. Journal of Manufacturing Science and Engineering, 139(7), 071018. https://doi.org/10.1115/1.4036350.
66. Wu, J. Y., Hsiao, Y. C., & Nian, M. W. (2018). Using supervised machine learning on large-scale online forums to classify course-related Facebook messages in predicting learning achievement within the personal learning environment. Interactive Learning Environments, 28(1), 65–80. https://doi.org/10.1080/10494820.2018.1515085.