Data mining approach for prediction of academic success in open and distance education

Abstract

Predicting and improving the academic achievement of university students is a multifactorial problem. Considering the low success rates and high dropout rates, particularly in open education programs characterized by mass enrollment, academic success is an important research area with its causes and consequences. This study aimed to solve a classification problem (successful or unsuccessful), predict students’ academic success, and identify those at risk. The primary objective was to predict the academic success status with 26,708 students enrolled in Istanbul University open and distance education programs between 2011 and 2017. Predictions were based demographic data and success grades in Turkish, Atatürk's Principles and History of Revolution, English, and Disaster Culture courses. The study utilized classification models from supervised learning algorithms and was conducted using the SPSS Modeler 18 program. Initially, the data was divided into 70% training and 30% test data. Then, models were constructed by using Random Forest, Tree-AS, C&RT, C5.0, CHAID, QUEST, Naive Bayes, Logistic Regression, NeuralNet, and SVM algorithms. Model performances were compared according to accuracy, sensitivity, specificity, F1 score, positive predictive value, negative predictive value, and Matthews Correlation Coefficient criteria. The C&RT model demonstrated the best performance, achieving the highest specificity value of 0.915.

Keywords

• Data mining in education
• Academic success
• Open and distance education
• Demographic
• C&R Tree

References

1. Alan, M., & Temiz, M. (2019). A study on profiling students via data mining. Alphanumeric Journal, 7(2), 239-248. https://doi.org/10.17093/alphanumeric.630866
2. Albreiki, B., Habuza, T., Shuqfa, Z., Serhani, M.A., Zaki, N., & Harous, S. (2021). Customized
3. rule-based model to identify at-risk students and propose rational remedial actions. Big Data and Cognitive Computing, 5(71), 1-17. https://doi.org/10.3390/bdcc5040071
4. Altıntaş, Ö., Başer, F., & Babadoğan, M. C. (2021). Yükseköğretimde akademik riskli öğrencilerin kestiriminde makine öğrenmesi yöntemleri (A. Apaydın & Ö. Kutlu, Eds.). Pegem Akademi.
5. Alyahyan, E., & Düştegör, D. (2020). Predicting academic success in higher education: literature review and best practices. International Journal of Educational Technology in Higher Education, 17(1), 3. https://doi.org/10.1186/s41239-020-0177-7
6. Aydemi̇r, E. (2019). Ders geçme notlarının veri madenciliği yöntemleriyle tahmin edilmesi. European Journal of Science and Technology, (15), 70-76. https://doi.org/10.31590/ejosat.518899
7. Bağrıacık Yılmaz, A.& Karataş, S. (2022). Why do open and distance education students drop out? Views from various stakeholders. International Journal of Educational Technology in Higher Education,19(28), 1-22. https://doi.org/10.1186/s41239-022-00333-x
8. Batool, S., Rashid, J., Nisar, M. W., Kim, J., Kwon, H.Y., & Hussain, A. (2023). Educational data mining to predict students' academic performance: A survey study. Education and Information Technologies, 28(1), 905–971. https://doi.org/10.1007/s10639-022-11152-y

9. Bhise, R., Thorat, S.S. & Supekar, A.K. (2013). Importance of data mining in higher education system. Journal of Humanities and Social Science (IOSR-JHSS), 6(6), 18–21. https://doi.org/10.9790/0837-0661821
10. Bilici, Z., & Özdemi̇r, D. (2021). Data mining studies in education: Literature review for the years 2014-2020. Bayburt Eğitim Fakültesi Dergisi. 17(33), 342 - 376. https://doi.org/10.35675/befdergi.849973
11. Bonde, S. N., & Kirange, D. K. (2018). Educational data mining survey for predicting student’s academic performance. A. P. Pandian, T. Senjyu, S. M. S. Islam, H. Wang (Eds), In Proceeding of the International Conference on Computer Networks, Big Data and IoT (ICCBI - 2018): Vol.31. (pp. 293-302). Springer International Publishing. https://doi.org/10.1007/978-3-030-24643-3
12. Bulut, O., Cormier, D. C., & Yildirim-Erbasli, S. N. (2022). Optimized screening for at-risk students in mathematics: A machine learning approach. Information, 13(8), 400. https://www.mdpi.com/2078-2489/13/8/400
13. Çırak, G. (2012). Yükseköğretimde öğrenci başarılarının sınıflandırılmasında yapay sinir ağları ve lojistik regresyon yöntemlerinin kullanılması [Unpublished master thesis, Ankara Üniversitesi]. Ankara.
14. de Oliveira, C.F., Sobral, S.R., Ferreira, M.J., &Moreira, F. (2021). How does learning analytics contribute to prevent students’ dropout in higher education: A systematic literature review. Big Data and Cognitive Computing, 5(64), 1-33. https:// doi.org/10.3390/bdcc5040064
15. Dabhade, P., Agarwal, R., Alameen, K. P., Fathima, A. T., Sridharan, R., & Gopakumar, G. (2021). Educational data mining for predicting students’ academic performance using machine learning algorithms. Materials Today: Proceedings, pp. 47, 5260–5267. https://doi.org/https://doi.org/10.1016/j.matpr.2021.05.646
16. Durairaj, M., & Vijitha, C. (2014). Educational data mining for prediction of student performance using clustering algorithms. International Journal of Computer Science and Information Technologies, 5(4), 5987-5991.
17. Elakia, G., Aarthi, N. J. (2014). Application of data mining in educational database for predicting behavioural patterns of the students. International Journal of Computer Science and Information Technologies (IJCSIT), 5 (3), 4649–4652.
18. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education. McGraw-Hill.
19. He, Y., & Zhang, S. (2011, May 28-29). Application of data mining on students' quality evaluation. In 2011 3rd International Workshop on Intelligent Systems and Applications. IEEE.
20. Hotaman, D. (2020). Öğrenci başarısının değerlendirilmesinde eğitsel veri madenciliğinin kullanılması. Ulakbilge Dergisi, 8(48), 577–587. https://doi.org/10.7816/ulakbilge-08-48-08
21. IBM Corporation. (2022). Decision Tree Nodes. Retrieved 23.07.2023 from https://www.ibm.com/docs/en/cloud-paks/cp-data/4.7.x?topic=palette-modeling
22. Issah, I., Appiah, O., Appiahene, P., & Inusah, F. (2023). A systematic review of the literature on machine learning application of determining the attributes influencing academic performance. Decision Analytics Journal, 7, 100204. https://doi.org/https://doi.org/10.1016/j.dajour.2023.100204
23. Khasanah, A. U., & Harwati. (2017). A Comparative Study to Predict Student’s Performance Using Educational Data Mining Techniques. IOP Conference Series: Materials Science and Engineering, 215, 012036. https://doi.org/10.1088/1757-899x/215/1/012036
24. Kotsiantis, S., Pierrakeas, C., & Pintelas, P. (2004). Predicting students' performance in distance learning using machine learning techniques. Applied Artificial Intelligence, 18(5), 411-426. https://doi.org/10.1080/08839510490442058
25. Kotsiantis, S. B., & Pintelas, P. E. (2005, 5-8 July 2005). Predicting students marks in Hellenic Open University. Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05),
26. Kovacic, Z. (2010). Early prediction of student success: Mining students' enrolment data. In Proceedings of Informing Science and IT Education Conference.
27. Nahar, K., Shova, B. I., Ria, T., Rashid, H. B., & Islam, A. H. M. S. (2021). Mining educational data to predict students' performance. Education and Information Technologies, 26(5), 6051–6067. https://doi.org/10.1007/s10639-021-10575-3
28. Natek,S., Zwilling, M. (2014. Student data mining solution–knowledge management system related to higher education institutions. Expert systems with applications, 41 (14), 6400–6407. https://doi.org/10.1016/j.eswa.2014.04.024
29. Okur M.R., Paşaoğlu Baş D., & Uça Güneş E.P., (2019). Açık ve uzaktan öğrenmede öğrenimi bırakma sebeplerinin incelenmesi. Journal of Higher Education and Science, 9(2), 225-235. https://doi.org/10.5961/jhes.2019.324
30. Orrego Granados, D., Ugalde, J., Salas, R., Torres, R., & López-Gonzales, J. L. (2022). Visual-predictive data analysis approach for the academic performance of students from a Peruvian university. Applied Sciences, 12(21), 11251. https://doi.org/10.3390/app122111251
31. Osmanbegović, E., & Suljic, M. (2012). Data mining approach for predicting student performance. Journal of Economics & Business/Economic Review, 10, 3-12.
32. Özbay, Ö. (2015). Veri madenciliği kavramı ve eğitimde veri madenciliği uygulamaları. Uluslararası Eğitim Bilimleri Dergisi, (5), 262-272. https://dergipark.org.tr/tr/pub/inesj/issue/40015/475764
33. Özcan, T. (2013). Veri Madenciliği. İÜ AUZEF. https://cdn-acikogretim.istanbul.edu.tr/auzefcontent/21_22_Bahar/veri_madenciligi/1/index.html
34. Radovan, M. (2019). Should I stay, or Should I go? Revisiting Learner retention models in distance education. Turkish Online Journal of Distance Education, 20(3), 29–40. https:// doi. org/ 10. 17718/ tojde. 598211
35. Ramesh, V., P.Parkavi, & Ramar, K. (2013). Predicting Student Performance: A Statistical and Data Mining Approach. International Journal Of Computer Applications, 63, 975-8887.
36. Rana, S., & Garg, R. (2016, February 12-13). Application of hierarchical clustering algorithm to evaluate students' performance of an institute. Second International Conference on Computational Intelligence & Communication Technology (CICT).
37. Saheed, Y. K., Oladele, T. O., Akanni, A. O., & Ibrahim, W. M. (2018). Student performance prediction based on data mining classification techniques. Nigerian Journal of Technology, 37(4), 1087. https://doi.org/10.4314/njt.v37i4.31
38. Shahiri, A. M., Husain, W., & Rashid, N. a. A. (2015). A Review on Predicting Student's Performance Using Data Mining Techniques. Procedia Computer Science, 72, 414-422. https://doi.org/https://doi.org/10.1016/j.procs.2015.12.157
39. Sembiring, S., Zarlis, M., Hartama, D., Ramliana, S., & Wani, E. (2011, April). Prediction of student academic performance by an application of data mining techniques International Conference on Management and Artificial Intelligence IPEDR (Vol. 6, pp. 110-114).
40. Şengür, D., & Tekin, A. (2013). Öğrencilerin mezuniyet notlarının veri madenciliği metotları ile tahmini. Bilişim Teknolojileri Dergisi,6(3), 7-16.
41. Tan, S. S., Göktaş, Y., & Koçak, Ö. (2018, December 12-13). Veri madenciliği ile ÖSYM verileri kullanılarak akademik başarı tahmini [Conference presentation abstract] 2. Uluslararası Uzaktan Eğitim ve Yenilikçi Eğitim Teknolojileri Konferansı, Amasya.
42. Taşdemir, M. (2012). Veri madenciliği (Öğrenci başarısına etki eden faktörlerin regresyon analizi ile tespiti). (Publication Number 326726) [Master's thesis, Dicle Üniversitesi]. Diyarbakır. https://acikerisim.dicle.edu.tr/xmlui/handle/11468/789
43. Teki̇n, A., & Özteki̇n, Z. (2018). Eğitsel veri madenciliği ile ilgili 2006-2016 yılları arasında yapılan çalışmaların incelenmesi. Eğitim Teknolojisi Kuram ve Uygulama, 8(2), 108-124. https://doi.org/10.17943/etku.351473
44. Tosun, M. (2016). Açık öğretim öğrencilerinin akademik başarı düzeylerinin karşılaştırılması. (Publication Number 427908) [Master's thesis, İstanbul Üniversitesi]. İstanbul.
45. Türel, Y. K., & Baz, E. (2016, 6-8 Ekim). Eğitsel veri madenciliği üzerine bir araştırma. 4. Uluslararası Öğretim Teknolojileri ve Öğretmen Eğitimi Konferansı Bildirileri, Fırat Üniversitesi, Elazığ.
46. Veeramuthu, P., & Periasamy, R. (2014). Application of higher education system for predicting student using data mining techniques. International Journal of Innovative Research in Advanced Engineering, 1(5), 36-38.
47. Yossy, E. H., & Heryadi, Y. (2019, December). Comparison of data mining classification algorithms for student performance. In 2019 IEEE International Conference on Engineering, Technology, and Education (TALE) (pp. 1-4). IEEE. doi:10.1109/TALE48000.2019.9225887
48. YÖKSİS. (2023). Higher Education Information Management System; 2022-2023 Academic Year Higher Education Statistics. Retrieved 12.11.2023 from https://istatistik.yok.gov.tr/
49. Zawacki-Richter, O., & Qayyum, A. (2019). Open and distance education in Asia, Africa and the Middle East: National perspectives in a digital age. Springer Nature.