Predicting early university dropout in open and distance education: A comparison of data mining models

Abstract

While open and distance education programs provide access to higher education, they face higher dropout rates than conventional campus-based programs. Early identification of students at risk is a critical step in supporting them. The existing literature on dropout has been quite limited. Despite the potential of Educational Data Mining (EDM) to transform student retention strategies, there is a noticeable lack of studies evaluating algorithm performance on national-level datasets. In particular, the use of data mining techniques in educational systems with big data has not been sufficiently investigated. This study fills the gap by comparing eight different classification algorithms, including various Decision Trees and Artificial Neural Networks (ANN), on a massive dataset of 650,317 students. Focusing on naturally occurring early-stage data, this research aims to determine which models provide the highest prediction validity in the context of big data. The results show that the C5.0 algorithm has the highest classification accuracy, and the ANN provided the most robust discriminative power with an AUC of .867. The findings have the potential to contribute to actions aimed at preventing school dropout in open and distance education.

Keywords

• School dropout
• Open and distance education
• Educational data mining
• C5.0
• Artificial neural networks

References

1. Agrusti, F., Bonavolonta, G., & Mezzini, M. (2019). University dropout prediction through educational data mining techniques: a systematic review. Journal of e-Learning and Knowledge Society, 15(3), 161-182. https://doi.org/10.20368/1971-8829/1135017
2. Aguş, F.S. (2019). Açık ve uzaktan eğitimde öğrencilerin okulu bırakma nedenleri: Anadolu Üniversitesi açıköğretim sistemi üzerine bir araştırma [Reasons for student dropout in open and distance education: A study at Anadolu University open education system]. [Unpublished master's thesis]. Anadolu University.
3. Ahmed, A.B.E.D., & Elaraby, I.S. (2014). Data mining: A prediction for student’s performance using classification method. World Journal of Computer Application and Technology, 2(2), 43-47. https://doi.org/10.13189/wjcat.2014.020203
4. Akbaş, G., & Apar, A. (2010). Avrupa 2020 Stratejisi: Akıllı, sürdürülebilir ve kapsayıcı büyüme için avrupa stratejisi özet bilgi notu [Europe 2020 Strategy: European strategy for smart, sustainable and inclusive growth summary factsheet]. Ankara
5. Akgün, K., & Özek, M.B. (2020). Eğitsel veri madenciliği yöntemi ile ilgili yapılmış çalışmaların incelenmesi: İçerik analizi [Examining the studies on educational data mining method: content analysis]. Uluslararası Eğitim Bilim ve Teknoloji Dergisi, 3(2), 127 142. https://doi.org/10.47714/uebt.753526
6. Aksu, G. (2018). PISA başarısını tahmin etmede kullanılan veri madenciliği yöntemlerinin incelenmesi [Investigation of data mining methods used for estimating PISA success]. [Unpublished doctoral dissertation]. Hacettepe University.
7. Aksu, G., & Doğan, N. (2018). Veri madenciliğinde kullanılan öğrenme yöntemlerinin farklı koşullar altında karşılaştırılması [Comparison of learning methods used in data mining under different conditions]. Ankara Üniversitesi Egitim Bilimleri Fakultesi Dergisi, 52(1), 163 188. https://doi.org/10.30964/auebfd.464262
8. Albreiki, B., Zaki, N., & Alashwal, H. (2021). A Systematic literature review of students’ performance prediction using machine learning techniques. Education Sciences, 11(9), Article 552. https://doi.org/10.3390/educsci11090552

9. Altınkurt, Y. (2008). Öğrenci devamsızlıklarının nedenleri ve devamsızlığın akademik başarıya olan etkisi [Reasons for student absenteeism and the impact of absenteeism on academic achievement]. Dumlupınar Üniversitesi Sosyal Bilimler Dergisi, 20, 129-142. https://izlik.org/JA72KM49NL
10. Altunkaynak, B. (2022). Veri madenciliği yöntemleri ve R uygulamaları kavramlar – modeller– algoritmalar [Data mining methods and R applications concepts - models - algorithms]. Seçkin Publishing.
11. Appavoo, P., Gungea, M., & Sohoraye, M. (2023). Drop-out among ODL learners: A case study at the Open University of Mauritius. Journal of Educational Technology and Online Learning, 6(3), 665-682. https://doi.org/10.31681/jetol.1273563
12. Araque, F., Roldán, C., & Salguero, A. (2009). Factors influencing university drop out rates. Computers & Education, 53(3), 563-574. https://doi.org/10.1016/j.compedu.2009.03.013
13. Asma, A., Nusari, A.N., & Al-Shargabi, A. (2010). Discovering vital patterns from UST students data by applying data mining techniques. In V. Mahadevan & Z. Jianhong (Eds.), Proceedings of the 2nd international conference on computer and automation engineering (ICCAE) (Vol. 1, pp. 625-629). IEEE. https://doi.org/10.1109/ICCAE.2010.5451653
14. Aulck, L., Velagapudi, N., Blumenstock, J., & West, J. (2017). Predicting student dropout in higher education. arXiv preprint arXiv:1606.06364. https://doi.org/10.48550/arXiv.1606.06364
15. Aydın, S., Öztürk, A., Büyükköse, G.T., Er, F., & Sönmez, H. (2019). An investigation of drop-out in open and distance education. Educational Sciences: Theory and Practice, 19(2), 40 57. https://jestp.com/article-detail/?id=747
16. Ballı, F.E., & Kartal, S. (2020). Okul terki araştırmaları: sistematik bir analiz çalışması [School dropout research: A systematic analysis]. Kırıkkale University Sosyal Bilimler Dergisi (KÜSBD), 10(2), 257-278. https://dergipark.org.tr/en/download/article-file/1217520
17. Bargmann, C., Thiele, L., & Kauffeld, S. (2022). Motivation matters: Predicting students’ career decidedness and intention to drop out after the first year in higher education. Higher Education, 83(4), 845-861. https://doi.org/10.1007/s10734-021-00707-6
18. Bayer, J., Bydzovská, H., Géryk, J., Obsivac, T., & Popelinsky, L. (2012). Predicting drop-out from social behaviour of students. In K. Yacef, O. Zaïane, A. Hershkovitz, M. Yudelson & J. Stamper (Eds.), Proceedings of the 5th international conference on educational data mining (pp. 103-109). International Educational Data Mining Society. https://files.eric.ed.gov/fulltext/ED537184.pdf
19. Bayhan, G., & Dalgıç, G. (2012). School dropout according to the views of high school leavers. İnönü University Journal of the Faculty of Education, 13(3), 107-130.
20. Bean, J.P., & Metzner, B.S. (1985). A conceptual model of nontraditional undergraduate student attrition. Review of Educational Research, 55(4), 485 540. https://doi.org/10.3102/00346543055004485
21. Behr, A., Giese, M., Teguim Kamdjou, H.D., & Theune, K. (2020). Dropping out of university: A literature review. Review of Education, 8(2), 614-652. https://doi.org/10.1002/rev3.3202
22. Berge, Z.L., & Huang, Y. (2004). A Model for sustainable student retention: A Holistic perspective on the student dropout problem with special attention to e-learning. DEOSNEWS, 13(5).
23. Bonde, S.N., & Kirange, D.K. (2018). Educational data mining survey for predicting student’s academic performance. In A.P. Pandian, , T. Senjyu, S.M.S. Islam, H. Wang (Eds.), Proceedings of the international conference on computer networks, big data and IoT (ICCBI - 2018). Springer. https://doi.org/10.1007/978-3-030-24643-3_35
24. Bulut, O., Cormier, D.C., & Yildirim-Erbasli, S.N. (2022). Optimized screening for at-risk students in mathematics: A machine learning approach. Information, 13(8), Article 400. https://doi.org/10.3390/info13080400
25. Cheung, L.L.W., & Kan, A.C.N. (2002). Evaluation of factors related to student performance in a distance-learning business communication course. Journal of Education for Business, 77(5), 257-263.
26. Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21(1), Article 6. https://doi.org/10.1186/s12864-019-6413-7
27. Cho, M.H., & Heron, M.L. (2015). Self-regulated learning: The role of motivation, emotion, and use of learning strategies in students’ learning experiences in a self-paced online mathematics course. Distance Education, 36(1), 80-99. https://doi.org/10.1080/01587919.2015.1019963
28. Choi, H.J., & Park, J.H. (2017). Testing a path-analytic model of adult dropout in online degree programs. Computers & Education, 116, 130-138. https://doi.org/10.1016/j.compedu.2017.09.005
29. CollegeAtlas. (2014, August 12). U.S. College dropout rate and dropout statistics. https://www.collegeatlas.org/college-dropout.html
30. Dekker, G., Pechenizkiy, M., & Vleeshouwers, J. (2009). Predicting students drop out: A case study. In T. Barnes, M. Desmarais, C. Romero & S. Ventura (Eds.) Proceedings of the 2nd International Conference on Educational Data Mining (pp. 41-50). International Educational Data Mining Society. https://www.educationaldatamining.org/EDM2009/uploads/proceedings/dekker.pdf
31. Delen, D. (2010). A comparative analysis of machine learning techniques for student retention management. Decision Support Systems, 49(4), 498-506. https://doi.org/10.1016/j.dss.2010.06.003
32. Dutt, A., Ismail, M.A., & Herawan, T. (2017). A Systematic review on educational data mining. IEEE Access, 5, 15991-16005. https://doi.org/10.1109/access.2017.2654247
33. Fraenkel, J.R., Wallen, N.E., & Hyun, H.H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill.
34. Gil, J.S., Jhone, A., Delima, P., & Vilchez, R.N. (2020). Predicting student’s dropout ındicators in public school using data mining approaches. International Journal of Advanced Trends in Computer Science and Engineering, 9(1), 774-778. https://doi.org/10.30534/ijatcse/2020/110912020
35. Grau-Valldosera, J. (2019). A dropout definition for continuance intention and effective re-enrolment models in online distance learning [Unpublished doctoral dissertation]. Universitat Oberta de Catalunya. https://openaccess.uoc.edu/bitstream/10609/112746/1/2019-12%20Tesi%20jgrauv.pdf
36. Grau-Valldosera, J., & Minguillón, J. (2014). Rethinking dropout in online higher education: The case of the Universitat Oberta de Catalunya. International Review of Research in Open and Distributed Learning, 15(1), 290-308. https://doi.org/10.19173/irrodl.v15i1.1628
37. Grau-Valldosera, J., Minguillón, J., & Blasco-Moreno, A. (2018). Returning after taking a break in online distance higher education: From intention to effective re-enrollment. Interactive Learning Environments, 27(3), 307-323. https://doi.org/10.1080/10494820.2018.1470986
38. Guarín, C.E.L., Guzmán, E.L., & González, F.A. (2015). A Model to predict low academic performance at a specific enrollment using data mining. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 10(3), 119-125. https://doi.org/10.1109/RITA.2015.2452632
39. Guzmán, A., Barragán, S., & Cala Vitery, F. (2021). Dropout in rural higher education: A systematic review. Frontiers in Education, 6, Article 727833. https://doi.org/10.3389/feduc.2021.727833
40. Günther, E., Koeszegi, S.T. (2012). Gender counts?! Analysis of student dropout at Vienna University of Technology. In A. Béraud, A.S. Godfroy & J. Michel (Eds.) GIEE 2011: Gender and interdisciplinary education for engineers. SensePublishers. https://doi.org/10.1007/978-94-6091-982-4_32
41. Hagedorn, L. S. (2006). How to define retention: A new look at an old problem. Transfer and Retention of Urban Community College Students (TRUCCS). https://eric.ed.gov/?id=ED493674
42. IBM Corporation. (2022). IBM SPSS modeler 18.4 algorithms guide. https://www.ibm.com/docs/en/SS3RA7_18.4.0/pdf/AlgorithmsGuide.pdf
43. Kabakchieva, D. (2013). Predicting student performance by using data mining methods for classification. Cybernetics and Information Technologies, 13(1), 61-72. https://doi.org/10.2478/cait-2013-0006
44. Kayhan, O. (2019). Uzaktan eğitim öğrencilerinin mezuniyet durumlarının veri madenciliği yöntemleri ile tahmini: Amasya Universitesi örneği [Predicting the status of the graduating students in distance learning with the help of data mining methods: Amasya University Sample]. [Unpublished master's thesis]. Amasya University.
45. Kayri, M., & Şevgin, H. (Eds.). (2023). Eğitimde veri madenciliği ve bilgisayar uygulamaları [Data mining and computer applications in education]. Pegem Akademi.
46. Kember, D. (1995). Open learning courses for adults: A model of student progress. Educational Technology Publications.
47. Kotsiantis, S.B., Pierrakeas, C.J., & Pintelas, P.E. (2003). Preventing student dropout in distance learning using ML techniques. In V. Palade, R.J. Howlett & L. Jain (Eds.) Knowledge-based intelligent information and engineering systems: KES 2003 (pp. 267 274). Springer. https://doi.org/10.1007/978-3-540-45226-3_37
48. Lee, Y., & Choi, J. (2011). A review of online course dropout research: Implications for practice and future research. Educational Technology Research and Development, 59(5) 593 618. https://doi.org/10.1007/s11423-010-9177-y
49. Lottering, R., Hans, R., & Lall, M. (2020). A model for the identification of students at risk of dropout at a university of technology. In S. Pudaruth & U. Singh (Eds.), Proceedings of 2020 international conference on artificial intelligence, big data, computing and data communication systems (icABCD). IEEE. https://doi.org/10.1109/icABCD49160.2020.9183874
50. Márquez, C., Cano, A., Romero, C., Mohammad, A., Fardoun, H., & Ventura, S. (2016). Early dropout prediction using data mining: A case study with high school students. Expert Systems, 33(1), 107-124. https://doi.org/10.1111/exsy.12135
51. Marr, L., Gribble, J., & Ubachs, G. (2024). Supporting retention and student services in online and distance education. European Association of Distance Teaching Universities. https://dx.doi.org/10.5281/zenodo.11120440
52. Metlek, S., & Kayaalp, Ü.K. (2020). Destek vektör makineleri: makine öğrenmesinde, teoriden örnek MATLAB uygulamalarına kadar [Support vector machines: in machine learning, from theory to example MATLAB applications]. İksad Publishing.
53. Moldovan, A.N., Ghergulescu, I., & Jha, N. (2019). OULAD MOOC Dropout and result prediction using ensemble, deep learning and regression techniques. In H. Lane, S. Zvacek & J. Uhomoibhi (Eds.), Proceedings of the 11th international conference on computer supported education (CSEDU) (pp. 154-164). SciTePress. https://doi.org/10.5220/0007767901540164
54. OECD. (2010). Education at a 2010: OECD indicators. OECD Publishing. http://www.voced.edu.au/content/ngv%3A43104
55. Okur, M.R., Paşaoğlu Baş, D., & Uça Güneş, E.P. (2019). Açık ve uzaktan öğrenmede öğrenimi bırakma sebeplerinin incelenmesi [Examination of dropout causes in open and distance learning]. Journal of Higher Education and Science, 9(2), 225-235. https://doi.org/10.5961/jhes.2019.324
56. Özcan, T. (2013). Veri madenciliği [Data mining]. Istanbul University AUZEF. https://cdn acikogretim.istanbul.edu.tr/auzefcontent/21_22_Bahar/veri_madenciligi/1/index.html
57. Özer, M. (1991). İköğretim okulları ikinci kademe (ortaokul) öğrencilerinin öğrenimi terk etme sorununun analizi [Analysis of the problem of dropout of students of the second stage of primary schools (secondary school)] [Unpublished master’s thesis]. Ankara University.
58. Park, J.H. (2007). Factors related to learner dropout in online learning (ED504556). ERIC. https://files.eric.ed.gov/fulltext/ED504556.pdf
59. Pinheiro, C.B., Ribeiro, J.L.L.D.S., & Fernandes, S.A.F. (2023). Theoretical models of dropout in higher education and notes on the national context. Avaliação: Revista da Avaliação da Educação Superior (Campinas) 28, Article e023015. https://doi.org/10.1590/S1414-40772023000100023
60. Pradeep, A., Das, S., & Kizhekkethottam, J.J. (2015). Students dropout factor prediction using EDM techniques. In Proceedings of 2015 international conference on soft computing and networks security (ICSNS). IEEE. https://doi.org/10.1109/ICSNS.2015.72923712
61. Pulikottil, S.C., & Gupta, M. (2020). ONet – A Temporal meta embedding network for MOOC dropout prediction. In Proceedings of 2020 IEEE international conference on big data. IEEE. https://ieeexplore.ieee.org/document/9378001
62. Qiu, L., Liu, Y., & Liu, Y. (2018). An integrated framework with feature selection for dropout prediction in massive open online courses. IEEE Access, 6, 71474 71484. https://doi.org/10.1109/ACCESS.2018.2881275
63. Quadri, M.M.N., & Kalyankar, D.N.V. (2010). Drop out feature of student data for academic performance using decision tree techniques. Global Journal of Computer Science and Technology, 10(2), 2-5.
64. Reed, K., Wise, N., Tynan, B., & Bossu, C. (2013). Attrition in the digital age: reminders from the literature. In B. Tynan, J. Willems, & R. James (Eds.), Outlooks and opportunities in blended and distance learning (pp. 279-293). IGI Global. https://doi.org/10.4018/978-1-4666-4205-8.ch020
65. Rovai, A.P. (2003). In search of higher persistence rates in distance education online programs. The Internet and Higher Education, 6(1), 1-16. https://doi.org/10.1016/s1096-7516(02)00158-6
66. Serra, A., Perchinunno, P., & Bilancia, M. (2018). Predicting student dropouts in higher education using supervised classification algorithms. In O. Gervasi, B. Murgante, S. Misra, E. Stankova, C. M. Torre, A.M.A.C. Rocha, D. Taniar, B.O. Apduhan, E. Tarantino, Y. Ryu (Eds.) Computational science and its applications ICCSA 2018. Lecture notes in computer science. (pp. 18-33). Springer. https://doi.org/10.1007/978-3-319-95168-3_2
67. Simpson, O. (2013). Student retention in distance education: Are we failing our students? Open Learning: The Journal of Open, Distance and e Learning, 28(2), 105 119. https://doi.org/10.1080/02680513.2013.847363
68. Singer, C.G. (2023). Educational data mining: An application of a predictive model of online student enrollment decisions (Publication No. 30695078) [Doctoral dissertation, Arizona State University]. ProQuest Dissertations & Theses Global.
69. Souza, J., Matwin, S., & Japkowicz, N. (2002). Evaluating data mining models: A pattern language. In Proceedings of the 9th conference on pattern language of programs (PLoP’02). https://www.plopcon.org/pastplops/plop2002/final/PLoP2002_jtsouza0_1.pdf
70. Srilekshmi, M., Sindhumol, S., Chatterjee, S., & Bijlani, K. (2016). Learning analytics to identify students at-risk in MOOCs. In V. Kumar & S.M. Kinshuk (Eds.), Proceedings of the 2016 IEEE eighth international conference on technology for education (T4E) (pp. 194-199). Conference Publishing Services. https://dx.doi.org/10.1109/t4e.2016.048
71. Şengür, D., & Tekin, A. (2013). Öğrencilerin mezuniyet notlarının veri madenciliği metotları ile tahmini [Prediction of students’ graduation grades with data mining methods]. Bilişim Teknolojileri Dergisi, 6(3), 7-16.
72. Şimşek, H. (2011). Lise öğrencilerinde okulu bırakma eğilimi ve nedenleri [Trends and reasons for dropping out of school among high school students]. Eğitim Bilimleri Araştırmaları Dergisi, 1(1), 45-62.
73. Şirin, H., Özdemir, S., & Sezgin, F. (2009, October 1-3). Okulu terk eden çocukların ve velilerin okul terkine ilişkin görüşleri: Nitel bir inceleme [Opinions of dropout children and parents on school dropout: A qualitative study] [Paper presentation]. XVIII. Ulusal eğitim bilimleri kurultayı, Izmir Ege University, Turkey.
74. Tarimo, A. (2013). Factors affecting students’ enrolment and dropout at the Open University of Tanzania, Lindi Region. Journal of the Open University of Tanzania, 14(1), 111-133.
75. Taylı, A. (2008). Eğitim sisteminde önemli bir sorun: Okulu bırakma [A major problem in the education system: dropping out of school]. Türk Psikolojik Danışma ve Rehberlik Dergisi, 3(30), 89-101. https://doi.org/10.17066/pdrd.53724
76. Teki̇n, A., & Özteki̇n, Z. (2018). Eğitsel veri madenciliği ile ilgili 2006-2016 yılları arasında yapılan çalışmaların incelenmesi [Examination of studies on educational data mining between 2006-2016]. Eğitim Teknolojisi Kuram ve Uygulama, 8(2), 108-124. https://doi.org/10.17943/etku.351473
77. Thomas, L., & Unger, M. (2015). Dropout and completion in higher education in Europe. European Commission. https://doi.org/10.2766/826962
78. Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45(1), 125-189. https://doi.org/10.3102/00346543045001089
79. Vossensteyn, H., Kottmann, A., Jongbloed, B., Kaiser, F., Cremonini, L., Stensaker, B., Hovdhaugen, E., & Wollscheid, S. (2015). Dropout and completion in higher education in Europe: Main report. Center for Higher Education Policy Studies (CHEPS). https://research.utwente.nl/files/5137444/dropout-completion-he-summary_en.pdf
80. Xavier, M., & Meneses, J. (2020). A Literature review on the definitions of dropout in online higher education. In S.K. Softic, D. Andone, & A. Szucs (Eds.) Proceedings of the EDEN 2020 online annual conference (pp. 73-80). European Distance and E-Learning Network.
81. Yakut, E. (2012). Veri Madenciliği tekniklerinden C5.0 algoritması ve destek vektör makineleri ile yapay sinir ağlarının sınıflandırma başarılarının karşılaştırılması: İmalat sektöründe bir uygulama [The comparison of the classification successes of the artifical neural networks through data mining techniques of C5.0 algorithm and supporting vector machines: An application in manufacturing sector]. [Unpublished doctoral dissertation]. Atatürk University.
82. Yılmaz, A. (2020). Açık ve uzaktan eğitim öğrencilerinin öğrenimi bırakma ve öğrenime devam nedenlerinin incelenmesi [Investigation of open and distance education students’ reasons for dropping out and continuing their education]. [Unpublished doctoral dissertation]. Gazi University.
83. YÖK Statistics. (2024). 2023-2024 Öğretim yılı yükseköğretim istatistikleri [Higher education statistics for the academic year 2023-2024]. Retrieved January 9, 2025, from https://istatistik.yok.gov.tr/
84. Yulia, L.W.S. (2019). The analysis of student performance using data mining. In S.K. Bhatia, S. Tiwari, K.K. Mishra & M.C. Trivedi (Eds.) Advances in computer communication and computational sciences (pp. 559-573). Springer. https://doi.org/10.1007/978-981-13-6861-5_48
85. Zawacki-Richter, O., & Qayyum, A. (Eds.) (2019). Open and distance education in Asia, Africa and the Middle East: National perspectives in a digital age. Springer. https://doi.org/10.1007/978-981-13-5787-9