Predicting Student Dropout Using Explainable Artificial Intelligence: The Impact of SMOTE-Based Class Balancing Techniques

Abstract

Student dropout poses a significant challenge to both individual and societal progress. While artificial intelligence is widely used to identify potential dropouts and analyze their causes, its success relies heavily on well-structured datasets. Imbalanced datasets may limit models from achieving good generalization by adapting to the majority class. This study examines the impact of dataset balancing techniques on dropout prediction using artificial intelligence models and interprets model decisions through explainable artificial intelligence, based on the publicly available Predict Students’ Dropout and Academic Success datasets. Accordingly, four different datasets were created using the SMOTE, SMOTE-ENN, and SMOTE-Tomek balancing techniques and were trained with machine learning models. The SMOTE-ENN + CATBoost combination, which achieved 98% accuracy, precision, recall, F1 score, and an AUC score of 1.00, was selected as the best-performing combination and was subjected to LIME and SHAP analyses for model interpretability. The analyses showed that low academic performance increases the risk of dropping out; in addition, support mechanisms such as scholarships encourage students to complete their education. Compared to similar studies, the findings address both class balancing techniques and model explainability within a broader scope and examine the underlying causes of student dropout through explainable artificial intelligence.

Keywords

• Data balancing; School dropout
• SMOTE
• SMOTE-ENN
• SMOTE-Tomek

Supporting Institution

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Ethical Statement

This study does not involve any human participants or animals, and no intervention was performed. The analyses were conducted on anonymized and/or publicly available secondary data. Therefore, formal ethical approval and informed consent were not required.

Thanks

We thank the reviewers and editors for their helpful comments that improved the manuscript.

Açıklanabilir Yapay Zekâ ile Öğrenci Terk Riskinin Tahmin Edilmesi: SMOTE Tabanlı Sınıf Dengeleme Yaklaşımlarının Etkisi

Abstract

Öğrenci terkleri, hem bireysel hem de toplumsal ilerleme açısından önemli bir sorun oluşturmaktadır. Yapay zekâ, olası terk durumlarını belirlemek ve bunların nedenlerini analiz etmek amacıyla yaygın biçimde kullanılmaktadır; ancak bu başarı büyük ölçüde veri kümelerinin iyi yapılandırılmasına bağlıdır. Dengesiz veri kümeleri, modellerin çoğunluk sınıfa uyum sağlayarak genelleme yeteneklerinin zayıflamasına yol açabilir. Bu çalışma, yapay zekâ modelleri kullanılarak öğrenci terkinin tahmin edilmesinde veri kümesi dengeleme tekniklerinin etkisini incelemekte ve model kararlarını açıklanabilir yapay zekâ yaklaşımlarıyla yorumlamaktadır. Analizler, kamuya açık Predict Students’ Dropout and Academic Success veri kümesi temelinde gerçekleştirilmiştir. Bu doğrultuda SMOTE, SMOTE-ENN ve SMOTE-Tomek dengeleme teknikleri kullanılarak dört farklı veri seti oluşturuldu ve makine öğrenme modelleri ile eğitildi. %98 doğruluk, kesinlik, duyarlılık, F1 skoru ve 1.00 AUC skoruna ulaşan SMOTE-ENN + CATBoost kombinasyonu en iyi performansı gösteren kombinasyon olarak seçildi ve model yorumlanabilirliği için LIME ve SHAP analizlerine tabi tutuldu. Yapılan analizler, düşük akademik performansın okulu bırakma riskini artırdığını; bunun yanı sıra burs gibi destek mekanizmalarının öğrencileri eğitimlerini tamamlamaya teşvik ettiğini göstermektedir. Bulgular benzer çalışmalara kıyasla hem sınıf dengeleme tekniklerini hem de model açıklanabilirliğini daha geniş bir kapsamla ele almakta ve öğrenci terkinin altında yatan nedenleri açıklanabilir yapay zekâ ile incelemektedir.

Keywords

• Veri dengeleme
• Okul terki
• SMOTE
• SMOTE-ENN
• SMOTE-Tomek

Supporting Institution

Bu çalışma herhangi bir kamu, ticari veya kâr amacı gütmeyen kuruluştan özel bir fon desteği almamıştır.

Ethical Statement

Bu çalışma insan katılımcı veya hayvan deneği içermemekte olup herhangi bir müdahale gerçekleştirilmemiştir. Analizler anonimleştirilmiş ve veya kamuya açık ikincil veriler üzerinde yürütülmüştür. Bu nedenle etik kurul onayı ve aydınlatılmış onam gerekmemiştir

Thanks

Makalenin geliştirilmesine katkı sağlayan editör ve hakemlere değerlendirmeleri için teşekkür ederiz.

References

1. H. M. Levin, “The economic payoff to investing in educational justice,” Educ. Res., vol. 38, no. 1, pp. 5–20, 2009.
2. E. M. Allensworth and J. Q. Easton, What Matters for Staying On-Track and Graduating in Chicago Public High Schools: A Close Look at Course Grades, Failures, and Attendance in the Freshman Year, Chicago, IL, USA: Consortium on Chicago School Research, Res. Rep., 2007.
3. R. W. Rumberger and S. A. Lim, Why Students Drop Out of School: A Review of 25 Years of Research, 2008.
4. R. Baker and G. Siemens, “Learning analytics and educational data mining,” in Cambridge Handbook of the Learning Sciences, 2014, pp. 253–272.
5. H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263–1284, 2009.
6. A. Fernández, S. Garcia, F. Herrera, and N. V. Chawla, “SMOTE for learning from imbalanced data: Progress and challenges, marking the 15-year anniversary,” J. Artif. Intell. Res., vol. 61, pp. 863–905, 2018.
7. M. Nagy and R. Molontay, “Predicting dropout in higher education based on secondary school performance,” in Proc. IEEE 22nd Int. Conf. Intell. Eng. Syst. (INES), Las Palmas de Gran Canaria, Spain, 2018, pp. 389–394.
8. B. Kiss, M. Nagy, R. Molontay, and B. Csabay, “Predicting dropout using high school and first-semester academic achievement measures,” in Proc. 17th Int. Conf. Emerg. eLearning Technol. Appl. (ICETA), Starý Smokovec, Slovakia, 2019, pp. 383–389.

9. S. Lee and J. Y. Chung, “The machine learning-based dropout early warning system for improving the performance of dropout prediction,” Appl. Sci., vol. 9, no. 15, p. 3093, 2019.
10. K. C. Kirana, S. Wibawanto, H. W. Herwanto, W. S. G. Irianto, W. N. Hidayat, and M. Aqshal, “Naive Bayes-based drop out recommendation system in vocational college,” in Proc. 5th Vocat. Educ. Int. Conf. (VEIC-5 2023), 2024, pp. 727–733.
11. M. V. Martins, L. Baptista, J. Machado, and V. Realinho, “Multi-class phased prediction of academic performance and dropout in higher education,” Appl. Sci., vol. 13, no. 8, p. 4702, Apr. 2023.
12. P. Padmasiri and S. Kasthuriarachchi, “Interpretable prediction of student dropout using explainable AI models,” in Proc. Int. Res. Conf. Smart Comput. Syst. Eng. (SCSE), vol. 7, Apr. 2024, pp. 1–7. IEEE formatında düzenlenmiş, DOI ve URL satırları kaldırılmış, dergi adları kısaltılmış hali:
13. M. Shoaib, N. Sayed, J. Singh, J. Shafi, S. Khan, and F. Ali, “AI student success predictor: Enhancing personalized learning in campus management systems,” Comput. Hum. Behav., vol. 158, p. 108301, 2024.
14. H. Strohmier, V. Langner, F. Mohamed, and E. Wood, “Examination of artificial intelligence integration and impact on higher education,” in Proc. 12th Int. Symp. Digit. Forensics Security (ISDFS), Apr. 2024, pp. 1–6.
15. H. El Aouifi, M. El Hajji, and Y. Es-Saady, “A hybrid approach for early-identification of at-risk dropout students using LSTM-DNN networks,” Educ. Inf. Technol., 2024.
16. UCI Machine Learning Repository, “Predict students’ dropout and academic success dataset,” [Online]. [Accessed: Aug. 31, 2024].
17. S. Alexandropoulos, S. Kotsiantis, and M. Vrahatis, “Data preprocessing in predictive data mining,” Knowl. Eng. Rev., vol. 34, 2019.
18. J. Chachi and S. Taheri, “Outlier detection in fuzzy regressions,” in Lect. Notes Comput. Sci., pp. 299–307, 2020.
19. J. Jeong, E. Park, W. Han, K. Kim, S. Choung, and I. Chung, “Identifying outliers of non-Gaussian groundwater state data based on ensemble estimation for long-term trends,” J. Hydrol., vol. 548, pp. 135–144, 2017.
20. N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” J. Artif. Intell. Res., vol. 16, pp. 321–357, 2002.
21. A. Fernández, S. Garcia, F. Herrera, and N. V. Chawla, “SMOTE for learning from imbalanced data: Progress and challenges, marking the 15-year anniversary,” J. Artif. Intell. Res., vol. 61, pp. 863–905, 2018.
22. H. Zhang, Y. Zhang, P. Lu, and C. Wang, “Research on network intrusion detection based on SMOTEENN and improved CatBoost algorithm,” Proc. SPIE, vol. 12800, pp. 128001U-1–128001U-6, 2023.
23. M. Zeng, B. Zou, F. Wei, X. Liu, and L. Wang, “Effective prediction of three common diseases by combining SMOTE with Tomek links technique for imbalanced medical data,” in Proc. IEEE Int. Conf. Online Anal. Comput. Sci. (ICOACS), 2016, pp. 225–228.