CLASSIFICATION OF STUDENTS' ACADEMIC SUCCESS USING ENSEMBLE LEARNING AND ATTRIBUTE SELECTION

Abstract

Students' success in high school plays an important role in shaping their lives, as it also affects their success in university placement. It is very important to be able to predict this situation so that in case of failure, precautions can be taken, and a solution can be produced. If success situations and failure can be predicted, success can be increased and stabilized with encouragement and support. In this study, students' academic performances were tried to be estimated with the datasets prepared with secondary school students in Portugal. The datasets include students' answers about the factors thought to affect their success-failure and their grades. The wide use and efficiency of machine learning algorithms have also affected studies on predicting student success. Different algorithms have been applied using different methods in the datasets and the correct prediction rate was tried to be maximized. Experiments were carried out using the 10-fold cross validation method. Deep learning, multilayer perceptrons, simple logistic regression, decision table, one rule, iterative classifier optimizer, logistic model tree and fuzzy unordered rule induction algorithm have been used to predict the student academic success. These algorithms have been tested with the classical and bagging methods. The experiments also tested the efficiency of the algorithms in predicting student success by selecting features and comparing the results.

Keywords

• Machine learning
• Classification
• Students’ academic success

References

1. [1] Rao K, Rao M, Ramesh B. Predicting learning behavior of students using classification techniques. International Journal of Computer Applications, 2016; 139(7).
2. [2] Al-Radaideh QA, Al-Shawakfa E, Al-Najjar M. Mining student data using decision trees. In: International Arab Conference on Information Technology; January 2006; Yarmouk University, Jordan.
3. [3] Şama E, Tarım K. Teachers’ attitudes and behaviors towards students perceived as unsuccessful. Türk Eğitim Bilimleri Dergisi, Kış 2007; 5(1):135-154.
4. [4] Sezer Ö. Some demographic characteristics of the repeating students and the opinions of the students and the teachers about repetition. (article in Turkish, summary in English). İnönü Üniversitesi Eğitim Fakültesi Dergisi, Güz 2007; 8(14): 31–48.
5. [5] Yadav SK, Pal S. Data Mining: A prediction for performance improvement of engineering students using classification. World of Computer Science and Information Technology Journal, 2012; 2(2): 51-56.
6. [6] Gadhavi M, Patel D. Student final grade prediction based on linear regression. Indian Journal of Computer Science and Engineering, 2017; 8(3): 274-279.
7. [7] Erdoğan Z, Namlı Ö, Akarsu C. Öğrenci başarısının bileşik makine öğrenme teknikleri kullanılarak tahmini. In: International Symposium on Industry 4.0 and Applications, October 2017; Karabük, Turkey. pp.150-155.
8. [8] Cortez P, Silva A. Using data mining to predict secondary school student performance. In: 5th Annual Future Business Technology Conference; 2008; Porto: pp. 5-12.

9. [9] Vijayalakshmi V, Venkatachalapathy K. Comparison of predicting student‘s performance using machine learning algorithms. Intelligent Systems and Applications, 2019; 11(12): 34-45.
10. [10] Tosunoğlu E, Yılmaz, R, Özeren E, Sağlam Z. Machine learning in education: a study on current trends in researchs. (article in Turkish, summary in English). Journal of Ahmet Keleşoğlu Education Faculty, 2021; 3(2): 178-199.
11. [11] Güvenç E, Sakal M, Çetin G, Özkaraca O. Classification of students' course qualifications using machine learning techniques (article in Turkish, summary in English). Düzce Üniversitesi Bilim ve Teknoloji Dergisi 2022; 10(3): 1359-1371.
12. [12] Salameh Shreem S, Turabieh H, Al Azwar S, Baothman F. Enhanced binary genetic algorithm as a feature selection to predict student performance. Soft Computing, 2022; 26(1):1-13.
13. [13] Quy TL, Nguyen TH, Friege G, Ntoutsi E. Evaluation of group fairness measures in student performance prediction problems. Communications in Computer and Information Science, January 2023.
14. [14] Özkan Y, Önay Koçoğlu F, Selçukcan Erol Ç. Prediction of student performance by deep learning algorithm (article in Turkish, summary in English). In: 7th International Conference on Innovations in Learning for the Future, 2018; İstanbul University, İstanbul pp.136-145.
15. [15] Başer SH, Hökelekli O, Adem K. Predicting the performance of students studying in secondary education using data mining methods (article in Turkish, summary in English). Bilgisayar Bilimleri ve Teknolojileri Dergisi, 2020; 1(1): 22-27.
16. [16] Çınar D, Yılmaz Gündüz S. Prediction of secondary school students' mathematics success with machine learning. In:13th International Istanbul Scientific Research Congress on Life, Engineering, and Applied Sciences, Nisan 2023. İstanbul:pp.94-104
17. [17] Yavuzarslan M, Erol Ç. Using learning management system logs to predict undergraduate students’ scademic performance. Bilişim Teknolojileri Dergisi 2022; 15(2): 199-207.
18. [18] Kayalı S, Buyrukoğlu S. Classification of academic performance of students with the implementation of machine learning algorithms (article in Turkish, summary in English). In: 2nd International Conference on Educational Technology and Online Learning; Ağustos 2022. Balıkesir, pp.330-336
19. [19] Bentaleb A, Abouchabaka J. Ensemble learning for mining educational data. Journal of Theoretical and Applied Information Technology 2022: 100(9): 2715-2722.
20. [20] Bozkurt Keser S,Aghalarova S. HELA: A novel hybrid ensemble learning algorithm for predicting academic performance of students. Education and Information Technologies 2022; 27: 4521–4552.
21. [21] Ünal F. Data mining for student performance prediction in education.In: Derya Birant,editor. Data Mining-Methods, Applications and Systems. e-book. 2020; pp. 423-432.
22. [22] Kızılkaya Y M, Oğuzlar A. Comparison of some supervised learning algorithms r programming language (article in Turkish, summary in English). Karadeniz Uluslararası Bilimsel Dergi, 2018; 37: 90-98.
23. [23] Felicia, Ferren. Exploring secondary school performance by using machine learning algorithms. Journal of Educational Analytics, 2022; 1(1): 41-60.
24. [24] Krizhevsky A, Sutskever I, Hinton EG. ImageNet classification with deep convolutional neural networks. In: F. Pereira and C.J. Burges and L. Bottou and K.Q. Weinberger,editors. Advances in Neural Information Processing Systems. 2012. pp. 1090–1098.
25. [25] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521:436-444
26. [26] Raiko T, Valpola H, Lecun, Y. Deep learning made easier by linear transformations in perceptrons. In: Fifteenth International Conference on Artificial Intelligence and Statistics 2012; 22:924-932.
27. [27] MW Gardner, SR Dorling. Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences. Atmospheric Environment 1998; 32: 2627-2636
28. [28] EB Baum, On the capabilities of multilayer perceptrons. Journal of Complexity 1988; 4: 193-215
29. [29] Bayır F. Artificial neural networks and application on forecasting. MSc İstanbul University, İstanbul, Turkey, 2006.
30. [30] Diaz-Quijano FA. A simple method for estimating relative risk using logistic regression. BMC Medical Research Methodology 2012; 12: 1-6.
31. [31] Hsieh FY, Bloch DA, Larsen MD. A simple method of sample size calculation for linear and logistic regression. Statistics in Medicine 1998;17(14):1623-1634.
32. [32] Manikandan G, Vasudev A, Balasubramanian A. A survey to identify an efficient classification algorithm for heart disease prediction. International Journal of Pure and Applied Mathematics 2018; 119(12): 13337-13345.
33. [33] Fan S, Huang B. Training iterative collective classifiers with back-propagation In: 12th International Workshop on Mining and Learning with Graphs, San Francisco, USA. [34] Vural T. Microwave brain mass lesion diagnosis by using data mining. MSc, Yıldız Technical University, İstanbul, Turkey, 2017.
34. [35] Vanthienen J. A more general comparison of the decision table and tree: a response. DTEW Research Report 9224, 1992.
35. [36] Kohavi R. The power of decision tables. In: Machine Learning: European Conference on Machine Learning;2005; 912; Springer, Berlin, Heidelberg: pp.174–189.
36. [37] Huysmans J. Dejaeger K. Mues C. Vanthienen J. Baesens B. An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models, Decision Support Systems; 2011;(51)1: 141-154.
37. [38] Hühn J, Hüllermeier E. FURIA: an algorithm for unordered fuzzy rule induction. Data Min Knowl Disc 2009; 19: 293–319.
38. [39] Olaru C. Wehenkel L. A complete fuzzy decision tree technique, Fuzzy Sets and Systems, 2003; 138(1): 221-254
39. [40] Al-diabat M. Arabic text categorization using classification rule mining. Applied Mathematical Sciences 2012; 6(81): 4033-4046.
40. [41] Karadağ M. Comparison of performances of decision trees and logistic regression analysis by a simulation study. MSc, Trakya University, Edirne, Turkey, 2014.
41. [42] Landwehr N. Logistic model trees. University of Freiburg, Freiburg, Germany, 2003.
42. [43] Witten IH., et al. Practical machine learning tools and techniques. In: Data mining. Amsterdam, The Netherlands: Elsevier, 2005. p. 403-413.
43. [44] Guyon I, Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research, 2003; 3: 1157-1182.
44. [45] Dash M, Liu H. Feature selection for classification. Intelligent Data Analysis, 1997; 1: 131-156.
45. [46] Wang Z, Wang Y, Srinivasan R. A novel ensemble learning approach to support building energy use prediction. Energy and Buildings 2018; 159:109-122.
46. [47] Onan A. A clustering based classifier ensemble approach to corporate bankruptcy prediction (article in Turkish, summary in English). Alphanumeric journal; 2018; 6(2); 365 – 376.
47. [48] Çınar D. Prediction of secondary school students' success with machine learning. MSc, Eskisehir Technical University, Eskisehir, Turkey, 2023.