ENSEMBLE LEARNING FOR ACADEMIC PERFORMANCE PREDICTION: A MACHINE LEARNING APPROACH USING AUTOGLUON

Abstract

This study investigates the application of machine learning techniques to predict students' final letter grades based on their midterm and quiz scores. The research utilizes a dataset comprising 5,001 students enrolled in courses taught by twelve faculty members. Following the application of predefined eligibility criteria, the final dataset consisted of 2,746 students. The AutoGluon framework, an Automated Machine Learning (AutoML) tool, was employed to train and optimize the models. The training process was conducted in two phases: first, hyperparameter tuning was performed on eleven machine learning models, and their performance metrics were evaluated. Subsequently, the four best-performing models were integrated into an ensemble model, which was retrained to enhance predictive accuracy. The ensemble model achieved a notable accuracy of 92.32%, demonstrating its effectiveness in predicting academic outcomes. This study underscores the potential of ensemble learning and AutoML in educational data mining, providing valuable insights for improving decision-making processes and supporting student success in academic settings.

Keywords

• AutoGluon
• Machine Learning
• AutoML
• Data Mining
• Academic Performance

References

1. J. Horne, "Education and State Formation: The Rise of Education Systems in England, France and the USA," Sociology, vol. 27, no. 2, pp. 318-321, 1993.
2. S. L. Pressey, "A simple device for teaching, testing, and research in learning," School and Society, vol. 23, pp. 373–376, 1926.
3. B. F. Skinner, "Teaching machines," Scientific American, vol. 205, no. 5, pp. 90–106, 1961.
4. J. R. Carbonell, "AI in CAI: An artificial-intelligence approach to computer-assisted instruction," IEEE Transactions on Man-Machine Systems, vol. 11, no. 4, pp. 190–202, 1970.
5. W. Holmes, Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign, 2019.
6. M. L. Owoc, A. Sawicka, and P. Weichbroth, "Artificial intelligence technologies in education: benefits, challenges and strategies of implementation," in Proc. IFIP Int. Workshop Artif. Intell. Knowl. Manag., Cham, Springer, Aug. 2019, pp. 37–58.
7. I. Khan, A. R. Ahmad, N. Jabeur, and M. N. Mahdi, "An artificial intelligence approach to monitor student performance and devise preventive measures," Smart Learning Environments, vol. 8, pp. 1–18, 2021.
8. O. Zawacki-Richter, V. I. Marín, M. Bond, and F. Gouverneur, "Systematic review of research on artificial intelligence applications in higher education–where are the educators?," International Journal of Educational Technology in Higher Education, vol. 16, no. 1, pp. 1–27, 2019.

9. C. G. H. Suarez, J. Llanos, and V. A. Bucheli, "Predicting the final grade using a machine learning regression model: insights from fifty percent of total course grades in CS1 courses," PeerJ Computer Science, vol. 9, p. e1689, 2023.
10. M. Gadhavi and C. Patel, "Student final grade prediction based on linear regression," Indian Journal of Computer Science and Engineering, vol. 8, no. 3, pp. 274–279, 2017.
11. M. Yağcı, "Educational data mining: prediction of students' academic performance using machine learning algorithms," Smart Learning Environments, vol. 9, no. 1, p. 11, 2022.
12. E. Latif and S. Miles, "The impact of assignments and quizzes on exam grades: A difference-in-difference approach," Journal of Statistics Education, vol. 28, no. 3, pp. 289–294, 2020.
13. M. Jawthari and V. Stoffová, "Predicting students’ academic performance using a modified KNN algorithm," Pollack Periodica, vol. 16, no. 3, pp. 20–26, 2021.
14. S. K. Ghosh and F. Janan, "Prediction of student’s performance using random forest classifier," in Proc. Annu. Int. Conf. Ind. Eng. Oper. Manag., Singapore, Mar. 2021, pp. 7–11.
15. A. Joshi, P. Saggar, R. Jain, M. Sharma, D. Gupta, and A. Khanna, "CatBoost—An ensemble machine learning model for prediction and classification of student academic performance," Advances in Data Science and Adaptive Analysis, vol. 13, no. 03n04, p. 2141002, 2021.
16. V. Matzavela and E. Alepis, "Decision tree learning through a predictive model for student academic performance in intelligent m-learning environments," Computers and Education: Artificial Intelligence, vol. 2, p. 100035, 2021.
17. A. Nabil, M. Seyam, and A. Abou-Elfetouh, "Prediction of students’ academic performance based on courses’ grades using deep neural networks," IEEE Access, vol. 9, pp. 140731–140746, 2021.
18. S. Li and T. Liu, "Performance prediction for higher education students using deep learning," Complexity, vol. 2021, no. 1, p. 9958203, 2021.
19. S. Garmpis, M. Maragoudakis, and A. Garmpis, "Assisting educational analytics with AutoML functionalities," Computers, vol. 11, no. 6, p. 97, 2022.
20. Y. N. Mnyawami, H. H. Maziku, and J. C. Mushi, "Comparative study of AutoML approach, conventional ensemble learning method, and KNearest Oracle-AutoML model for predicting student dropouts in Sub-Saharan African countries," Applied Artificial Intelligence, vol. 36, no. 1, p. 2145632, 2022.
21. H. Villarreal-Torres, J. Ángeles-Morales, J. Cano-Mejía, C. Mejía-Murillo, G. Flores-Reyes, O. Cruz-Cruz, et al., "Comparative analysis of performance of AutoML algorithms: Classification model of payment arrears in students of a private university," EAI Endorsed Transactions on Scalable Information Systems, vol. 11, no. 4, 2024.
22. M. Liu, W. He, G. Zhou, and H. Zhu, "A new student performance prediction method based on belief rule base with automated construction," Mathematics, vol. 12, no. 15, p. 2418, 2024.
23. Y. Zhao and P. Wang, "The prediction and investigation of factors in the adaptability level of online learning based on AutoML and K-means algorithm," in Proc. IEEE 2nd Int. Conf. Control, Electron. Comput. Technol. (ICCECT), Apr. 2024, pp. 1313–1319.
24. A. S. Turan, B. Köksoy and A. Paşaoğlu, "An Artificial Intelligence Approach for Analyzing Students' Academic Performance Using Machine Learning Algorithms," 2025 4th International Conference on Sentiment Analysis and Deep Learning (ICSADL), Bhimdatta, Nepal, 2025, pp. 927-932.
25. S. R. Bharamagoudar, R. B. Geeta, and S. G. Totad, "Web based student information management system," International Journal of Advanced Research in Computer and Communication Engineering, vol. 2, no. 6, pp. 2342–2348, 2013.
26. L. Laloux, P. Cizeau, M. Potters, and J.-P. Bouchaud, "Random matrix theory and financial correlations," International Journal of Theoretical and Applied Finance, vol. 3, no. 03, pp. 391–397, 2000.
27. E. Fix and J. L. Hodges, "Discriminatory analysis. Nonparametric discrimination: consistency properties," Rep. No. 4, USAF School of Aviation Medicine, 1951.
28. T. Cover and P. Hart, "Nearest neighbor pattern classification," IEEE Transactions on Information Theory, vol. 13, no. 1, pp. 21–27, 1967.
29. N. S. Altman, "An introduction to kernel and nearest-neighbor nonparametric regression," The American Statistician, vol. 46, no. 3, pp. 175–185, 1992.
30. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. Wiley-Interscience, 2001.
31. D. Shepard, "A two-dimensional interpolation function for irregularly spaced data," in Proc. 23rd ACM National Conference, 1968, pp. 517–524.
32. S. A. Dudani, "The distance-weighted k-nearest-neighbor rule," IEEE Transactions on Systems, Man, and Cybernetics, vol. 6, no. 4, pp. 325–327, 1976.
33. J. Gou, L. Du, Y. Zhang, and T. Xiong, "A new distance-weighted k-nearest neighbor classifier," Journal of Information and Computational Science, vol. 9, no. 6, pp. 1429–1436, 2012.
34. D. Wettschereck and T. G. Dietterich, "An experimental comparison of the nearest-neighbor and nearest-hyperrectangle algorithms," Machine Learning, vol. 19, no. 1, pp. 5–27, 1995.
35. V. García, E. Debreuve, F. Nielsen, and M. Barlaud, "k-nearest neighbor search: fast GPU-based implementations and application to high-dimensional feature matching," in Proc. IEEE Int. Conf. Image Process., 2010, pp. 3757–3760.
36. K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, "When is 'nearest neighbor' meaningful?," in Proc. Int. Conf. Database Theory, 1999, pp. 217–235. Springer.
37. J. Schmidhuber, "Deep learning in neural networks: an overview," Neural Networks, vol. 61, pp. 85–117, 2015.
38. C. M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
39. C. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall, "Activation functions: comparison of trends in practice and research for deep learning," arXiv preprint, arXiv:1811.03378, 2018.
40. L. Bottou, "Large-scale machine learning with stochastic gradient descent," in Proc. COMPSTAT 2010, Springer, 2010, pp. 177–186.
41. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, no. 6088, pp. 533–536, 1986.
42. J. Howard and S. Gugger, "Fastai: a layered API for deep learning," Information, vol. 11, no. 2, p. 108, 2020.
43. L. N. Smith, "A disciplined approach to neural network hyper-parameters: part 1—learning rate, batch size, momentum, and weight decay," arXiv preprint, arXiv:1803.09820, 2018.
44. C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, "A survey on deep transfer learning," in Proc. Int. Conf. Artificial Neural Networks, Springer, 2018, pp. 270–279.
45. J. Howard and S. Gugger, Deep Learning for Coders with fastai and PyTorch: AI Applications without a PhD. O’Reilly Media, 2020.
46. T. G. Dietterich, "Ensemble methods in machine learning," in Proc. Int. Workshop on Multiple Classifier Systems, Springer, 2000, pp. 1–15.
47. L. Breiman, "Random forests," Machine Learning, vol. 45, no. 1, pp. 5–32, 2001.
48. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2009.
49. P. Geurts, D. Ernst, and L. Wehenkel, "Extremely randomized trees," Machine Learning, vol. 63, no. 1, pp. 3–42, 2006. J. H. Friedman, "Greedy function approximation: A gradient boosting machine," Annals of Statistics, vol. 29, no. 5, pp. 1189–1232, 2001.
50. G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, "LightGBM: A highly efficient gradient boosting decision tree," in Advances in Neural Information Processing Systems, vol. 30, 2017.
51. T. Chen and C. Guestrin, "XGBoost: A scalable tree boosting system," in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2016, pp. 785–794.
52. V. Dorogush, V. Ershov, and A. Gulin, "CatBoost: gradient boosting with categorical features support," arXiv preprint, arXiv:1810.11363, 2018.
53. J. Hancock and T. M. Khoshgoftaar, "Survey on categorical data for neural networks," Journal of Big Data, vol. 7, no. 1, pp. 1–41, 2020.
54. P. Joshi, Artificial Intelligence with Python. Packt Publishing Ltd., 2017.
55. A. Kadiyala and A. Kumar, "Applications of Python to evaluate environmental data science problems," Environmental Progress & Sustainable Energy, vol. 36, no. 6, pp. 1580–1586, 2017.
56. D. Rolon-Mérette, M. Ross, T. Rolon-Mérette, and K. Church, "Introduction to Anaconda and Python: installation and setup," Quantitative Methods for Psychology, vol. 16, no. 5, pp. S3–S11, 2016.
57. J. M. Ponce-Ortega, R. Ochoa-Barragán, and C. Ramírez-Márquez, "Optimization using the software Python with Spyder," in Optimization in Chemical Processes: Sustainable Perspectives, Cham: Springer Nature Switzerland, 2024, pp. 465–489.
58. L. Bognár and T. Fauszt, "Factors and conditions that affect the goodness of machine learning models for predicting the success of learning," Computers and Education: Artificial Intelligence, vol. 3, p. 100100, 2022.
59. V. R. Joseph, "Optimal ratio for data splitting," Statistical Analysis and Data Mining: The ASA Data Science Journal, vol. 15, no. 4, pp. 531–538, 2022.
60. B. Bischl, M. Binder, M. Lang, T. Pielok, J. Richter, S. Coors, and M. Lindauer, "Hyperparameter optimization: foundations, algorithms, best practices, and open challenges," Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 13, no. 2, p. e1484, 2023.
61. G. Naidu, T. Zuva, and E. M. Sibanda, "A review of evaluation metrics in machine learning algorithms," in Proc. Computer Science Online Conf., Cham: Springer International Publishing, Apr. 2023, pp. 15–25.
62. D. Chicco, M. J. Warrens, and G. Jurman, "The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation," PeerJ Computer Science, vol. 7, p. e623, 2021.
63. B. Juba and H. S. Le, "Precision-recall versus accuracy and the role of large data sets," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 1, pp. 4039–4048, Jul. 2019.
64. C. Halimu, A. Kasem, and S. S. Newaz, "Empirical comparison of area under ROC curve (AUC) and Mathew correlation coefficient (MCC) for evaluating machine learning algorithms on imbalanced datasets for binary classification," in Proceedings of the 3rd International Conference on Machine Learning and Soft Computing, Jan. 2019, pp. 1–6.
65. W. Zou, W. Zhong, J. Du, and L. Yuan, "Prediction of Student Academic Performance Utilizing a Multi-Model Fusion Approach in the Realm of Machine Learning," Applied Sciences, vol. 15, no. 7, p. 3550, 2025.
66. E. F. Agyemang, J. A. Mensah, O.-A. Ampomah, et al., "Predicting students’ academic performance via machine learning algorithms: An empirical review and practical application," Computer Engineering and Intelligent Systems, vol. 15, no. 1, pp. 86–102, 2024.
67. Y. Rimal and N. Sharma, "Ensemble machine learning prediction accuracy: local vs. global precision and recall for multiclass grade performance of engineering students," Frontiers in Education, vol. 10, 2025.
68. E. Kalita, A. M. Alfarwan, H. El Aouifi, A. Kukkar, S. Hussain, T. Ali, and S. Gaftandzhieva, "Predicting student academic performance using Bi-LSTM: a deep learning framework with SHAP-based interpretability and statistical validation," Frontiers in Education, vol. 10, 2025.