Natural Language Processing and Machine Learning Applications For Assessment and Evaluation in Education: Opportunities and New Approaches

Abstract

This study examines the applications of Artificial Intelligence (AI), Machine Learning (ML) and Natural Language Processing (NLP) technologies in education, particularly in educational assessment and evaluation processes. The study examines the potential of these technologies to contribute to educational assessment and evaluation processes in areas such as automatic item generation, text mining, sentiment analysis, sentence similarity, and providing feedback to students. The study includes both a literature review and sample applications. In the automatic item generation process of the study, language models such as GPT and Gemini are used to generate new educational questions and this process is supported by NLP technologies. The study is enriched with Turkish examples and the results show that these applications can be further developed for Turkish and have potential for other applications.

Keywords

• machine learning in education
• natural language processing in education
• artificial intelligence
• educational technologies

References

1. Adalı, E. (2012). Doğal dil işleme. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, 5(2). https://dergipark.org.tr/tr/pub/tbbmd/issue/22245/238797
2. Alghamdi, R., & Alfalqi, K. (2015). A survey of topic modeling in text mining. International Journal of Advanced Computer Science and Applications (IJACSA), 6(1), 147-153.
3. Balas, V. E., Kumar, R., & Srivastava, R. (Eds.). (2020). Recent trends and advances in AI and internet of things. Springer Nature. https://doi.org/10.1007/978-3-030-32644-9.
4. Başarır, L. (2022). Modelling AI in architectural education. Gazi University Journal of Science, 35(4), 1260-1278. https://doi.org/10.35378/gujs.967981
5. Berger, M., Kinsley, A., & Chawla, S. (2024). A novel multi-stage prompting approach for language agnostic MCQ generation using GPT. arXiv preprint arXiv:2401.07098. https://arxiv.org/abs/2401.07098
6. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of ML Research, 3, 993-1022. https://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf
7. Boden, M. A. (2018). What is AI? In AI: A Very Short Introduction. Oxford University Press. https://doi.org/10.1093/actrade/9780199602919.003.0001
8. Bostancı, B., & Albayrak, A. (2021). Duygu Analizi İle Kişiye Özel İçerik Önermek. Veri Bilimi, 4(1), 53-60. https://dergipark.org.tr/tr/pub/veri/issue/59505/777675#article_cite

9. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., & Amodei, D. (2020). Language models are few-shot learners. arXiv preprint arXiv:2005.14165. https://arxiv.org/abs/2005.14165
10. Buchanan, B. G., & Shortliffe, E. H. (1984). Rule-based expert systems. Addison-Wesley.
11. Burkov, A. (2019). The hundred-page machine learning book. Andriy Burkov.
12. Cavalcanti, A. P., Ferreira Leite de Mello, R., Rolim, V., André, M., Freitas, F., & Gaševic, D. (2019). An analysis of the use of good feedback practices in online learning courses. In 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT) (pp. 153-157). Maceio, Brazil. https://doi.org/10.1109/ICALT.2019.00061
13. Cavus, M., & Kuzilek, J. (2024a). An effect analysis of the balancing techniques on the counterfactual explanations of student success prediction models. arXiv preprint arXiv:2408.00676.
14. Cavus, M., & Kuzilek, J. (2024b). The Actionable Explanations for Student Success Prediction Models: A Benchmark Study on the Quality of Counterfactual Methods. arXiv preprint arXiv:2405.14016.
15. Chamidah, N., Santoni, M. M., Irmanda, H. N., Astriratma, R., Tua, L. M. & Yuniati, T. (2021). Word Expansion using Synonyms in Indonesian Short Essay Auto Scoring. International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS). doi:10.1109/ICIMCIS53775.2021.9699374
16. Chapelle, O., Schölkopf, B., & Zien, A. (2006). Semi-Supervised Learning. MIT Press. Retrieved from http://www.acad.bg/ebook/ml/MITPress-%20SemiSupervised%20Learning.pdf
17. Chaudhry, M. A., & Kazim, E. (2021). AI in Education (AIEd): a high-level academic and industry note 2021. AI and Ethics, 2(157-165). https://link.springer.com/article/10.1007/s43681-021-00074-z
18. Chen, J., Chen, C., & Liang, Z. (2016). Optimized TF-IDF algorithm with the adaptive weight of position of word. In 2nd International Conference on AI and Industrial Engineering (AIIE2016) Advances in Intelligent Systems Research (Vol. 133).
19. Coelho, A. M. L., da Silva, H. F., da Silva, L. A. C., Andrade, M. E., & Rodrigues, R. G. da S. (2023). Inteligência artificial: Suas vantagens e limites em cursos à distância. Revista Ilustração, 4(2), 23-27. https://doi.org/10.46550/ilustracao.v4i2.150
20. Deary, I. J., Penke, L., & Johnson, W. (2010). The neuroscience of human intelligence differences. Nature Reviews Neuroscience, 11(3), 201-211. https://doi.org/10.1038/nrn2793
21. Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for ML. Cambridge University Press. https://mml-book.com
22. Dong, J. (2023). NLP pretraining language model for computer intelligent recognition technology. ACM Transactions on Asian and Low-Resource Language Information Processing. Retrieved from : https://dl.acm.org/doi/pdf/10.1145/3605210
23. European Commission. (2018). Definition of AI. High-Level Expert Group on AI (AI HLEG). Retrieved from https://ec.europa.eu/futurium/en/system/files/ged/ai_hleg_definition_of_ai_18_december_1.pdf.
24. Farouk, M. (2019). Measuring sentences similarity: A survey. Indian Journal of Science and Technology, 12(25). https://doi.org/10.17485/ijst/2019/v12i25/143977
25. Ferreira, R., André, M., Pinheiro, A., Costa, E., & Romero, C. (2020). Text Mining in Education. Retrieved from: https://arxiv.org/pdf/2403.00769
26. Gierl, M. J., & Lai, H. (2016). A process for reviewing and evaluating generated test items. Educational Measurement: Issues and Practice, 35(4), 6-20. Retrieved from https://doi.org/10.1111/emip.12136.
27. Gierl, M. J., Zhou, J., & Alves, C. (2008). Developing a taxonomy of item model types to promote assessment engineering. The Journal of Technology, Learning and Assessment, 7(2).
28. Göloğlu Demir, C., & Yılmaz, H. (2018). Sınıf dışı eğitim faaliyetlerinin öğrencilerin bilim ve teknolojiye yönelik tutumlarına etkisi ve duygu analizi. İnsan ve Toplum Bilimleri Araştırmaları Dergisi, 7(5), 101-116. https://doi.org/10.15869/itobiad.483404
29. Grun, B., & Hornik, K. (2011). topicmodels: An R package for fitting topic models. Retrieved from https://cran.r-project.org/web/packages/topicmodels/vignettes/topicmodels.pdf
30. Guu, H., Hashimoto, T. B., & Oren, Y. (2018). Generating sentences by editing prototypes. Transactions of the Association for Computational Linguistics, 6, 437-450. https://doi.org/10.1162/tacl_a_00030
31. Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis Campbell Systematic Reviews, 18, e1230. https://doi.org/10.1002/cl2.1230
32. Hamal, O., El Faddouli, N., Alaoui Harouni, M. H., & Lu, J. (2022). AI in Education. Sustainability, 14(2862). https://doi.org/10.3390/su14052862
33. Hendler, J. (2008). Avoiding another AI winter. IEEE Intelligent Systems, 23(2), 2-4.
34. Holstein, K., McLaren, B. M., & Aleven, V. (2018). Student learning benefits of a mixed-reality teacher awareness tool in AI-enhanced classrooms. AI in Education: 19th International Conference, 154-168. https://doi.org/10.1007/978-3-319-93843-1_12
35. Hu, Z., Yang, Z., Shi, H., Tan, B., Zhao, T., He, J., Liang, X., Wang, W., Yu, X., Wang, D., Qin, L., Ma, X., Liu, H., Singh, D., Zhu, W., & Xing, E. P. (2018). Texar: A modularized, versatile, and extensible toolbox for text generation. Proceedings of Workshop for NLP Open Source Software, 13-22. https://doi.org/10.18653/v1/W18-2503
36. Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3), 90-95.
37. Hutto, C. J., & Gilbert, E. (2014). VADER: A parsimonious rule-based model for sentiment analysis of social media text. In E. Adar & P. Resnick (Eds.), Proceedings of the eighth international AAAI conference on weblogs and social media. Retrieved from https://ojs.aaai.org/index.php/ICWSM/issue/view/274
38. İlikhan, S., Özer, M., Tanberkan, H., & Bozkurt, V. (2024). How to mitigate the risks of deployment of AI in medicine? Turkish Journal of Medical Sciences, 54(3), 483-492. https://doi.org/10.55730/1300-0144.5814
39. Jurafsky, D., & Martin, J. H. (2024). Speech and language processing: An introduction to NLP, computational linguistics, and speech recognition.
40. Kasumba, R., & Neumman, M. (2024). Practical Sentiment Analysis for Education: The Power of Student Crowdsourcing. Proceedings of the AAAI Conference on AI, 38(21), 23110-23118. https://doi.org/10.1609/aaai.v38i21.30356
41. Kayalı, B., Balat, Ş., Kurşun, E., & Karaman, S. (2019). Lisansüstü eğitimde etkili ve nitelikli geribildirim. Journal of Instructional Technologies & Teacher Education, 1(8), 10-20.
42. Kış, A. (2019). Eğitimde yapay zeka. In 14. Uluslararası Eğitim Yönetimi Kongresi Tam Metin Bildiri Kitabı.
43. Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: Reengineering educational pedagogy—Building a learning companion. Proceedings IEEE International Conference on Advanced Learning Technologies, 43-46. https://doi.org/10.1109/ICALT.2001.943850
44. Kotlyarova, I. O. (2022). AI technologies in education. Bulletin of the South
45. Kühl, N., Goutier, M., Hirt, R., & Satzger, G. (2020). Machine learning in artificial intelligence:
46. Towards a common understanding. arXiv:2004.04686 [cs.LG]. https://doi.org/10.48550/arXiv.2004.04686
47. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature.
48. Leskovec, J., Rajaraman, A., & Ullman, J. D. (2014). Mining of Massive Datasets. Cambridge University Press. Retrieved from http://infolab.stanford.edu/~ullman/mmds/book0n.pdf
49. Lewkow, N., Kode, S., Feild, J., Zimmerman, N., Riedesel, M., Essa, A., Boulanger, D., Seanosky, J., Kumar, V., & Kinshuk. (2016). A scalable learning analytics platform for automated writing feedback. In Proceedings of the Third (2016) ACM Conference on Learning @ Scale - L@S ’16. https://doi.org/10.1145/2876034.2893380
50. Liddy, E. D. (2001). NLP. In Encyclopedia of library and information science (2nd ed.). Marcel Decker, Inc. Retrieved from https://surface.syr.edu/cgi/viewcontent.cgi?article=1043&context=istpub
51. Lin, F. (2023). Sentiment analysis in online education: An analytical approach and application. Proceedings of the 2023 International Conference on ML and Automation. Retrieved from https://doi.org/10.54254/2755-2721/33/20230225.
52. Lipnevich, A. A., & Panadero, E. (2021). A review of feedback models and theories: Descriptions, definitions, and conclusions. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.720195
53. Lighthill, J. (1973). Artificial intelligence: A general survey. In Artificial intelligence: A paper symposium (pp. 1-77). Science Research Council. https://www.aiai.ed.ac.uk/events/lighthill1973/lighthill.pdf
54. Lu, C., & Cutumisu, M. (2021). Integrating deep learning into an automated feedback generation system for automated essay scoring. Paper presented at the International Conference on Educational Data Mining (EDM). International Educational Data Mining Society. https://files.eric.ed.gov/fulltext/ED615567.pdf
55. Lubis, F. F., Mutaqin, A. P., Waskita, D., Sulistyaningtyas, T., Arman, A. A., & Rosmansyah, Y. (2021). Automated short-answer grading using semantic similarity based on word embedding. International Journal of Technology, 12(3), 571-581.
56. Mahesh, B. (2018). ML algorithms - A review. International Journal of Science and Research (IJSR), 9(1). https://www.ijsr.net/archive/v9i1/ART20203995.pdf
57. Mazidi, K. (2018). Automatic Question Generation From Passages. In A. Gelbukh (Ed.), CICLing 2017, LNCS 10762 (pp. 655-665). Springer. https://doi.org/10.1007/978-3-319-77116-8_49
58. Merdun, G., Okçular, E., Altınok, D., & Akkurt, F. (2024). Turkish NLP Resources. GitHub repository. Retrieved July 10, 2024, from https://github.com/agmmnn/turkish-nlp-resources
59. McCarthy, J. (2007). What is AI? Computer Science Department, Stanford University. Retrieved from http://www-formal.stanford.edu/jmc/whatisai.pdf
60. McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (1956). A proposal for the Dartmouth summer research project on AI. Retrieved from http://jmc.stanford.edu/articles/dartmouth/dartmouth.pdf
61. McKinney, W. (2010). Data analysis in Python. Proceedings of the 9th Python in Science Conference (pp. 51-56).
62. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
63. Mulianingsih, F., Anwar, K., Shintasiwi, F. A., & Rahma, A. J. (2020). AI dengan Pembentukan Nilai dan Karakter di Bidang Pendidikan. Ijtimaiya: Journal of Social Science Teaching, 4(2), 148-154. Retrieved from http://journal.stainkudus.ac.id/index.php/Ijtimaia
64. Mohler, M., & Mihalcea, R. (2009). Text-to-text semantic similarity for automatic short answer grading. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), 567–575. Association for Computational Linguistics.
65. Nafea, I. T. (2016). ML in educational technology. In ML- Advanced techniques and emerging applications. IntechOpen. http://dx.doi.org/10.5772/intechopen.72906
66. Newell, A., & Simon, H. A. (1956). The logic theory machine. IRE Transactions on Information Theory, 2(3), 61-79.
67. Nilsson, N. J. (1984). Shakey the robot. SRI International. Retrieved from https://www.sri.com/publication/artificial-intelligence-pubs/shakey-the-robot-pub/
68. Nkechi, A. A., Ojo, A. O., & Eneh, O. A. (2024). Impact of AI in Achieving Quality Education. IntechOpen. https://doi.org/10.5772/intechopen.1004871
69. Oflazer, K. (2016). Türkçe ve doğal dil işleme. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, 5(2). https://dergipark.org.tr/tr/pub/tbbmd/issue/22245/238795
70. Oflazer, K., & Saraçlar, M. (2018). Turkish NLP. Springer.
71. OpenAI. (2023). GPT-4 Turbo and GPT-4. OpenAI. https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4
72. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., … & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf
73. Peña-Torres, J. A. (2024). Towards an improved teaching practice using sentiment analysis in student evaluation. Ingeniería y Competitividad, 26(2), e-21013759. Retrieved from: https://www.researchgate.net/publication/381598280_Towards_an_improved_of_teaching_practice_using_Sentiment_Analysis_in_Student_Evaluation.
74. Perveen, A. (2021). Use of word clouds for task-based assessment in asynchronous e-language learning. MEXTESOL Journal, 45(2).
75. Ramachandran, D., & Rana, R. S. (2024). AI for legal system: Jurisprudence in the digital age. International Journal of Advanced Academic Studies, 6(5), 03-13. https://doi.org/10.33545/27068919.2024.v6.i5a.1158
76. Russell, S., & Norvig, P. (2010). AI: A Modern Approach (3rd ed.). Pearson.
77. Sadiku, M. N. O., Ashaolu, T. J., Ajayi-Majebi, A., & Musa, S. M. (2021). AI in education. International Journal of Scientific Advances, 2(1), 1-11. https://typeset.io/pdf/artificial-intelligence-in-education-5ggabmq2kf.pdf
78. Sak, H., Güngör, T. & Saraçlar, M. (2011) Resources for Turkish morphological processing. Lang Resources & Evaluation 45, 249–261. https://doi.org/10.1007/s10579-010-9128-6
79. Shalev-Shwartz, S., & Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781107298019
80. Sheikh, H., Prins, C., & Schrijvers, E. (2023). Mission AI: The new system technology. Springer Nature Switzerland AG. https://doi.org/10.1007/978-3-031-21448-6
81. Shin, E. (2021). Automated item generation by combining the non-template and template- based approaches to generate reading inference test items (Doctoral dissertation, University of Alberta). Department of Educational Psychology.
82. Singley, M. K., & Bennett, R. E. (2002). Item generation and beyond: Applications of schema theory to mathematics assessment. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 361–384). Routledge. https://doi.org/10.4324/9781410602145
83. Silge, J., & Robinson, D. (2017). Text mining with R: A tidy approach. O’Reilly Media.
84. Smith, B. I., Chimedza, C., & Bührmann, J. H. (2022). Individualized help for at-risk students using model-agnostic and counterfactual explanations. Educational and Information Technologies, 27(2), 1539–1558. https://doi.org/10.1007/s10639-021-10661-6
85. Smith, J., Li, H., & Patel, R. (2024). Automated generation of multiple-choice cloze questions for assessing English vocabulary using GPT-turbo 3.5. arXiv preprint arXiv:2403.02078. https://arxiv.org/abs/2403.02078
86. Sukmana, R., & Rusydiana, A. S. (2023). Social media sentiment analysis on waqf and education. Islamic Marketing Review, 2(2). Retrieved from http://journals.smartinsight.id/index.php/IMR
87. Sytnyk, L., & Podlinyayeva, O. (2024). AI in education: Main possibilities and challenges. In Proceedings of the 8th International Scientific and Practical Conference "International Scientific Discussion: Problems, Tasks and Prospects" (pp. 569-579). Brighton, United Kingdom. https://doi.org/10.51582/interconf.19-20.05.2024.058
88. Şeker, S. E. (2015). Metin madenciliği (Text mining). YBS Ansiklopedi, 2(3). https://ybsansiklopedi.com/wp-content/uploads/2015/08/MetinMadenciligi30_32.pdf
89. Tsiakmaki, M., & Ragos, O. (2021). A case study of interpretable counterfactual explanations for the task of predicting student academic performance. 2021 25th International Conference on Circuits, Systems, Communications and Computers (CSCC), 120-125. https://doi.org/10.1109/CSCC53858.2021.00029
90. Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433-460. Retrieved From: https://phil415.pbworks.com/f/TuringComputing.pdf
91. Ural State University. Ser. Education. Educational Sciences, 14(3), 69-82. https://vestnik.susu.ru/ped/article/view/12330
92. Usta, Y. (2024). Awesome Turkish NLP. GitHub repository. Retrieved July 10, 2024, from https://github.com/yusufusta/awesome-turkish-nlp
93. Uysal, İ. (2019). Açık uçlu maddelerde otomatik puanlamanın güvenirliği ve test eşitleme hatalarına etkisi (Doktora tezi, Hacettepe Üniversitesi Eğitim Bilimleri Enstitüsü). YÖK Ulusal Tez Merkezi. https://tez.yok.gov.tr/UlusalTezMerkezi/tezDetay.jsp?id=55GArTnn6vLwQ3HOxnwo_w&no=gj27xzLBIdoGFgSUJzjT6Q
94. Wang, J., & Dong, Y. (2020). Measurement of Text Similarity: A Survey. Information, 11(9), 421. doi:10.3390/info11090421
95. Waskom, M. (2020). Seaborn: Statistical data visualization. Erişim adresi: https://seaborn.pydata.org
96. Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36-45.
97. Wijeratne, Y., Silva, N., & Shanmugarajah, Y. (2009). NLP for government: Problems and potential. Retrieved from https://lirneasia.net/wp-content/uploads/2019/04/Natural_Language_Processing_for_Government__Problems_and_Potential.pdf
98. Woods, B., Adamson, D., Miel, S., & Mayfield, E. (2017). Beyond Automated Essay Scoring: Forecasting and Improving Outcomes in Middle and High School Writing. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '17) (pp. 2071-2080). ACM. https://doi.org/10.1145/3097983.3098160
99. Yang, A., & Halim, S. (2022). Natural language generation using ML techniques. Journal of Student Research, 11(2). Retrieved from https://typeset.io/papers/natural-language-generation-using-machine-learning-1itplfnn.
100. Yıldırım, S., & Yıldız, T. (2018). A Comparison of Different Approaches to Document Representation in Turkish Language. Journal of Natural and Applied Sciences, 22(2), 569-576. https://doi.org/10.19113/sdufbed.15893
101. Young, J. (2024). The rise of AI in education. International Journal of Innovative Research & Development, 13(2), 74. https://www.internationaljournalcorner.com/index.php/ijird_ojs/article/view/173518/118319
102. Zeinalipour, K., Keptiğ, Y. G., Maggini, M., & Gori, M. (2024). Automating Turkish educational quiz generation using large language models. https://doi.org/10.48550/arXiv.2406.03397
103. Zeyrek, A. (2020). Zeyrek: Morphological analysis for Turkish. GitHub. https://github.com/ahmetaa/zeyrek