Natural Language Processing and Machine Learning Applications For Assessment and Evaluation in Education: Opportunities and New Approaches

Year 2024, Volume: 15 Issue: 4, 421 - 445, 31.12.2024

Kübra Yılmaz , Kaan Zulfikar Deniz

https://doi.org/10.21031/epod.1551568

Abstract

This study examines the applications of Artificial Intelligence (AI), Machine Learning (ML) and Natural Language Processing (NLP) technologies in education, particularly in educational assessment and evaluation processes. The study examines the potential of these technologies to contribute to educational assessment and evaluation processes in areas such as automatic item generation, text mining, sentiment analysis, sentence similarity, and providing feedback to students. The study includes both a literature review and sample applications. In the automatic item generation process of the study, language models such as GPT and Gemini are used to generate new educational questions and this process is supported by NLP technologies. The study is enriched with Turkish examples and the results show that these applications can be further developed for Turkish and have potential for other applications.

Keywords

machine learning in education, natural language processing in education, artificial intelligence, educational technologies

References

• Adalı, E. (2012). Doğal dil işleme. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, 5(2). https://dergipark.org.tr/tr/pub/tbbmd/issue/22245/238797
• Alghamdi, R., & Alfalqi, K. (2015). A survey of topic modeling in text mining. International Journal of Advanced Computer Science and Applications (IJACSA), 6(1), 147-153.
• Balas, V. E., Kumar, R., & Srivastava, R. (Eds.). (2020). Recent trends and advances in AI and internet of things. Springer Nature. https://doi.org/10.1007/978-3-030-32644-9.
• Başarır, L. (2022). Modelling AI in architectural education. Gazi University Journal of Science, 35(4), 1260-1278. https://doi.org/10.35378/gujs.967981
• Berger, M., Kinsley, A., & Chawla, S. (2024). A novel multi-stage prompting approach for language agnostic MCQ generation using GPT. arXiv preprint arXiv:2401.07098. https://arxiv.org/abs/2401.07098
• Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of ML Research, 3, 993-1022. https://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf
• Boden, M. A. (2018). What is AI? In AI: A Very Short Introduction. Oxford University Press. https://doi.org/10.1093/actrade/9780199602919.003.0001
• Bostancı, B., & Albayrak, A. (2021). Duygu Analizi İle Kişiye Özel İçerik Önermek. Veri Bilimi, 4(1), 53-60. https://dergipark.org.tr/tr/pub/veri/issue/59505/777675#article_cite
• Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., & Amodei, D. (2020). Language models are few-shot learners. arXiv preprint arXiv:2005.14165. https://arxiv.org/abs/2005.14165
• Buchanan, B. G., & Shortliffe, E. H. (1984). Rule-based expert systems. Addison-Wesley.
• Burkov, A. (2019). The hundred-page machine learning book. Andriy Burkov.
• Cavalcanti, A. P., Ferreira Leite de Mello, R., Rolim, V., André, M., Freitas, F., & Gaševic, D. (2019). An analysis of the use of good feedback practices in online learning courses. In 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT) (pp. 153-157). Maceio, Brazil. https://doi.org/10.1109/ICALT.2019.00061
• Cavus, M., & Kuzilek, J. (2024a). An effect analysis of the balancing techniques on the counterfactual explanations of student success prediction models. arXiv preprint arXiv:2408.00676.
• Cavus, M., & Kuzilek, J. (2024b). The Actionable Explanations for Student Success Prediction Models: A Benchmark Study on the Quality of Counterfactual Methods. arXiv preprint arXiv:2405.14016.
• Chamidah, N., Santoni, M. M., Irmanda, H. N., Astriratma, R., Tua, L. M. & Yuniati, T. (2021). Word Expansion using Synonyms in Indonesian Short Essay Auto Scoring. International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS). doi:10.1109/ICIMCIS53775.2021.9699374
• Chapelle, O., Schölkopf, B., & Zien, A. (2006). Semi-Supervised Learning. MIT Press. Retrieved from http://www.acad.bg/ebook/ml/MITPress-%20SemiSupervised%20Learning.pdf
• Chaudhry, M. A., & Kazim, E. (2021). AI in Education (AIEd): a high-level academic and industry note 2021. AI and Ethics, 2(157-165). https://link.springer.com/article/10.1007/s43681-021-00074-z
• Chen, J., Chen, C., & Liang, Z. (2016). Optimized TF-IDF algorithm with the adaptive weight of position of word. In 2nd International Conference on AI and Industrial Engineering (AIIE2016) Advances in Intelligent Systems Research (Vol. 133).
• Coelho, A. M. L., da Silva, H. F., da Silva, L. A. C., Andrade, M. E., & Rodrigues, R. G. da S. (2023). Inteligência artificial: Suas vantagens e limites em cursos à distância. Revista Ilustração, 4(2), 23-27. https://doi.org/10.46550/ilustracao.v4i2.150
• Deary, I. J., Penke, L., & Johnson, W. (2010). The neuroscience of human intelligence differences. Nature Reviews Neuroscience, 11(3), 201-211. https://doi.org/10.1038/nrn2793
• Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for ML. Cambridge University Press. https://mml-book.com
• Dong, J. (2023). NLP pretraining language model for computer intelligent recognition technology. ACM Transactions on Asian and Low-Resource Language Information Processing. Retrieved from : https://dl.acm.org/doi/pdf/10.1145/3605210
• European Commission. (2018). Definition of AI. High-Level Expert Group on AI (AI HLEG). Retrieved from https://ec.europa.eu/futurium/en/system/files/ged/ai_hleg_definition_of_ai_18_december_1.pdf.
• Farouk, M. (2019). Measuring sentences similarity: A survey. Indian Journal of Science and Technology, 12(25). https://doi.org/10.17485/ijst/2019/v12i25/143977
• Ferreira, R., André, M., Pinheiro, A., Costa, E., & Romero, C. (2020). Text Mining in Education. Retrieved from: https://arxiv.org/pdf/2403.00769
• Gierl, M. J., & Lai, H. (2016). A process for reviewing and evaluating generated test items. Educational Measurement: Issues and Practice, 35(4), 6-20. Retrieved from https://doi.org/10.1111/emip.12136.
• Gierl, M. J., Zhou, J., & Alves, C. (2008). Developing a taxonomy of item model types to promote assessment engineering. The Journal of Technology, Learning and Assessment, 7(2).
• Göloğlu Demir, C., & Yılmaz, H. (2018). Sınıf dışı eğitim faaliyetlerinin öğrencilerin bilim ve teknolojiye yönelik tutumlarına etkisi ve duygu analizi. İnsan ve Toplum Bilimleri Araştırmaları Dergisi, 7(5), 101-116. https://doi.org/10.15869/itobiad.483404
• Grun, B., & Hornik, K. (2011). topicmodels: An R package for fitting topic models. Retrieved from https://cran.r-project.org/web/packages/topicmodels/vignettes/topicmodels.pdf
• Guu, H., Hashimoto, T. B., & Oren, Y. (2018). Generating sentences by editing prototypes. Transactions of the Association for Computational Linguistics, 6, 437-450. https://doi.org/10.1162/tacl_a_00030
• Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis Campbell Systematic Reviews, 18, e1230. https://doi.org/10.1002/cl2.1230
• Hamal, O., El Faddouli, N., Alaoui Harouni, M. H., & Lu, J. (2022). AI in Education. Sustainability, 14(2862). https://doi.org/10.3390/su14052862
• Hendler, J. (2008). Avoiding another AI winter. IEEE Intelligent Systems, 23(2), 2-4.
• Holstein, K., McLaren, B. M., & Aleven, V. (2018). Student learning benefits of a mixed-reality teacher awareness tool in AI-enhanced classrooms. AI in Education: 19th International Conference, 154-168. https://doi.org/10.1007/978-3-319-93843-1_12
• Hu, Z., Yang, Z., Shi, H., Tan, B., Zhao, T., He, J., Liang, X., Wang, W., Yu, X., Wang, D., Qin, L., Ma, X., Liu, H., Singh, D., Zhu, W., & Xing, E. P. (2018). Texar: A modularized, versatile, and extensible toolbox for text generation. Proceedings of Workshop for NLP Open Source Software, 13-22. https://doi.org/10.18653/v1/W18-2503
• Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3), 90-95.
• Hutto, C. J., & Gilbert, E. (2014). VADER: A parsimonious rule-based model for sentiment analysis of social media text. In E. Adar & P. Resnick (Eds.), Proceedings of the eighth international AAAI conference on weblogs and social media. Retrieved from https://ojs.aaai.org/index.php/ICWSM/issue/view/274
• İlikhan, S., Özer, M., Tanberkan, H., & Bozkurt, V. (2024). How to mitigate the risks of deployment of AI in medicine? Turkish Journal of Medical Sciences, 54(3), 483-492. https://doi.org/10.55730/1300-0144.5814
• Jurafsky, D., & Martin, J. H. (2024). Speech and language processing: An introduction to NLP, computational linguistics, and speech recognition.
• Kasumba, R., & Neumman, M. (2024). Practical Sentiment Analysis for Education: The Power of Student Crowdsourcing. Proceedings of the AAAI Conference on AI, 38(21), 23110-23118. https://doi.org/10.1609/aaai.v38i21.30356
• Kayalı, B., Balat, Ş., Kurşun, E., & Karaman, S. (2019). Lisansüstü eğitimde etkili ve nitelikli geribildirim. Journal of Instructional Technologies & Teacher Education, 1(8), 10-20.
• Kış, A. (2019). Eğitimde yapay zeka. In 14. Uluslararası Eğitim Yönetimi Kongresi Tam Metin Bildiri Kitabı.
• Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: Reengineering educational pedagogy—Building a learning companion. Proceedings IEEE International Conference on Advanced Learning Technologies, 43-46. https://doi.org/10.1109/ICALT.2001.943850
• Kotlyarova, I. O. (2022). AI technologies in education. Bulletin of the South
• Kühl, N., Goutier, M., Hirt, R., & Satzger, G. (2020). Machine learning in artificial intelligence:
• Towards a common understanding. arXiv:2004.04686 [cs.LG]. https://doi.org/10.48550/arXiv.2004.04686
• LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature.
• Leskovec, J., Rajaraman, A., & Ullman, J. D. (2014). Mining of Massive Datasets. Cambridge University Press. Retrieved from http://infolab.stanford.edu/~ullman/mmds/book0n.pdf
• Lewkow, N., Kode, S., Feild, J., Zimmerman, N., Riedesel, M., Essa, A., Boulanger, D., Seanosky, J., Kumar, V., & Kinshuk. (2016). A scalable learning analytics platform for automated writing feedback. In Proceedings of the Third (2016) ACM Conference on Learning @ Scale - L@S ’16. https://doi.org/10.1145/2876034.2893380
• Liddy, E. D. (2001). NLP. In Encyclopedia of library and information science (2nd ed.). Marcel Decker, Inc. Retrieved from https://surface.syr.edu/cgi/viewcontent.cgi?article=1043&context=istpub
• Lin, F. (2023). Sentiment analysis in online education: An analytical approach and application. Proceedings of the 2023 International Conference on ML and Automation. Retrieved from https://doi.org/10.54254/2755-2721/33/20230225.
• Lipnevich, A. A., & Panadero, E. (2021). A review of feedback models and theories: Descriptions, definitions, and conclusions. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.720195
• Lighthill, J. (1973). Artificial intelligence: A general survey. In Artificial intelligence: A paper symposium (pp. 1-77). Science Research Council. https://www.aiai.ed.ac.uk/events/lighthill1973/lighthill.pdf
• Lu, C., & Cutumisu, M. (2021). Integrating deep learning into an automated feedback generation system for automated essay scoring. Paper presented at the International Conference on Educational Data Mining (EDM). International Educational Data Mining Society. https://files.eric.ed.gov/fulltext/ED615567.pdf
• Lubis, F. F., Mutaqin, A. P., Waskita, D., Sulistyaningtyas, T., Arman, A. A., & Rosmansyah, Y. (2021). Automated short-answer grading using semantic similarity based on word embedding. International Journal of Technology, 12(3), 571-581.
• Mahesh, B. (2018). ML algorithms - A review. International Journal of Science and Research (IJSR), 9(1). https://www.ijsr.net/archive/v9i1/ART20203995.pdf
• Mazidi, K. (2018). Automatic Question Generation From Passages. In A. Gelbukh (Ed.), CICLing 2017, LNCS 10762 (pp. 655-665). Springer. https://doi.org/10.1007/978-3-319-77116-8_49
• Merdun, G., Okçular, E., Altınok, D., & Akkurt, F. (2024). Turkish NLP Resources. GitHub repository. Retrieved July 10, 2024, from https://github.com/agmmnn/turkish-nlp-resources
• McCarthy, J. (2007). What is AI? Computer Science Department, Stanford University. Retrieved from http://www-formal.stanford.edu/jmc/whatisai.pdf
• McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (1956). A proposal for the Dartmouth summer research project on AI. Retrieved from http://jmc.stanford.edu/articles/dartmouth/dartmouth.pdf
• McKinney, W. (2010). Data analysis in Python. Proceedings of the 9th Python in Science Conference (pp. 51-56).
• Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
• Mulianingsih, F., Anwar, K., Shintasiwi, F. A., & Rahma, A. J. (2020). AI dengan Pembentukan Nilai dan Karakter di Bidang Pendidikan. Ijtimaiya: Journal of Social Science Teaching, 4(2), 148-154. Retrieved from http://journal.stainkudus.ac.id/index.php/Ijtimaia
• Mohler, M., & Mihalcea, R. (2009). Text-to-text semantic similarity for automatic short answer grading. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), 567–575. Association for Computational Linguistics.
• Nafea, I. T. (2016). ML in educational technology. In ML- Advanced techniques and emerging applications. IntechOpen. http://dx.doi.org/10.5772/intechopen.72906
• Newell, A., & Simon, H. A. (1956). The logic theory machine. IRE Transactions on Information Theory, 2(3), 61-79.
• Nilsson, N. J. (1984). Shakey the robot. SRI International. Retrieved from https://www.sri.com/publication/artificial-intelligence-pubs/shakey-the-robot-pub/
• Nkechi, A. A., Ojo, A. O., & Eneh, O. A. (2024). Impact of AI in Achieving Quality Education. IntechOpen. https://doi.org/10.5772/intechopen.1004871
• Oflazer, K. (2016). Türkçe ve doğal dil işleme. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi, 5(2). https://dergipark.org.tr/tr/pub/tbbmd/issue/22245/238795
• Oflazer, K., & Saraçlar, M. (2018). Turkish NLP. Springer.
• OpenAI. (2023). GPT-4 Turbo and GPT-4. OpenAI. https://platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4
• Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., … & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf
• Peña-Torres, J. A. (2024). Towards an improved teaching practice using sentiment analysis in student evaluation. Ingeniería y Competitividad, 26(2), e-21013759. Retrieved from: https://www.researchgate.net/publication/381598280_Towards_an_improved_of_teaching_practice_using_Sentiment_Analysis_in_Student_Evaluation.
• Perveen, A. (2021). Use of word clouds for task-based assessment in asynchronous e-language learning. MEXTESOL Journal, 45(2).
• Ramachandran, D., & Rana, R. S. (2024). AI for legal system: Jurisprudence in the digital age. International Journal of Advanced Academic Studies, 6(5), 03-13. https://doi.org/10.33545/27068919.2024.v6.i5a.1158
• Russell, S., & Norvig, P. (2010). AI: A Modern Approach (3rd ed.). Pearson.
• Sadiku, M. N. O., Ashaolu, T. J., Ajayi-Majebi, A., & Musa, S. M. (2021). AI in education. International Journal of Scientific Advances, 2(1), 1-11. https://typeset.io/pdf/artificial-intelligence-in-education-5ggabmq2kf.pdf
• Sak, H., Güngör, T. & Saraçlar, M. (2011) Resources for Turkish morphological processing. Lang Resources & Evaluation 45, 249–261. https://doi.org/10.1007/s10579-010-9128-6
• Shalev-Shwartz, S., & Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781107298019
• Sheikh, H., Prins, C., & Schrijvers, E. (2023). Mission AI: The new system technology. Springer Nature Switzerland AG. https://doi.org/10.1007/978-3-031-21448-6
• Shin, E. (2021). Automated item generation by combining the non-template and template- based approaches to generate reading inference test items (Doctoral dissertation, University of Alberta). Department of Educational Psychology.
• Singley, M. K., & Bennett, R. E. (2002). Item generation and beyond: Applications of schema theory to mathematics assessment. In S. H. Irvine & P. C. Kyllonen (Eds.), Item generation for test development (pp. 361–384). Routledge. https://doi.org/10.4324/9781410602145
• Silge, J., & Robinson, D. (2017). Text mining with R: A tidy approach. O’Reilly Media.
• Smith, B. I., Chimedza, C., & Bührmann, J. H. (2022). Individualized help for at-risk students using model-agnostic and counterfactual explanations. Educational and Information Technologies, 27(2), 1539–1558. https://doi.org/10.1007/s10639-021-10661-6
• Smith, J., Li, H., & Patel, R. (2024). Automated generation of multiple-choice cloze questions for assessing English vocabulary using GPT-turbo 3.5. arXiv preprint arXiv:2403.02078. https://arxiv.org/abs/2403.02078
• Sukmana, R., & Rusydiana, A. S. (2023). Social media sentiment analysis on waqf and education. Islamic Marketing Review, 2(2). Retrieved from http://journals.smartinsight.id/index.php/IMR
• Sytnyk, L., & Podlinyayeva, O. (2024). AI in education: Main possibilities and challenges. In Proceedings of the 8th International Scientific and Practical Conference "International Scientific Discussion: Problems, Tasks and Prospects" (pp. 569-579). Brighton, United Kingdom. https://doi.org/10.51582/interconf.19-20.05.2024.058
• Şeker, S. E. (2015). Metin madenciliği (Text mining). YBS Ansiklopedi, 2(3). https://ybsansiklopedi.com/wp-content/uploads/2015/08/MetinMadenciligi30_32.pdf
• Tsiakmaki, M., & Ragos, O. (2021). A case study of interpretable counterfactual explanations for the task of predicting student academic performance. 2021 25th International Conference on Circuits, Systems, Communications and Computers (CSCC), 120-125. https://doi.org/10.1109/CSCC53858.2021.00029
• Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433-460. Retrieved From: https://phil415.pbworks.com/f/TuringComputing.pdf
• Ural State University. Ser. Education. Educational Sciences, 14(3), 69-82. https://vestnik.susu.ru/ped/article/view/12330
• Usta, Y. (2024). Awesome Turkish NLP. GitHub repository. Retrieved July 10, 2024, from https://github.com/yusufusta/awesome-turkish-nlp
• Uysal, İ. (2019). Açık uçlu maddelerde otomatik puanlamanın güvenirliği ve test eşitleme hatalarına etkisi (Doktora tezi, Hacettepe Üniversitesi Eğitim Bilimleri Enstitüsü). YÖK Ulusal Tez Merkezi. https://tez.yok.gov.tr/UlusalTezMerkezi/tezDetay.jsp?id=55GArTnn6vLwQ3HOxnwo_w&no=gj27xzLBIdoGFgSUJzjT6Q
• Wang, J., & Dong, Y. (2020). Measurement of Text Similarity: A Survey. Information, 11(9), 421. doi:10.3390/info11090421
• Waskom, M. (2020). Seaborn: Statistical data visualization. Erişim adresi: https://seaborn.pydata.org
• Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36-45.
• Wijeratne, Y., Silva, N., & Shanmugarajah, Y. (2009). NLP for government: Problems and potential. Retrieved from https://lirneasia.net/wp-content/uploads/2019/04/Natural_Language_Processing_for_Government__Problems_and_Potential.pdf
• Woods, B., Adamson, D., Miel, S., & Mayfield, E. (2017). Beyond Automated Essay Scoring: Forecasting and Improving Outcomes in Middle and High School Writing. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '17) (pp. 2071-2080). ACM. https://doi.org/10.1145/3097983.3098160
• Yang, A., & Halim, S. (2022). Natural language generation using ML techniques. Journal of Student Research, 11(2). Retrieved from https://typeset.io/papers/natural-language-generation-using-machine-learning-1itplfnn.
• Yıldırım, S., & Yıldız, T. (2018). A Comparison of Different Approaches to Document Representation in Turkish Language. Journal of Natural and Applied Sciences, 22(2), 569-576. https://doi.org/10.19113/sdufbed.15893
• Young, J. (2024). The rise of AI in education. International Journal of Innovative Research & Development, 13(2), 74. https://www.internationaljournalcorner.com/index.php/ijird_ojs/article/view/173518/118319
• Zeinalipour, K., Keptiğ, Y. G., Maggini, M., & Gori, M. (2024). Automating Turkish educational quiz generation using large language models. https://doi.org/10.48550/arXiv.2406.03397
• Zeyrek, A. (2020). Zeyrek: Morphological analysis for Turkish. GitHub. https://github.com/ahmetaa/zeyrek