Assisting tool for essay grading for Turkish language instructors

Year 2019, Volume: 7 Issue: 2, 141 - 146, 25.12.2019

Mustafa Alp Çetin , Rita Ismailova

Abstract

When learning languages, writing an essay is one of the main methods for assessing students’ knowledge. However, with the development of ICT, language learning is also being transferred to online platforms. At the same time, as the number of students’ increases, the problem of evaluating students’ essays arises. In this paper, we offer an automated system that facilitates instructors while evaluating students’ essays. Currently, the system works for essays written in Turkish. The system was built using the Zemberek library. It allows one to extract text features the essay of several people at the same time on several indicators, namely, morphological analysis, vocabulary, the use of different language structures, etc. Currently, many automated essay grading tools are proposed, and one of the main factors that defined their accuracy it the extraction of text features. Thus, as further work, it is planned to use the data obtained using this essay assessment system together with instructors’ evaluation to create an expert system for automatic essay evaluation using machine learning techniques.

Keywords

Natural language processing, Automated tool, Automatic essay evaluation, computer assisted language learning

References

• [1] Akın, A.A. and Akın, M.D. (2007). Zemberek, an open source NLP framework for Turkic languages. Structure, 10, pp.1-5.
• [2] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
• [3] Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python: analyzing text with the natural language toolkit. Sebastopol: O'Reilly Media, Inc.
• [4] Brants, T., Popat, A. C., Xu, P., Och, F. J., & Dean, J. (2007, June). Large language models in machine translation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL) (pp. 858-867).
• [5] Brown, P. F., Desouza, P. V., Mercer, R. L., Pietra, V. J. D., & Lai, J. C. (1992). Class-based n-gram models of natural language. Computational linguistics, 18(4), 467-479.
• [6] Burstein, J., Chodorow, M., & Leacock, C. (2003, August). CriterionSM Online Essay Evaluation: An Application for Automated Evaluation of Student Essays. In IAAI (pp. 3-10).
• [7] Cavnar, W. B., & Trenkle, J. M. (1994, April). N-gram-based text categorization. In Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval (Vol. 161175).
• [8] Chowdhury, G.G. (2003). Natural language processing. Annual review of information science and technology, 37(1), pp.51-89.
• [9] Collobert, R., & Weston, J. (2008, July). A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning. ACM, pp. 160-167.
• [10] Goyal, A., Jagarlamudi, J., Daumé III, H., & Venkatasubramanian, S. (2010, June). Sketching techniques for large scale NLP. In Proceedings of the NAACL HLT 2010 Sixth Web as Corpus Workshop (pp. 17-25). Association for Computational Linguistics.
• [11] Habash, N.Y. (2010). Introduction to Arabic natural language processing. Synthesis Lectures on Human Language Technologies, 3(1), pp.1-187.
• [12] Khreisat, L. (2009). A machine learning approach for Arabic text classification using N-gram frequency statistics. Journal of Informetrics, 3(1), 72-77.
• [13] Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., ... & Dyer, C. (2007, June). Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting of the association for computational linguistics companion volume proceedings of the demo and poster sessions (pp. 177-180).
• [14] Page, E. B. (1966). The imminence of... grading essays by computer. The Phi Delta Kappan, 47(5), 238-243.
• [15] Page, E. B. (1967). Grading essays by computer: Progress report. In Proceedings of the Invitational Conference on Testing Problems.
• [16] Page, E. B. (1994). Computer grading of student prose, using modern concepts and software. The Journal of experimental education, 62(2), 127-142.
• [17] Page, E. B., Poggio, J. P., & Keith, T. Z. (1997, March). Computer analysis ofstudent essays: Finding trait differences in the studentprofile. Paper presented at the annual meeting ofthe American Educational Research Association, Chicago
• [18] Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002, July). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 311-318). Association for Computational Linguistics.
• [19] Schraudner, M. (2014). The online teacher’s assistant: Using automated correction programs to supplement learning and lesson planning. CELE Journal, 22, 128-140.
• [20] Shermis, M. D., Koch, C. M., Page, E. B., Keith, T. Z., & Harrington, S. (2002). Trait ratings for automated essay grading. Educational and Psychological Measurement, 62(1), 5-18.
• [21] Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., & Tsujii, J. I. (2012, April). BRAT: a web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (pp. 102-107). Association for Computational Linguistics.
• [22] Taghipour, K., & Ng, H. T. (2016, November). A neural approach to automated essay scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 1882-1891).
• [23] Tyers, F.M. and Alperen, M.S. (2010). South-east European times: A parallel corpus of Balkan languages. In Proceedings of the LREC Workshop on Exploitation of Multilingual Resources and Tools for Central and (South-) Eastern European Languages (pp. 49-53).
• [24] Van Ewijk, R. (2011). Same work, lower grade? Student ethnicity and teachers’ subjective assessments. Economics of Education Review, 30(5), 1045-1058.
• [25] Zupanc, K., & Bosnić, Z. (2017). Automated essay evaluation with semantic analysis. Knowledge-Based Systems, 120, 118-132.