Use of Open-Ended Questions in Measurement and Evaluation Methods in Distance Education

Year 2018, Volume: 2 Issue: 1, 1 - 8, 30.06.2018

İbrahim Benli , Rita İsmailova

Abstract

In the 21st century, which is characterized as the Information Age, information access, knowledge quick learning is vital to the development of individuals and societies. With the use of technological innovations in the field of education in the information society, it will be possible to acquire a lasting place in the globalizing world. Distance Education refers to a model of the education system that students and teachers carry out through their learning and teaching activities by communicating via technologies and postal services. According to the year 2015 data, 68 out of 184 higher education institutions in Turkey offer an open and distance learning program. 47 of them are undergraduate, 17 are graduate, 11 are graduate completion and 56 are in master degree level. In total there are 505 different programs. The measurement and evaluation process of results of training is as important as developing content in distance education applications. When question types used in Distance Education Measurement and Assessment are examined, the use of Open-ended Questions is less than other methods. However, it is well-known fact that these type of questions are good predictors of the students’ knowledge. The biggest problem that arises with the usage of open-ended questions is the evaluation part. The different answers that students will give to the questions, their personal narrative skills, or the interpretations they will answer in response to the questions make the evaluation process difficult. At this point, it would be better to interpret the answers recorded in the database with Text Mining methods and Natural Language Processing techniques. In this work, we implement an algorithm for evaluation of open-ended question. The experimental results showed that a correlation was found between 0,89 - 0,96 when evaluating open-ended questions of our system by teacher evaluation.

Keywords

Distance Education, Open Ended Questions, Natural Language Processing

References

• Akın, A. A., & Akın, M. D. (2007). Zemberek, an open source nlp framework for turkic languages. Structure, 10, 1–5. Balta, Y., & Türel, Y. K. (2013). Çevrimiçi uzaktan eğitimde kullanılan farklı ölçme değerlendirme yaklaşımlarına ilişkin bir inceleme. Electronic Turkish Studies, 8(3),37-45. Çallı, İ., İşman, A., & Torkul, O. (2002). Sakarya üniversitesi’nde uzaktan eğitimin dünü bugünü ve geleceği. Sakarya Üniversitesi Eğitim Fakültesi Dergisi, 3,1-7. Carro, R. M., Pulido, E., & Rodríguez, P. (2000). Adaptive internet-based learning with the Tangow system: Computers and Education in the 21st Century (pp. 127–135). Dordrecht:Springer. doi:10.1007/0-306- 47532-4_12 Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391. International Technology and Education Journal Vol. 2, No. 1; June 2018 8 Dessus, P., Lemaire, B., & Vernier, A. (2000). Free-text assessment in a virtual campus. Proc. 3rd International Conference on Human System Learning (CAPS’3) (pp. 61–76). Paris, France:Learning's W.W.W. Epignosis, L. L. C. (2014). E-learning concepts, trends, applications. [Version 1.1]. Retrieved from https://www.talentlms.com/elearning/elearning-101-jan2014-v1.1.pdf Foltz, P. W., Kintsch, W., & Landauer, T. K. (1998). The measurement of textual coherence with latent semantic analysis. Discourse Processes, 25(2–3), 285–307.doi: 10.1080/01638539809545029 Foltz, P. W., Laham, D., & Landauer, T. K. (1999). Automated essay scoring: Applications to educational technology. EdMedia: World Conference on Educational Media and Technology (pp. 939–944). Seattle, WA USA:Association for the Advancement of Computing in Education (AACE). Gelbal, S., & Kelecioğlu, H. (2007). Öğretmenlerin ölçme ve değerlendirme yöntemleri hakkındaki yeterlik algıları ve karşılaştıkları sorunlar. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 33, 135–145. Hotho, A., Nurnberger, A., & Paas, G. (2005). A brief survey of text mining. LDV Forum-GLDV Journal for Computational Linguistics and Language Technology, 20(1),19–62. Internet World Stats. (2017). Internet usage statıstıcs the ınternet big picture world ınternet users and 2017 population stats. Retrieved November 24, 2017, from https://www.internetworldstats.com/stats.htm Islam, A., & Inkpen, D. (2008). Semantic text similarity using corpus-based word similarity and string similarity. ACM Transactions on Knowledge Discovery from Data (TKDD), 2(2), 10. doi: 10.1145/1376815.1376819 İşman, A. (2005). Uzaktan eğitim. Ankara: Öğreti Yayınları. Kanejiya, D., Kumar, A., & Prasad, S. (2003). Automatic evaluation of students’ answers using syntactically enhanced LSA. Proceedings of the HLT-NAACL 03 workshop on Building educational applications using natural language processing-Volume 2 (pp. 53–60). Stroudsburg, PA USA:Association for Computational Linguistics. doi:10.3115/1118894.1118902 Kaya, Z. (2002). Uzaktan eğitim. Pegem A Yayıncılık. Koçdar, S., & Görü Doğan, T. (2015). Türkiye’deki açık ve uzaktan öğrenme programlarının bir analizi: Eğilimler ve öneriler. Eğitim ve Öğretim Araştırmaları Dergisi, 4(4), 23–36. Landauer, T. K. (2003). Automatic essay assessment. Assessment in Education: Principles, Policy & Practice, 10(3), 295–308.doi: 10.1080/0969594032000148154 Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389–405. doi: 10.1023/A:1025779619903 MEB. (2017). TEOG açık uçlu soru örnekleri. Retrieved August 31, 2017, from http://abide.meb.gov.tr/orneksorular.asp Noorbehbahani, F., & Kardan, A. A. (2011). The automatic assessment of free text answers using a modified BLEU algorithm. Computers & Education, 56(2), 337–345. doi: 10.1016/j.compedu.2010.07.013 ÖSYM. (2017). LYS açık uçlu soru örnekleri. Retrieved August 31, 2017, from http://www.osym.gov.tr/TR,12909/2017-lisans-yerlestirme-sinavlari-2017-lys-acik-uclu-sorular-hakkindabilgilendirme-ve-acik-uclu-soru-ornekleri-05012017.html Pérez-Marín, D., Alfonseca, E., & Rodríguez, P. (2006). On the dynamic adaptation of computer assisted assessment of free-text answers. International Conference on Adaptive Hypermedia and Adaptive WebBased Systems (pp. 374–377). Berlin:Springer.doi: 10.1007/11768012_54 Perez-Marin, D., Pascual-Nieto, I., Alfonseca, E., Anguiano, E., & Rodriguez, P. (2007). A study on the impact of the use of an automatic and adaptive free-text assessment system during a university course. Workshop on Blended Learning, (pp. 186-195). Edinburgh, United Kingdom: The Hong Kong Web Society Pérez, D., Alfonseca, E., & Rodríguez, P. (2004). Application of the bleu method for evaluating free-text answers in an e-learning environment. The International Conference on Language Resources and Evaluation (pp. 1351-1354) .Lisbon, Portugal: European Language Resources Association Roy, S., Narahari, Y., & Deshmukh, O. D. (2015). A perspective on computer assisted assessment techniques for short free-text answers. International Computer Assisted Assessment Conference (pp. 96–109). Cham:Springer. doi:10.1007/978-3-319-27704-2_10 Scalise, K., & Gifford, B. (2006). A framework for constructing “Intermediate Constraint” questions and tasks for technology platforms computer-based assessment in E-learning. The Journal of Technology, Learning, and Assessment, 4(6),1-45. Wiemer-Hastings, P., & Zipitria, I. (2001). Rules for syntax, vectors for semantics. In Proceedings of the Annual Meeting of the Cognitive Science Society. Retrieved from https://escholarship.org/uc/item/057457h4