In the 21st century, which is characterized as the Information Age, information access, knowledge quick learning is vital to the development of individuals and societies. With the use of technological innovations in the field of education in the information society, it will be possible to acquire a lasting place in the globalizing world. Distance Education refers to a model of the education system that students and teachers carry out through their learning and teaching activities by communicating via technologies and postal services. According to the year 2015 data, 68 out of 184 higher education institutions in Turkey offer an open and distance learning program. 47 of them are undergraduate, 17 are graduate, 11 are graduate completion and 56 are in master degree level. In total there are 505 different programs. The measurement and evaluation process of results of training is as important as developing content in distance education applications. When question types used in Distance Education Measurement and Assessment are examined, the use of Open-ended Questions is less than other methods. However, it is well-known fact that these type of questions are good predictors of the students’ knowledge. The biggest problem that arises with the usage of open-ended questions is the evaluation part. The different answers that students will give to the questions, their personal narrative skills, or the interpretations they will answer in response to the questions make the evaluation process difficult. At this point, it would be better to interpret the answers recorded in the database with Text Mining methods and Natural Language Processing techniques. In this work, we implement an algorithm for evaluation of open-ended question. The experimental results showed that a correlation was found between 0,89 - 0,96 when evaluating open-ended questions of our system by teacher evaluation.
Akın, A. A., & Akın, M. D. (2007). Zemberek, an open source nlp framework for turkic languages. Structure,
10, 1–5.
Balta, Y., & Türel, Y. K. (2013). Çevrimiçi uzaktan eğitimde kullanılan farklı ölçme değerlendirme
yaklaşımlarına ilişkin bir inceleme. Electronic Turkish Studies, 8(3),37-45.
Çallı, İ., İşman, A., & Torkul, O. (2002). Sakarya üniversitesi’nde uzaktan eğitimin dünü bugünü ve geleceği.
Sakarya Üniversitesi Eğitim Fakültesi Dergisi, 3,1-7.
Carro, R. M., Pulido, E., & Rodríguez, P. (2000). Adaptive internet-based learning with the Tangow system:
Computers and Education in the 21st Century (pp. 127–135). Dordrecht:Springer. doi:10.1007/0-306-
47532-4_12
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent
semantic analysis. Journal of the American Society for Information Science, 41(6), 391.
International Technology and Education Journal Vol. 2, No. 1; June 2018
8
Dessus, P., Lemaire, B., & Vernier, A. (2000). Free-text assessment in a virtual campus. Proc. 3rd International
Conference on Human System Learning (CAPS’3) (pp. 61–76). Paris, France:Learning's W.W.W.
Epignosis, L. L. C. (2014). E-learning concepts, trends, applications. [Version 1.1]. Retrieved from
https://www.talentlms.com/elearning/elearning-101-jan2014-v1.1.pdf
Foltz, P. W., Kintsch, W., & Landauer, T. K. (1998). The measurement of textual coherence with latent
semantic analysis. Discourse Processes, 25(2–3), 285–307.doi: 10.1080/01638539809545029
Foltz, P. W., Laham, D., & Landauer, T. K. (1999). Automated essay scoring: Applications to educational
technology. EdMedia: World Conference on Educational Media and Technology (pp. 939–944). Seattle,
WA USA:Association for the Advancement of Computing in Education (AACE).
Gelbal, S., & Kelecioğlu, H. (2007). Öğretmenlerin ölçme ve değerlendirme yöntemleri hakkındaki yeterlik
algıları ve karşılaştıkları sorunlar. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 33, 135–145.
Hotho, A., Nurnberger, A., & Paas, G. (2005). A brief survey of text mining. LDV Forum-GLDV Journal for
Computational Linguistics and Language Technology, 20(1),19–62.
Internet World Stats. (2017). Internet usage statıstıcs the ınternet big picture world ınternet users and 2017
population stats. Retrieved November 24, 2017, from https://www.internetworldstats.com/stats.htm
Islam, A., & Inkpen, D. (2008). Semantic text similarity using corpus-based word similarity and string similarity.
ACM Transactions on Knowledge Discovery from Data (TKDD), 2(2), 10. doi: 10.1145/1376815.1376819
İşman, A. (2005). Uzaktan eğitim. Ankara: Öğreti Yayınları.
Kanejiya, D., Kumar, A., & Prasad, S. (2003). Automatic evaluation of students’ answers using syntactically
enhanced LSA. Proceedings of the HLT-NAACL 03 workshop on Building educational applications using
natural language processing-Volume 2 (pp. 53–60). Stroudsburg, PA USA:Association for Computational
Linguistics. doi:10.3115/1118894.1118902
Kaya, Z. (2002). Uzaktan eğitim. Pegem A Yayıncılık.
Koçdar, S., & Görü Doğan, T. (2015). Türkiye’deki açık ve uzaktan öğrenme programlarının bir analizi:
Eğilimler ve öneriler. Eğitim ve Öğretim Araştırmaları Dergisi, 4(4), 23–36.
Landauer, T. K. (2003). Automatic essay assessment. Assessment in Education: Principles, Policy & Practice,
10(3), 295–308.doi: 10.1080/0969594032000148154
Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the
Humanities, 37(4), 389–405. doi: 10.1023/A:1025779619903
MEB. (2017). TEOG açık uçlu soru örnekleri. Retrieved August 31, 2017, from http://abide.meb.gov.tr/orneksorular.asp
Noorbehbahani, F., & Kardan, A. A. (2011). The automatic assessment of free text answers using a modified
BLEU algorithm. Computers & Education, 56(2), 337–345. doi: 10.1016/j.compedu.2010.07.013
ÖSYM. (2017). LYS açık uçlu soru örnekleri. Retrieved August 31, 2017, from
http://www.osym.gov.tr/TR,12909/2017-lisans-yerlestirme-sinavlari-2017-lys-acik-uclu-sorular-hakkindabilgilendirme-ve-acik-uclu-soru-ornekleri-05012017.html
Pérez-Marín, D., Alfonseca, E., & Rodríguez, P. (2006). On the dynamic adaptation of computer assisted
assessment of free-text answers. International Conference on Adaptive Hypermedia and Adaptive WebBased
Systems (pp. 374–377). Berlin:Springer.doi: 10.1007/11768012_54
Perez-Marin, D., Pascual-Nieto, I., Alfonseca, E., Anguiano, E., & Rodriguez, P. (2007). A study on the impact
of the use of an automatic and adaptive free-text assessment system during a university course. Workshop
on Blended Learning, (pp. 186-195). Edinburgh, United Kingdom: The Hong Kong Web Society
Pérez, D., Alfonseca, E., & Rodríguez, P. (2004). Application of the bleu method for evaluating free-text
answers in an e-learning environment. The International Conference on Language Resources and
Evaluation (pp. 1351-1354) .Lisbon, Portugal: European Language Resources Association
Roy, S., Narahari, Y., & Deshmukh, O. D. (2015). A perspective on computer assisted assessment techniques for
short free-text answers. International Computer Assisted Assessment Conference (pp. 96–109).
Cham:Springer. doi:10.1007/978-3-319-27704-2_10
Scalise, K., & Gifford, B. (2006). A framework for constructing “Intermediate Constraint” questions and tasks
for technology platforms computer-based assessment in E-learning. The Journal of Technology, Learning,
and Assessment, 4(6),1-45.
Wiemer-Hastings, P., & Zipitria, I. (2001). Rules for syntax, vectors for semantics. In Proceedings of the Annual
Meeting of the Cognitive Science Society. Retrieved from https://escholarship.org/uc/item/057457h4
Akın, A. A., & Akın, M. D. (2007). Zemberek, an open source nlp framework for turkic languages. Structure,
10, 1–5.
Balta, Y., & Türel, Y. K. (2013). Çevrimiçi uzaktan eğitimde kullanılan farklı ölçme değerlendirme
yaklaşımlarına ilişkin bir inceleme. Electronic Turkish Studies, 8(3),37-45.
Çallı, İ., İşman, A., & Torkul, O. (2002). Sakarya üniversitesi’nde uzaktan eğitimin dünü bugünü ve geleceği.
Sakarya Üniversitesi Eğitim Fakültesi Dergisi, 3,1-7.
Carro, R. M., Pulido, E., & Rodríguez, P. (2000). Adaptive internet-based learning with the Tangow system:
Computers and Education in the 21st Century (pp. 127–135). Dordrecht:Springer. doi:10.1007/0-306-
47532-4_12
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent
semantic analysis. Journal of the American Society for Information Science, 41(6), 391.
International Technology and Education Journal Vol. 2, No. 1; June 2018
8
Dessus, P., Lemaire, B., & Vernier, A. (2000). Free-text assessment in a virtual campus. Proc. 3rd International
Conference on Human System Learning (CAPS’3) (pp. 61–76). Paris, France:Learning's W.W.W.
Epignosis, L. L. C. (2014). E-learning concepts, trends, applications. [Version 1.1]. Retrieved from
https://www.talentlms.com/elearning/elearning-101-jan2014-v1.1.pdf
Foltz, P. W., Kintsch, W., & Landauer, T. K. (1998). The measurement of textual coherence with latent
semantic analysis. Discourse Processes, 25(2–3), 285–307.doi: 10.1080/01638539809545029
Foltz, P. W., Laham, D., & Landauer, T. K. (1999). Automated essay scoring: Applications to educational
technology. EdMedia: World Conference on Educational Media and Technology (pp. 939–944). Seattle,
WA USA:Association for the Advancement of Computing in Education (AACE).
Gelbal, S., & Kelecioğlu, H. (2007). Öğretmenlerin ölçme ve değerlendirme yöntemleri hakkındaki yeterlik
algıları ve karşılaştıkları sorunlar. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 33, 135–145.
Hotho, A., Nurnberger, A., & Paas, G. (2005). A brief survey of text mining. LDV Forum-GLDV Journal for
Computational Linguistics and Language Technology, 20(1),19–62.
Internet World Stats. (2017). Internet usage statıstıcs the ınternet big picture world ınternet users and 2017
population stats. Retrieved November 24, 2017, from https://www.internetworldstats.com/stats.htm
Islam, A., & Inkpen, D. (2008). Semantic text similarity using corpus-based word similarity and string similarity.
ACM Transactions on Knowledge Discovery from Data (TKDD), 2(2), 10. doi: 10.1145/1376815.1376819
İşman, A. (2005). Uzaktan eğitim. Ankara: Öğreti Yayınları.
Kanejiya, D., Kumar, A., & Prasad, S. (2003). Automatic evaluation of students’ answers using syntactically
enhanced LSA. Proceedings of the HLT-NAACL 03 workshop on Building educational applications using
natural language processing-Volume 2 (pp. 53–60). Stroudsburg, PA USA:Association for Computational
Linguistics. doi:10.3115/1118894.1118902
Kaya, Z. (2002). Uzaktan eğitim. Pegem A Yayıncılık.
Koçdar, S., & Görü Doğan, T. (2015). Türkiye’deki açık ve uzaktan öğrenme programlarının bir analizi:
Eğilimler ve öneriler. Eğitim ve Öğretim Araştırmaları Dergisi, 4(4), 23–36.
Landauer, T. K. (2003). Automatic essay assessment. Assessment in Education: Principles, Policy & Practice,
10(3), 295–308.doi: 10.1080/0969594032000148154
Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the
Humanities, 37(4), 389–405. doi: 10.1023/A:1025779619903
MEB. (2017). TEOG açık uçlu soru örnekleri. Retrieved August 31, 2017, from http://abide.meb.gov.tr/orneksorular.asp
Noorbehbahani, F., & Kardan, A. A. (2011). The automatic assessment of free text answers using a modified
BLEU algorithm. Computers & Education, 56(2), 337–345. doi: 10.1016/j.compedu.2010.07.013
ÖSYM. (2017). LYS açık uçlu soru örnekleri. Retrieved August 31, 2017, from
http://www.osym.gov.tr/TR,12909/2017-lisans-yerlestirme-sinavlari-2017-lys-acik-uclu-sorular-hakkindabilgilendirme-ve-acik-uclu-soru-ornekleri-05012017.html
Pérez-Marín, D., Alfonseca, E., & Rodríguez, P. (2006). On the dynamic adaptation of computer assisted
assessment of free-text answers. International Conference on Adaptive Hypermedia and Adaptive WebBased
Systems (pp. 374–377). Berlin:Springer.doi: 10.1007/11768012_54
Perez-Marin, D., Pascual-Nieto, I., Alfonseca, E., Anguiano, E., & Rodriguez, P. (2007). A study on the impact
of the use of an automatic and adaptive free-text assessment system during a university course. Workshop
on Blended Learning, (pp. 186-195). Edinburgh, United Kingdom: The Hong Kong Web Society
Pérez, D., Alfonseca, E., & Rodríguez, P. (2004). Application of the bleu method for evaluating free-text
answers in an e-learning environment. The International Conference on Language Resources and
Evaluation (pp. 1351-1354) .Lisbon, Portugal: European Language Resources Association
Roy, S., Narahari, Y., & Deshmukh, O. D. (2015). A perspective on computer assisted assessment techniques for
short free-text answers. International Computer Assisted Assessment Conference (pp. 96–109).
Cham:Springer. doi:10.1007/978-3-319-27704-2_10
Scalise, K., & Gifford, B. (2006). A framework for constructing “Intermediate Constraint” questions and tasks
for technology platforms computer-based assessment in E-learning. The Journal of Technology, Learning,
and Assessment, 4(6),1-45.
Wiemer-Hastings, P., & Zipitria, I. (2001). Rules for syntax, vectors for semantics. In Proceedings of the Annual
Meeting of the Cognitive Science Society. Retrieved from https://escholarship.org/uc/item/057457h4
Benli, İ., & İsmailova, R. (2018). Use of Open-Ended Questions in Measurement and Evaluation Methods in Distance Education. International Technology and Education Journal, 2(1), 1-8.