A Novel Semi-Automated Chatbot Model: Providing Consistent Response of Students’ Email in Higher Education based on Case-Based Reasoning and Latent Semantic Analysis

Öz

Learning is a process that requires interaction, asking and responding to the question is one of the most crucial interactions in learning. Students ask questions to instructors through online learning systems or emails outside the class. Providing a consistent, correct, detailed and personalised response to these questions is essential for developing their skills and abilities about the subject. Several automated chatbot models have been developed to answer students’ questions consistently. However, these automated chatbot models lack the ability to comprehensively and reliably respond to students’ questions. Thus, other forms of Chabot models are required to provide detailed and reliable responses along with consistent responses. This study aims to develop a semi-automated chatbot model to answer students’ questions via email. This study also focuses on ensuring the consistency of answers. Therefore, a novel chatbot model is developed based on the semi-automated approach which supports new ways of answering students’ questions via email. A feasibility study was carried out to investigate and demonstrate the efficiency of the proposed model. The Case-Based Reasoning (CBR) cycle is adopted in the answering process to ensure the consistency of the answer. Semantic analysis tool was used to measure the semantic textual similarity between questions/sentences. The feasibility study results confirm that the proposed semi-automated assessment approach is feasible for use in higher education. Also, these results highlighted that this model enables the instructor to provide a consistent and personalised answer to students while considerably reducing the instructor’s workload.

Anahtar Kelimeler

• Chabot
• Case-Based Reasoning
• Latent Semantic Analysis
• Consistent Response
• Higher Education

Kaynakça

1. [1] A. Augello, G. Vassallo, S. Gaglio, and G. Pilato, “A semantic layer on semi-structured data sources for intuitive chatbots,” Proc. Int. Conf. Complex, Intell. Softw. Intensive Syst. CISIS 2009, pp. 760–765, 2009, doi: 10.1109/CISIS.2009.165.
2. [2] B. AbuShawar and E. Atwell, “Usefulness, localizability, humanness, and language-benefit: additional evaluation criteria for natural language dialogue systems,” Int. J. Speech Technol., vol. 19, no. 2, pp. 373–383, 2016, doi: 10.1007/s10772-015-9330-4.
3. [3] W. Green, S. Hammer, and C. Star, “Facing up to the challenge: Why is it so hard to develop graduate attributes?,” High. Educ. Res. Dev., vol. 28, no. 1, pp. 17–29, 2009, doi: 10.1080/07294360802444339.
4. [4] A. Kerly, P. Hall, and S. Bull, “Bringing chatbots into education: Towards natural language negotiation of open learner models,” Knowledge-Based Syst., vol. 20, no. 2, pp. 177–185, 2007, doi: 10.1016/j.knosys.2006.11.014.
5. [5] A. Latham, K. Crockett, D. McLean, and B. Edmonds, “Oscar: An Intelligent Adaptive Conversational Agent Tutoring System Annabel,” Int. Symp. Agent Multi-Agent Syst. Technol. Appl. 9(1), 76–99., vol. 07/80, no. 2, p. 125, Nov. 2010, [Online]. Available: https://arxiv.org/pdf/1707.06526.pdf%0Ahttps://www.yrpri.org%0Ahttp://weekly.cnbnews.com/news/article.html?no=124000%0Ahttps://www.fordfoundation.org/%0Ahttp://bibliotecavirtual.clacso.org.ar/Republica_Dominicana/ccp/20120731051903/prep%0Ahttp://webpc.ciat.cgiar.or.
6. [6] A. C. Graesser, P. Chipman, B. C. Haynes, and A. Olney, “Auto tutor: An intelligent tutoring system with mixed-initiative dialogue,” IEEE Trans. Educ., vol. 48, no. 4, pp. 612–618, 2005, doi: 10.1109/TE.2005.856149.
7. [7] A. Shaw, “Using chatbots to teach socially intelligent computing principles in introductory computer science courses,” Proc. 9th Int. Conf. Inf. Technol. ITNG 2012, pp. 850–851, 2012, doi: 10.1109/ITNG.2012.70.
8. [8] J. P. Bigham, M. B. Aller, J. T. Brudvik, J. O. Leung, L. A. Yazzolino, and R. E. Ladner, “Inspiring blind high school students to pursue computer science with instant messaging chatbots,” SIGCSE’08 - Proc. 39th ACM Tech. Symp. Comput. Sci. Educ., pp. 449–453, 2008, doi: 10.1145/1352135.1352287.

9. [9] D. Duijst, J. Sandberg, and D. Buzzo, “Can we Improve the User Experience of Chatbots with Personalisation ?,” Univ. Amsterdam, vol. MASTERTHES, no. July, pp. 1–23, 2017.
10. [10] I. Ahmed and S. Singh, “AIML Based Voice Enabled Artificial Intelligent Chatterbot,” Int. J. u- e-Service, Sci. Technol., vol. 8, no. 2, pp. 375–384, 2015, doi: 10.14257/ijunesst.2015.8.2.36.
11. [11] L. Benotti, M. C. Martínez, and F. Schapachnik, “A Tool for Introducing Computer Science with Automatic Formative Assessment,” IEEE Trans. Learn. Technol., vol. 11, no. 2, pp. 179–192, 2018, doi: 10.1109/TLT.2017.2682084.
12. [12] F. A. M. Fonte, M. L. Nistal, J. C. B. Rial, and M. C. Rodriguez, “NLAST: A natural language assistant for students,” IEEE Glob. Eng. Educ. Conf. EDUCON, vol. 10-13-April-2016, no. April, pp. 709–713, 2016, doi: 10.1109/EDUCON.2016.7474628.
13. [13] M. Naveen Kumar, P. C. Linga Chandar, A. Venkatesh Prasad, and K. Sumangali, “Android based educational Chatbot for visually impaired people,” 2016 IEEE Int. Conf. Comput. Intell. Comput. Res. ICCIC 2016, pp. 3–6, 2017, doi: 10.1109/ICCIC.2016.7919664.
14. [14] S. Buyrukoglu, F. Batmaz, and R. Lock, “Improving marking efficiency for longer programming solutions based on a semi-automated assessment approach,” Comput. Appl. Eng. Educ., vol. 27, no. 3, pp. 733–743, 2019, doi: 10.1002/cae.22094.
15. [15] A. Tarko, N. E. Tsendbazar, S. de Bruin, and A. K. Bregt, “Producing consistent visually interpreted land cover reference data: learning from feedback,” Int. J. Digit. Earth, vol. 0, no. 0, pp. 1–19, 2020, doi: 10.1080/17538947.2020.1729878.
16. [16] I. Clark, “Formative Assessment: Policy, Perspectives and Practice.,” Florida J. Educ. Adm. Policy, vol. 4, no. 2, pp. 158–180, 2011.
17. [17] B. Crossouard and J. Pryor, “How Theory Matters: Formative Assessment Theory and Practices and Their Different Relations to Education,” Stud. Philos. Educ., vol. 31, no. 3, pp. 251–263, 2012, doi: 10.1007/s11217-012-9296-5.
18. [18] G. Conole and B. Warburton, “A review of computer-assisted assessment,” Alt-J, vol. 13, no. 1, pp. 17–31, 2005, doi: 10.1080/0968776042000339772.
19. [19] M. Radhwan, “Investigating the Impact of Applying Different Strategies of Formative Assessments on Students ’ Learning Outcomes in Summative Assessments in a Private School in Sharjah , UAE,” vol. 2, no. 1, pp. 57–79, 2019.
20. [20] K. M. Ala-Mutka, “A survey of automated assessment approaches for programming assignments,” Comput. Sci. Educ., vol. 15, no. 2, pp. 83–102, 2005, doi: 10.1080/08993400500150747.
21. [21] D. Chalmers, “Computer-assisted Assessment,” 2002.
22. [22] C. Gütl, “Moving towards a fully automatic knowledge assessment tool,” Int. J. Emerg. Technol. Learn., vol. 3, no. 1, pp. 36–44, 2008.
23. [23] A. M. Ducasse and K. Hill, “Developing Student Feedback Literacy Using Educational Technology and the Reflective Feedback Conversation.,” Pract. Res. High. Educ., vol. 12, no. 1, pp. 24–37, 2019.
24. [24] N. Evangelopoulos, X. Zhang, and V. R. Prybutok, “Latent semantic analysis: Five methodological recommendations,” Eur. J. Inf. Syst., vol. 21, no. 1, pp. 70–86, 2012, doi: 10.1057/ejis.2010.61.
25. [25] J. Samuel, G. G. M. N. Ali, M. M. Rahman, E. Esawi, and Y. Samuel, “COVID-19 public sentiment insights and machine learning for tweets classification,” Inf., vol. 11, no. 6, pp. 1–22, 2020, doi: 10.3390/info11060314.
26. [26] K. Zupanc and Z. Bosnić, “Automated essay evaluation with semantic analysis,” Knowledge-Based Syst., vol. 120, pp. 118–132, 2017, doi: 10.1016/j.knosys.2017.01.006.
27. [27] H. Kwon, J. Kim, and Y. Park, “Applying LSA text mining technique in envisioning social impacts of emerging technologies: The case of drone technology,” Technovation, vol. 60–61, no. December 2015, pp. 15–28, 2017, doi: 10.1016/j.technovation.2017.01.001.
28. [28] N. Kishore Kumar and J. Schneider, “Literature survey on low rank approximation of matrices,” Linear Multilinear Algebr., vol. 65, no. 11, pp. 2212–2244, 2017, doi: 10.1080/03081087.2016.1267104.
29. [29] A. H. Ababneh, J. Lu, and Q. Xu, An efficient framework of utilizing the latent semantic analysis in text extraction, vol. 22, no. 3. Springer US, 2019.