Exploring the potential use of generative ai for learner support in ODL at scale

Abstract

This study addresses the applicability of generative artificial intelligence (GenAI) within the administrative learner support services at Anadolu University Open Education System, a giga university with more than one million learners in Türkiye. The study reveals the performance differences between a rule-based chatbot and different GenAI-based chatbots using a qualitative case study approach. Learner inquiries that the rule-based chatbot could not answer, and frequently asked questions (FAQs) were posed to ChatGPT and Bing Copilot applications in four different cases. The 708 answers in these scenarios were evaluated by three experts. It was observed that GenAI matched learner questions more effectively than the rule-based chatbot. Additionally, Bing Copilot was more successful in generating responses to learners’ questions from the internet compared to ChatGPT, which utilized the FAQ dataset. The study reveals that Gen-AI based chatbots that can work together with rule-based chatbots are more successful in generating answers. Findings demonstrate the potential of GenAI to provide learners with continuous, more natural, and personalized support. These findings offer significant insights into how administrative support services in mass-scale educational institutions can be transformed.

Keywords

• Generative artificial intelligence
• ChatGPT
• chatbot
• learner support services
• open and distance learning

References

1. Abdelkader, O. A. (2023). ChatGPT's influence on customer experience in digital marketing: Investigating the moderating roles. Heliyon, 9(8). https://doi.org/10.1016/j.heliyon.2023.e18770
2. Agharia, S., Szatkowski, J., Fraval, A., Stevens, J., & Zhou, Y. (2024). The ability of artificial intelligence tools to formulate orthopaedic clinical decisions in comparison to human clinicians: An analysis of ChatGPT 3.5, ChatGPT 4, and Bard. Journal of Orthopaedics, 50, 1-7. https://doi.org/10.1016/j.jor.2023.11.063
3. Ait-Mlouk, A., & Jiang, L. (2020). KBot: A knowledge graph based chatBot for natural language understanding over linked data. IEEE Access, 8, 149220-149230. https://doi.org/10.1109/ACCESS.2020.3016142
4. Akiba, D., & Fraboni, M. C. (2023). AI-supported academic advising: Exploring ChatGPT’s current state and future potential toward learner empowerment. Education Sciences, 13(9), 885. https://doi.org/10.3390/educsci13090885
5. Akter, S., Hossain, M. A., Sajib, S., Sultana, S., Rahman, M., Vrontis, D., & McCarthy, G. (2023). A framework for AI-powered service innovation capability: Review and agenda for future research. Technovation, 125, 102768.
6. Al-Fatlawi, A., Al-Khazaali, A. A. T., & Hasan, S. H. (2024) AI-based model for fraud detection in bank systems. Fusion: Practice and Applications, 14(1), 19-27. https://doi.org/10.54216/FPA.140102
7. Araujo, T. (2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051
8. Arun, K., Sri Nagesh, A., & Ganga, P. (2019). A multi-model and AI-based collegebot management system (AICMS) for professional engineering colleges. International Journal of Innovative Technology and Exploring Engineering, 8(9), 2910-2914. https://doi.org/10.35940/ijitee.I8818.078919

9. Berge, Z. L. (1995). Facilitating computer conferencing: Recommendations from the field. Educational Technology, 35(1), 22-30.
10. Bozkurt, A. (2023a). Generative AI, synthetic contents, open educational resources (OER), and open educational practices (OEP): A new front in the openness landscape. Open Praxis, 15(3), 1–7. https://doi.org/10.55982/openpraxis.15.3.579
11. Bozkurt, A. (2023b). Unleashing the Potential of Generative AI, Conversational Agents and Chatbots in Educational Praxis: A Systematic Review and Bibliometric Analysis of GenAI in Education. Open Praxis, 15(4), 261–270. https://doi.org/10.55982/openpraxis.15.4.609
12. Bressane, A., Zwirn, D., Essiptchouk, A., Saraiva, A. C. V., de Campos Carvalho, F. L., Formiga, J. K. S., ... & Negri, R. G. (2024). Understanding the role of study strategies and learning disabilities on learner academic performance to enhance educational approaches: A proposal using artificial intelligence. Computers and Education: Artificial Intelligence, 6. https://doi.org/10.1016/j.caeai.2023.100196
13. Brynjolfsson, E., Li, D., & Raymond, L. R. (2023). Generative AI at work (No. w31161). National Bureau of Economic Research.
14. Cascella, M., Montomoli, J., Bellini, V., & Bignami, E. (2023). Evaluating the feasibility of ChatGPT in healthcare: An analysis of multiple clinical and research scenarios. Journal of Medical Systems, 47(1), 1-5. https://doi.org/10.1007/s10916-023-01925-4
15. Chang, D. H., Lin, M. P. C., Hajian, S., & Wang, Q. Q. (2023). Educational Design Principles of Using AI Chatbot That Supports Self-Regulated Learning in Education: Goal Setting, Feedback, and Personalization. Sustainability (Switzerland), 15(17). https://doi.org/10.3390/su151712921
16. Chaves, A. P., Doerry, E., Egbert, J., & Gerosa, M. (2019). It's how you say it: Identifying appropriate register for chatbot language design. In Proceedings of the 7th International Conference on Human-Agent Interaction (HAI '19), Kyoto, Japan (pp. 102-109). Association for Computing Machinery. https://doi.org/10.1145/3349537.3351901
17. Chaves, A. P., & Gerosa, M. A. (2021). How Should My Chatbot Interact? A Survey on Social Characteristics in Human–Chatbot Interaction Design. International Journal of Human-Computer Interaction, 37(8), 729–758. https://doi.org/10.1080/10447318.2020.1841438
18. Chocarro, R., Cortiñas, M., & Marcos-Matás, G. (2023). Teachers’ attitudes towards chatbots in education: a technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics. Educational Studies, 49(2), 295–313. https://doi.org/10.1080/03055698.2020.1850426
19. Choque-Díaz, M., Armas-Aguirre, J., & Shiguihara-Juárez, P. (2018, August). Cognitive technology model to enhanced academic support services with chatbots. In 2018 IEEE XXV International Conference on Electronics, Electrical Engineering and Computing (INTERCON) (pp. 1-4). IEEE. https://doi.org/10.1109/INTERCON.2018.8526411
20. Chung, M., Ko, E., Joung, H., & Kim, S. J. (2020). Chatbot e-service and customer satisfaction regarding luxury brands. Journal of Business Research, 117, 587-595. https://doi.org/10.1016/j.jbusres.2018.10.004
21. Creswell, J. W., & Poth, C. N. (2024). Qualitative Inquiry and Research Design: Choosing Among Five Approaches (5th ed.). United States of America: SAGE Publications.
22. de-Lima-Santos, M. F., & Ceron, W. (2022). Artificial intelligence in news media: current perceptions and future outlook. Journalism and Media, 3(1), 13-26. https://doi.org/10.3390/journalmedia3010002
23. Dempere, J., Modugu, K., Hesham, A., & Ramasamy, L. K. (2023). The impact of ChatGPT on higher education. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1206936
24. Erdoğdu, E., (2022). Learner Support Services in Open and Distance Education During the Covid-19 Period: A Case of Anadolu University Open Education System. ICETOL 2022, Balıkesir, Turkey
25. Erdoğdu, E., & Gümüş, S. (2023). Açıköğretim Sisteminde Öğrenci Destek Hizmetleri. In A. Z. Özgür, K. Çekerol, S. Koçdar and İ. Kayabaş, Açıköğretim ile 40 Yıl: Uygulamalar ve Araştırmalar (1st ed., pp. 71-132). Eskişehir: Anadolu Üniversitesi Yayınları.
26. Firat, M., & Kuleli, S. (2024). GPT vs. Google: A comparative study of self-code learning in ODL students. Journal of Educational Technology and Online Learning, 7(3), 308-320. https://doi.org/10.31681/jetol.1508675
27. Geckin, V., Kızıltaş, E., & Çınar, Ç. (2023). Assessing second-language academic writing: AI vs. Human raters. Journal of Educational Technology and Online Learning, 6(4), 1096-1108. https://doi.org/10.31681/jetol.1336599
28. Goel, A. K., & Polepeddi, L. (2018). Jill Watson: A Virtual Teaching Assistant for Online Education. In C. Dede, J. Rıchards and B. Saxberg, Learning engineering for online education: Theoretical contexts and design-based examples (1st ed., pp. 120-143). New York: Routledge. https://doi.org/10.4324/9781351186193-7
29. Gunawardena, C. N., & McIsaac, M. S. (2004). Distance education. In D. H. Jonassen, Handbook of Research in Educational Communications and Technology (2nd ed., pp. 355-395). Mahwah, NJ: Lawrence Erlbaum Associates.
30. Han, S., & Lee, M. K. (2022). FAQ chatbot and inclusive learning in massive open online courses. Computers & Education, 179. https://doi.org/10.1016/j.compedu.2021.104395
31. Hien, H. T., Cuong, P. N., Nam, L. N. H., Nhung, H. L. T. K., & Thang, L. D. (2018, December). Intelligent assistants in higher-education environments: the FIT-EBot, a chatbot for administrative and learning support. In Proceedings of the 9th International Symposium on Information and Communication Technology (pp. 69-76). https://doi.org/10.1145/3287921.3287937
32. Hmoud, M., Swaity, H., Hamad, N., Karram, O., & Daher, W. (2024). Higher education learners’ task motivation in the generative artificial intelligence context: The Case of ChatGPT. Information. 15(1), 1–18. https://doi.org/10.3390/info15010033
33. Kalla, D., & Kuraku, S. (2023). Advantages, disadvantages and risks associated with chatgpt and ai on cybersecurity. Journal of Emerging Technologies and Innovative Research, 10(10).
34. Keast, D. A. (1997). Toward an effective model for implementing distance education programs, American Journal of Distance Education, 11(2), 39-55. https://doi.org/10.1080/08923649709526960
35. Keegan, D. (1996). Foundations of Distance Education (3rd ed.). London Routledge.
36. Kim, J., Lee, H., & Cho, Y. H. (2022). Learning design to support learner-AI collaboration: Perspectives of leading teachers for AI in education. Education and Information Technologies, 27(5), 6069-6104. https://doi.org/10.1007/s10639-021-10831-6
37. Kukkar, A., Mohana, R., Sharma, A., & Nayyar, A. (2023). Prediction of learner academic performance based on their emotional wellbeing and interaction on various e-learning platforms. Education and Information Technologies, 28(8), 1–30. https://doi.org/10.1007/s10639-022-11573-9
38. Lappalainen, Y., & Narayanan, N. (2023). Aisha: A Custom AI Library Chatbot Using the ChatGPT API. Journal of Web Librarianship, 17(3), 37-58. https://doi.org/10.1080/19322909.2023.2221477
39. Lentell, H. (2003). The importance of the tutor in open and distance learning. In A. Tait, & R. Mills, Rethinking learner support in distance education: Change and continuity in an international context (1st ed., pp. 64-76). London: RoutledgeFalmer.
40. Liebrecht, C., Sander, L., & Van Hooijdonk, C. (2021). Too informal? How a chatbot’s communication style affects brand attitude and quality of interaction. In Chatbot Research and Design: 4th International Workshop, CONVERSATIONS 2020, Virtual Event, November 23–24, 2020, Revised Selected Papers 4 (pp. 16-31). Springer International Publishing.
41. Liu, P. C., Wang, W., Wang, Z., & Yang, Y. (2024). Will artificial intelligence undermine the effects of guanxi on relationship performance? Evidence from China's banking industry. Industrial Marketing Management, 116, 12-25. https://doi.org/10.1016/j.indmarman.2023.11.007
42. Malik, S., & Zaheer, S. (2024). ChatGPT as an aid for pathological diagnosis of cancer. Pathology-Research and Practice, 253. https://doi.org/10.1016/j.prp.2023.154989
43. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage Publications.
44. Mills, R. (2003). The centrality of learner support in open and distance learning: A paradigm shift in thinking. In A. Tait, & R. Mills, Rethinking learner support in distance education: Change and continuity in an international context (1st ed., pp. 102-113). London: RoutledgeFalmer. https://doi.org/10.4324/9780203006191
45. Moore, M. G., & Kearsley, G. (2012). Distance Education: A Systems View of Online Learning. (3rd ed.). Wadsworth Cengage Learning, Belmont, CA.
46. Mpu, Y. (2023). Bridging the Knowledge Gap on Special Needs Learner Support: The Use of Artificial Intelligence (AI) to Combat Digital Divide Post-COVID-19 Pandemic and beyond–A Comprehensive Literature Review. https://doi.org/10.5772/intechopen.113054
47. Mukherjee, S., Hudeček, V., & Dušek, O. (2023). Polite Chatbot: A Text Style Transfer Application. In E. Bassignana, M. Lindemann, & A. Petit (Eds.), *Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Learner Research Workshop* (pp. 87–93). Association for Computational Linguistics. https://aclanthology.org/2023.eacl-srw.9
48. Murad, D. F., Irsan, M., Akhirianto, P. M., Fernando, E., Murad, S. A., & Wijaya, M. H. (2019, July). Learning support system using chatbot in" Kejar C Package" homeschooling program. In 2019 international conference on information and communications technology (ICOIACT) (pp. 32-37). IEEE. https://doi.org/10.1109/ICOIACT46704.2019.8938479
49. Naidu, K., & Sevnarayan, K. (2023). ChatGPT: An ever-increasing encroachment of artificial intelligence in online assessment in distance education. Online Journal of Communication and Media Technologies, 13(3), e202336. https://doi.org/10.30935/ojcmt/13291
50. Öncü, S. E. (2024). User Opinions About Anadolu University Open Education System Virtual Assistant Application. Unpublished Master Dissertation. Eskişehir: Osmangazi University, Institute of Educational Sciences.
51. Paniagua, A. S. E., & Simpson, O. (2018). Developing learner support for open and distance learning: The EMPOWER project. Journal of Interactive Media in Education, 2018(1). https://doi.org/10.5334/jime.470
52. Panda, S., & Kaur, N. (2023). Exploring the viability of ChatGPT as an alternative to traditional chatbot systems in library and information centers. Library Hi Tech News, 40(3), 22-25. https://doi.org/10.1108/LHTN-02-2023-0032
53. Prinz, K. (2022). The Smiling Chatbot: Investigating Emotional Contagion in Human-to-Chatbot Service Interactions. In The Smiling Chatbot. Springer. https://doi.org/10.1007/978-3-658-40028-6
54. Pulist, S. K. (2022) Use of Chatbots as AI Agents to Augment Services in Open and Distance Learning System. Online International Conference on Applications of Artificial Intelligence in Education for Sustainable Development, Ambedkar University, New Delhi.
55. Qudah, M. A. A., & Muradkhanli, L. (2024). The use of generative artificial intelligence for customer services. Problems of Information Technology, 15(1), 10-17.
56. Rad, D., & Rad, G. (2023). Exploring the psychological implications of ChatGPT: A qualitative study. Journal Plus Education, 32(1), 43–55.
57. Rahm, D., & Reed, B. J. (1997). Going remote: The use of distance learning, the world wide Web, and the internet in graduate programs of public affairs and administration. Public Productivity & Management Review, 20(4), 459–474. https://doi.org/10.2307/3380685
58. Rasul, T., Nair, S., Kalendra, D., Robin, M., Santini, F. de O, Ladeira, W. J., Sun, M., Day, I., Rather, R. A. & Heathcote, L. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning and Teaching, 6(1), 41-56. https://doi.org/10.37074/jalt.2023.6.1.29
59. Reinhard, P., Li, M. M., Peters, C., & Leimeister, J. M. (2024). Generative AI in Customer Support Services: A Framework for Augmenting the Routines of Frontline Service Employees. Hawaii International Conference on System Sciences (HICSS). Waikiki, Hawaii, USA.
60. Rekkedal, T. & Qvist-Eriksen, S. (2003) Internet based e-learning, pedagogy and support systems. In T. Rekkedal, S. Qvist-Eriksen, T. Fagerberg, M. F. Paulsen, A. K. L. Aakre & J. Sjaastad, Learner Support Services in E-learning (pp. 1-25). Norway: NKI Distance Education.
61. Rocio, V., & Wesley, A. (2020). Building a chatbot for learner support. Revista de Ciências da Computação, 15.
62. Rowntree, D. (1992). Exploring open and distance learning (1st ed.). London: Kogan Page.
63. Rumble, G. (2000). Learner support in distance education in the 21st century: Learning from service management. Distance Education, 21 (2), 216-235. https://doi.org/10.1080/0158791000210202
64. Rudolph, J., Tan, S., & Tan, S. (2023). War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. Journal of Applied Learning and Teaching, 6(1), 364-389. https://doi.org/10.37074/jalt.2023.6.1.23
65. Sanchez-Elvira Paniagua, A., & Simpson, O. (2018). Developing learner support for open and distance learning: The EMPOWER project. Journal of Interactive Media in Education, 2018(1), 1–10. https://doi.org/10.5334/jime.470
66. Sadhotra, N., & Gupta, N. (2023). Generative AI in customer service. Khanna, P., & Sadhotra, N. (Editors), Digital Paradigm Shift: Unravelling Technological Disruption in Business (31-46).
67. Shumanov, M., & Johnson, L. (2021). Making conversations with chatbots more personalized. Computers in Human Behavior, 117. https://doi.org/10.1016/j.chb.2020.106627
68. Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2012). Teaching and learning at a distance: foundations of distance education (5th ed.). Boston, MA: Pearson Education.
69. Simpson, O. (2012). Supporting Learners for Success in Online and Distance Education (3rd ed.). Routledge. https://doi.org/10.4324/9780203095737
70. Suta, P., Lan, X., Wu, B., Mongkolnam, P., & Chan, J. H. (2020). An overview of machine learning in chatbots. International Journal of Mechanical Engineering and Robotics Research, 9(4), 502-510. https://doi.org/10.18178/ijmerr.9.4.502-510
71. Szyrocka, J. R., Zywiolek, J., Nayyar, A., & Naved, M. (Eds.). (2023). Advances in distance learning in times of pandemic. CRC Press.
72. Tait, A. (2000). Planning learner support for open and distance learning. Open Learning, 15(3), 287-299. https://doi.org/10.1080/713688410
73. Tait, A. (2014). From Place to Virtual Space: Reconfiguring Learner Support for Distance and E-Learning in the Digital Age. Open Praxis, 6(1), 5. https://doi.org/10.5944/openpraxis.6.1.102
74. Thorpe, M. (2003) Collaborative online learning: Transforming learner support and course design. In A. Tait, & R. Mills, Rethinking learner support in distance education: Change and continuity in an international context (1st ed., pp. 198-211). London: RoutledgeFalmer.
75. Tonbuloğlu, B. (2023). An Evaluation of the Use of Artificial Intelligence Applications in Online Education. Journal of Educational Technology and Online Learning, 6(4), 866-884. https://doi.org/10.31681/jetol.1335906
76. Wedemeyer, C. A. (1975). Implications of open learning for independent study. 10th ICCE Conference, Brighton, United Kingdom, May 15, 1975, 27.
77. Yin, R. K. (2018). Case Study Research and Applications: Design and Methods (6th ed.). Thousand Oaks, CA: SAGE Publications.