Using Generative Artificial Intelligence in Exams: A Research on KPSS with ChatGPT
Abstract
ChatGPT is a language model developed by OpenAI that uses generative artificial intelligence (GenAI) algorithms for natural language processing (NLP) tasks. It is a popular Chatbot due to its impressive ability to respond consistently to users' various prompts. In this study, the Public Personnel Selection Examination (KPSS) questions, organized by ÖSYM (Turkish Student Selection and Placement Center) every year, were asked to ChatGPT, and its performance was evaluated. The evaluation was made based on fundamental text questions asked in the last three years (2021-2023) in Biology, History, Geography, Turkish Language Literature, and Social Studies in KPSS. This study also aims to investigate the accuracy and reliability of the questions asked to ChatGPT in the field of KPSS educational sciences. According to the research results, ChatGPT's KPSS knowledge and interpretation skills are significantly lower than the university graduate candidates who took the exam. Some suggestions are included for better Chatbot performance, more effective prompt engineering and ethical use of these tools.
Keywords
References
- Awasthi, I., Gupta, K., Bhogal, P. S., Anand, S. S., & Soni, P. K. (2021). Natural language processing (NLP) based text summarization-a survey. 2021 6th International Conference on Inventive Computation Technologies (ICICT). 10.24926/iip.v13i2.4839. https://doi.org/10.1109/ICICT50816.2021.9358703