REGULATORY EFFECTS OF GENERATIVE AI ON SELF-REGULATED LEARNING AND AI LITERACY: A MIXED-METHODS STUDY
Abstract
This study aimed to explicitly examine cognitive patterns of prompt engineering, the role of pre-service teachers’ artificial intelligence literacy (AIL) and online self-regulated learning (SRL) abilities in their use of prompting skills to develop lesson plans. The study was designed as a convergent parallel exploratory mixed-methods study involving quantitative analysis of students’ data on achievement in lesson planning, SRL and AIL, and qualitative interpretative exploration of students’ prompt sets based on cognitive domain and argumentation levels. The quantitative data was collected from 63 sophomore college students in a Faculty of Education through a quasi-experimental study with pre and post achievement tests, an AIL scale and an online learning self-regulation scale. The participants attended the experimental procedure with AIPRO. ORG in a computer laboratory within two hours. Two raters analyzed a total of 1070 prompts on the basis of hierarchical level of taxonomy of cognitive domain, and prompt sets on the basis of argumentative characteristics in three levels. The thematic analysis of the prompts revealed that the prompts were categorized into seven different themes. Sets of statistical tests, including general linear model 2X2 ANOVA tests and t tests, were conducted with the participants’ quantitative data revealed that the study was effective in helping participants, regardless of level of SRL and AIL, to develop knowledge of lesson planning. The study then merged quantitative and qualitative results revealing implications for generative AI based college learning environments, and perspectives to inform new studies of learning and teaching with AI applications.
Keywords
Artificial intelligence literacy, self-regulated learning, lesson planning
Supporting Institution
Project Number
Ethical Statement
Thanks
References
- Acquah, B. Y. S., Arthur, F., Salifu, I., Quayson, E., & Nortey, S. A. (2024). Preservice teachers’ behavioral intention to use AI in lesson planning. Computers and Education: Artificial Intelligence, 7, 100307.
- Adams, D., Chuah, K. M., Devadason, E., & Azzis, M. S. A. (2024). From novice to navigator: Students’ academic help-seeking behavior, readiness, and perceived usefulness of ChatGPT in learning. Education and Information Technologies, 29(11), 13617-13634.
- Alsafari, B., Atwell, E., Walker, A., & Callaghan, M. (2024). Towards effective teaching assistants: From intent-based chatbots to LLM-powered teaching assistants. Natural Language Processing Journal, 8, 100101.
- Anggrella, D. P., Izzati, L. R., & Sudrajat, A. K. (2023). Improving the quality of learning through lesson plan preparation workshops for an independent learning model. Journal of Community Service & Empowerment, 4(1), 162-171.
- Bannert, M., Sonnenberg, C., Mengelkamp, C., & Pieger. E., (2015). Short-and long-term effects of students’ self-directed metacognitive prompts on navigation behavior and learning performance. Computers in Human Behavior, 52, 293–306.