Shaping Mathematics Activities with Generative AI: Prompt Types, Models and Pedagogical Outcomes

Abstract

This study investigates the relationship between prompt types and the quality of mathematics activities generated by artificial intelligence (AI) tools. Within a multiple-case study design, two advanced AI systems, ChatGPT-5 (OpenAI, September 2025) and Gemini 2.5 Pro (Google DeepMind, September, 2025), were examined using command (C) and request (R) prompts under standardised settings (temperature = 0.7, top-p = 0.9). Four activities were produced and evaluated with the Activity Evaluation and Feedback Tool, which assesses both component-level features (intended outcome, materials, instructions, responsibility, inclusivity, depth, complexity, and mathematical focus) and overall quality. The analysis revealed that three of the four AI-generated activities reached the high-quality range, with total scores of 22, 19, and 23 out of 24 points for Gemini-R, Gemini-C, and ChatGPT-C, respectively, whereas ChatGPT-R scored 15 points, indicating a medium level but close to the high threshold. ChatGPT demonstrated greater effectiveness with command prompts, whereas Gemini produced consistently high-quality outputs, performing better with request prompts. At the component level, intended outcome and materials were consistently strong, while weaknesses were observed in instructions, responsibility, and complexity, depending on the AI–prompt combination. These findings demonstrate that activity quality is shaped not only by prompt design but also by model-specific affordances. Implications are discussed for teacher education, curriculum development, and comparative research on the integration of generative AI in mathematics education.

Keywords

• generative artificial intelligence
• chatgpt
• gemini
• mathematics activity design
• prompt engineering
• teacher education

Yapay Zekâ ile Matematik Etkinliklerinin Şekillendirilmesi: Prompt Türleri, Modeller ve Pedagojik Çıktılar

Öz

Bu çalışma, prompt türleri ile yapay zekâ (YZ) araçları tarafından üretilen matematik etkinliklerinin kalitesi arasındaki ilişkiyi incelemektedir. Çoklu durum çalışması deseni kapsamında, iki gelişmiş yapay zekâ sistemi olan ChatGPT-5 (OpenAI, Eylül 2025) ve Gemini 2.5 Pro (Google DeepMind, Eylül 2025), emir ve rica türü promptlar kullanılarak standart ayarlarda (temperature = 0.7, top-p = 0.9) test edilmiştir. Bu süreçte dört etkinlik üretilmiş ve bu etkinlikler, hem bileşen düzeyindeki özellikleri (hedeflenen kazanım, materyaller, yönergeler, sorumluluk, kapsayıcılık, derinlik, karmaşıklık ve matematiksel odak) hem de genel kaliteyi ölçen Etkinlik Değerlendirme ve Geri Bildirim Aracı ile analiz edilmiştir. Bulgular, dört etkinlikten üçünün yüksek kalite aralığında (Gemini-R = 22, Gemini-C = 19, ChatGPT-C = 23 / 24 puan) yer aldığını, ChatGPT-R çıktısının ise 15 puanla orta düzeyde olup yüksek kalite eşiğine yaklaştığını göstermiştir. ChatGPT emir türü promptlarda daha yüksek etkinlik gösterirken, Gemini'nin rica türü promptlarla daha iyi performans sergileyerek tutarlı bir şekilde yüksek kaliteli çıktılar ürettiği görülmüştür. Bileşen düzeyinde, “hedeflenen kazanım” ve “materyaller” tutarlı bir şekilde güçlü yönler olarak öne çıkarken; “yönergeler”, “sorumluluk” ve “karmaşıklık” alanlarında ise kullanılan YZ ve prompt türü kombinasyonuna bağlı olarak zayıflıklar tespit edilmiştir. Bu bulgular, etkinlik kalitesinin yalnızca prompt tasarımına göre değil, aynı zamanda kullanılan modelin kendine özgü olanaklarına göre de şekillendiğini ortaya koymaktadır. Çalışmanın sonuçlarının öğretmen eğitimi, müfredat geliştirme ve matematik eğitiminde üretken yapay zekâ entegrasyonuna yönelik karşılaştırmalı araştırmalar için doğurguları tartışılmıştır

Anahtar Kelimeler

• üretken yapay zekâ
• chatgpt
• gemini
• matematik etkinliği tasarımı
• prompt mühendisliği
• öğretmen eğitimi

Kaynakça

1. Ainley, J., Pratt, D., & Hansen, A. (2006). Connecting engagement and focus in pedagogic task design. British Educational Research Journal, 32(1), 2338. https://doi.org/10.1080/01411920500401971
2. Bozkurt, A., Özmantar M. F., Agaç, G. & Güzel, M. (2022). Matematik Öğretiminde Etkinlik Tasarımı ve Uygulamaları: Bir Değerlendirme Çerçevesi. Pegem Akademi.
3. Chmiliar, l. (2010). Multiple-case designs. In A. J. Mills, G. Eurepas & E. Wiebe (Eds.), Encyclopedia of case study research (pp 582–583). SAGE Publications.
4. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
5. Choy, B. H. (2016). Snapshots of mathematics teacher noticing during task design. Mathematics Education Research Journal, 28(3), 421-440. https://doi.org/10.1007/s13394-016-0173-3
6. Cooper, G. (2023). Examining science education in chatgpt: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32(3), 444–452. https://doi.org/10.1007/s10956-023-10039-y
7. Correia, A. P., Hickey, S., & Xu, F. (2025). Realizing the possibilities of the large language models: Strategies for prompt engineering in educational inquiries. Theory Into Practice, 64(4), 434–447. https://doi.org/10.1080/00405841.2025.2528545
8. Creswell, J. W. (2017). Eğitim Araştırmaları. Nicel ve Nitel Araştırmanın Planlanması Yürütülmesi ve Değerlendirilmesi. Çev. Ed. Halil Ekşi. EDAM Yayıncılık.

9. Dumlu, B. Ö., Gezer, E., & Yıldız, B. (2024). Examination of lesson plans prepared with ChatGPT on inequality. The Journal of Turkish Educational Sciences, 22(1), 337–358. https://doi.org/10.37217/tebd.1338959
10. Ergene, O., Caylan-Ergene, B. (2025a) Pre-service Mathematics Teachers’ and Engineering Students’ Perceptions of ChatGPT in Mathematics: Development, Validation And Implementation Study. Digital Experience in Mathematics Education . 1–27. https://doi.org/10.1007/s40751-025-00176-x
11. Ergene, O., & Caylan Ergene, B. (2025b). AI ChatBots’ solutions to mathematical problems in interactive e-textbooks: Affordances and constraints from the eyes of students and teachers. Education and Information Technologies, 30(1), 509–545. https://doi.org/10.1007/s10639-024-13121-z
12. Franceschelli, G., & Musolesi, M. (2024). On the creativity of large language models. AI & Society. 40, 3785–3795. https://doi.org/10.1007/s00146-024-02127-3
13. Gattupalli, S., Maloy, R. W., & Edwards, S. A. (2023). Prompt literacy: A pivotal educational skill in the age of AI. Educational Technology & Society.1–6. https://hdl.handle.net/20.500.14394/22905
14. Griffin, P. (2009). What Makes a Rich Task? Mathematics teaching, 212, 32–34.
15. Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation. Contemporary Educational Technology, 15(2), ep421. https://doi.org/10.30935/cedtech/13036
16. Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., ... & Koedinger, K. R. (2022). Ethics of AI in education: Towards a community-wide framework. International Journal of Artificial Intelligence in Education, 32(3), 504–526. https://doi.org/10.1007/s40593-021-00239-1
17. Hossain, K. A. (2023). Analysis of present and future use ofartificial intelligence (ai) in line of fouth industrial revolution (4ir). Scientific Research Journal, 11(8), 1–50.
18. Hwang, G. J., & Tu, Y. F. (2021). Roles and research trends of artificial intelligence in mathematics education: A bibliometric mapping analysis and systematic review. Mathematics, 9(6), 584. https://doi.org/10.3390/math9060584
19. Jin, H. S., & Suh, B. (2024). Research on a statistics education program utilizing deep learning predictions in high school mathematics. The Mathematical Education, 63(2), 209–231. https://doi.org/10.63311/mathedu.25.64311
20. Jones, K., & Pepin, B. (2016). Research on mathematics teachers as partners in task design. Journal of Mathematics Teacher Education, 19(2), 105-121. https://doi.org/10.1007/s10857-016-9345-z
21. Kieran, C., Doorman, M., & Ohtani, M. (2015). Frameworks and principles for task design. Anne, W. & Minoru O. (Eds.) Task Design in Mathematics Education (p.19–81). Springer International Publishing.
22. Kim, J. (2024). An analysis of perceptions of elementary and secondary mathematics teachers on the use of artificial intelligence in mathematics education. The Mathematical Education, 63(2), 351–368. https://doi.org/10.7468/mathedu.2024.63.2.351
23. Li, M. (2025). Integrating artificial intelligence in primary mathematics education: Investigating internal and external influences on teacher adoption. International Journal of Science and Mathematics Education, 1283–1308. https://doi.org/10.1007/s10763-024-10515-w
24. Miles, M. B. & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks: Sage.
25. Moorhouse, B. L., Wong, K. M., & Li, L. (2023). Teaching with technology in the post-pandemic digital age: Technological normalisation and AI-induced disruptions. Relc Journal, 54(2), 311–320.
26. Opesemowo, O. A. G., & Adewuyi, H. O. (2024). A systematic review of artificial intelligence in mathematics education: The emergence of 4IR. EURASIA Journal of Mathematics, Science and Technology Education, 20(7), em2478. https://doi.org/10.29333/ejmste/14762
27. Özgen, K. (2017). Matematiksel öğrenme etkinliği türlerine yönelik kuramsal bir çalışma: fonksiyon kavramı örneklemesi [A Theoretical Study On Mathematical Learning Activities: A Case Of Function Concept]. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 17(3), 1437–1464. https://doi.org/10.17240/aibuefd.2017.17.31178-338839
28. Park, H. J., & Manley, S. (2024). Using ChatGPT as a proof assistant in a mathematics pathways course. The Mathematical Education, 63(2), 139–163.
29. Rahman, M. M. & Watanobe, Y. (2023). ChatGPT for education and research: Opportunities, threats, and strategies. Applied Sciences, 13(9), 1–21. https://doi.org/10.3390/app13095783
30. Saralar-Aras, I., & Cicek Schoenberg, Y. (2024). Unveiling the synergistic nexus: AI-driven coding integration in mathematics education for enhanced computational thinking and problem-solving. The Mathematical Education, 63(2), 233–254. https://doi.org/10.7468/mathedu.2024.63.2.233
31. Seggie, N. ve Bayyurt, Y. (2017). Nitel araştırma yöntem, teknik, analiz ve yaklaşımlar (2. Baskı). Anı Yayıncılık. Shi, B., Huang, L., & Lu, X. (2020). Effect of prompt type on test-takers’ writing performance and writing strategy use in the continuation task. Language Testing, 37(3), 361–388. https://doi.org/10.1177/0265532220911626
32. Shin, B., Lee, J., & Yoo, Y. (2024). Exploring automatic scoring of mathematical descriptive assessment using prompt engineering with the GPT-4 model: Focused on permutations and combinations. The Mathematical Education, 63(2), 187–207. https://doi.org/10.63311/mathedu.2024.63.2.187
33. Şimşek, N. (2025). Integration of ChatGPT in mathematical story-focused 5E lesson planning: Teachers and pre-service teachers' interactions with ChatGPT. Education and Information Technologies, 30, 11391–11462. https://doi.org/10.1007/s10639-024-13258-x
34. Stojanov, A. (2023). Learning with ChatGPT 3.5 as a more knowledgeable other: an autoethnographic study. International Journal of Educational Technology in Higher Education, 20(1), 1–17. https://doi.org/10.1186/s41239-023-00404-7
35. Sullivan, P., Askew, M., Cheeseman, J., Clarke, D., Mornane, A., & Roche, A. (2015). Supporting teachers in structuring mathematics lessons involving challenging tasks. Journal of Mathematics Teacher Education, 18(2), 123–140. https://doi.org/10.1007/s10857-014-9279-2
36. Tapan Broutın, M. S. (2024). Exploring Mathematics Teacher Candidates’ Instrumentation Process of Generative Artificial Intelligence for Developing Lesson Plans. Yükseköğretim Dergisi, 14(1), 165-176. https://doi.org/10.53478/yuksekogretim.1347061
37. Trust, T., Whalen, J., & Mouza, C. (2023). Editorial: ChatGPT: Challenges, opportunities, and implications for teacher education. Contemporary Issues in Technology and Teacher Education, 23(1).
38. UNESCO. (2021). AI and education: Guidance for policy-makers (F. Miao, W. Holmes, R. Huang, & H. Zhang, Eds.). UNESCO. https://doi.org/10.54675/PCSP7350
39. Wardat, Y., Tashtoush, M. A., AlAli, R., & Jarrah, A. M. (2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. EURASIA Journal of Mathematics, Science and Technology Education, 19(7), em2286. https://doi.org/10.29333/ejmste/13272
40. Yanar, A. N., & Ergene, Ö. (2025). Integrating artificial intelligence in education: How preservice mathematics teachers use ChatGPT for 5E lesson plan design. Journal of Pedagogical Research, 9(2), 158–176. https://doi.org/10.33902/JPR.202533163
41. Yeo, S., Moon, J., & Kim, D. J. (2024). Transforming mathematics education with AI: Innovations, implementations, and insights. The Mathematical Education, 63(2), 387–392. https://doi.org/10.7468/mathedu.2024.63.2.387
42. Yi, L., Liu, D., Jiang, T., & Xian, Y. (2025). The effectiveness of AI on K-12 students’ mathematics learning: A systematic review and meta-analysis. International Journal of Science and Mathematics Education, 23(4), 1105–1126. https://doi.org/10.1007/s10763-024-10499-7
43. Yin, R. K. (2014). Getting started: How to know whether and when to use the case study as a research method. Case study research: design and methods, 5, 2–25.