Artificial Intelligence's Geometry Exam: Competencies in Generating Geometry Questions According to Bloom's Taxonomy

Gizem Ünsal; Behice Nur Kalkan; Nazan Sezen Yüksel

doi:10.17275/per.26.79.13.6

Artificial Intelligence's Geometry Exam: Competencies in Generating Geometry Questions According to Bloom's Taxonomy

Abstract

This study examines the extent to which generative artificial intelligence tools (ChatGPT, Gemini, and DeepSeek) can generate middle school geometry questions aligned with the cognitive levels of Bloom’s Revised Taxonomy. The study also evaluates the quality and taxonomic appropriateness of these questions based on expert judgments. A convergent parallel mixed-methods design was employed. A total of 30 geometry questions, including open-ended and multiple-choice items, were generated by the AI tools using structured prompts. Ten mathematics education experts evaluated each question using a rubric that assessed alignment with the intended Bloom level. Quantitative data were analyzed using frequencies, percentages, and Gwet’s AC1 agreement coefficient, while qualitative data were analyzed through thematic content analysis to identify patterns in expert justifications. The findings indicate that all three AI tools perform relatively well at lower and middle cognitive levels (understanding, applying, and analyzing), whereas their performance declines at higher levels (evaluating and creating). DeepSeek and Gemini showed higher overall alignment with targeted cognitive levels compared to ChatGPT. However, expert agreement decreased significantly at higher-order levels, indicating inconsistencies in how these levels are interpreted and represented. Qualitative findings revealed issues related to insufficient cognitive depth, weak distractors in multiple-choice items, and mismatches between intended and actual cognitive levels. Overall, the results highlight both the potential and limitations of generative AI in educational assessment and emphasize the necessity of expert supervision in ensuring pedagogical and taxonomic accuracy.This study compares the performance of three generative artificial intelligence tools—ChatGPT, Gemini, and DeepSeek—in generating questions aligned with Bloom’s six cognitive levels in middle school geometry. The participants consisted of ten mathematics teachers with expertise in mathematics education. The teachers evaluated the AI-generated questions for alignment with the intended cognitive level and provided judgments on their taxonomic appropriateness and overall quality. Quantitative analyses examined the agreement between the intended cognitive level and the levels assigned by participants. In addition, qualitative content analysis was used to analyze participants’ evaluations and comments. The findings highlight both the potential and limitations of generative AI tools and emphasize the importance of expert supervision in AI-supported assessment practices.

Keywords

Generative AI, Bloom Taxonomy, Middle school mathematics, Geometry, Question generation

References

Adams, N. E. (2015). Bloom’s Taxonomy of Cognitive Learning Objectives. Journal of the Medical Library Association : JMLA, 103(3), 152–153. doi:10.3163/1536-5050.103.3.010
Akdeniz, H., Clark, T., & Roberts, J. L. (2025). Can AI generate questions aligned with Bloom’s taxonomy? A framework for gifted education to support teachers. Journal of Advanced Academics, 36(4), 671-694. https://doi.org/10.1177/1932202X25134991
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives: Complete edition. Addison Wesley Longman, Inc.
Bhandari, S., Liu, Y., Kwak, Y., & Pardos, Z. A. (2024). Evaluating the psychometric properties of chatgpt-generated questions. Computers and Education: Artificial Intelligence, 7, 100284. https://doi.org/10.1016/j.caeai.2024.100284
Chen, Z. (2025). Reliable generation of isomorphic physics problems using generative AI with prompt-chaining and tool use. https://doi.org/10.48550/arXiv.2508.14755
Cohn, C., Hutchins, N., Le, T., & Biswas, G. (2024). A chain-of-thought prompting approach with LLMs for evaluating students’ formative assessment responses in science. In Proceedings of the AAAI Conference on Artificial Intelligence, 38(21), 23182–23190. doi: 10.48550/arXiv.2403.14565
Creswell, J. W., & Clark, V. L. P. (2017). Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: Sage Publications.
Debnath, T., Siddiky, M. N., Rahman, M. E., Das, P., Guha, A. K., Rahman, M. R., & Kabir, H. M. (2025). A Comprehensive Survey of Prompt Engineering Techniques in Large Language Models. doi:10.36227/techrxiv.174140719.96375390/v2
Elkins, S., Kochmar, E., Cheung, J. C. K., & Serban, I. (2024). How teachers can use large language models and Bloom’s taxonomy to create educational quizzes. Proceedings of the AAAI Conference on Artificial Intelligence, 38(21), 23084-23091. doi: 10.1609/aaai.v38i21.30353
Eren, E., Ayata, N., Çetinkaya, N. B., Yardımcı, T., Koç, A., & Akpınar, E. N. (2025). Eğitimde yapay zekâ kullanımı: Öğretmenlerin perspektifinden bir değerlendirme [The use of artificial intelligence in education: An evaluation from teachers’ perspectives]. Toplum, Eğitim ve Kültür Araştırmaları Dergisi [Journal of Society, Education and Cultural Research], 4(1), 23–38. https://doi.org/10.5281/zenodo.15460358

Feinstein, A. R., & Cicchetti, D. V. (1990). High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology, 43(6), 543–549. doi: 10.1016/0895-4356(90)90158-L
Gopal, A., Chen, P., Oh, W., Xu, S. X., & Sarker, S. (2024). On crafting effective theoretical contributions for empirical papers in economics of Information Systems: Some editorial reflections. Information Systems Research, 35(3), 917–935. doi: 10.1287/isre.2024.editorial.v35.n3
Grévisse, C., Pavlou, M. A. S. , & Schneider, J.G. (2024). Docimological quality analysis of LLM-generated multiple choice questions in computer science and medicine. Sn Comput. Sci. 5, 636. https://doi.org/10.1007/s42979-024-02963-6
Gündeğer Kılcı, C. (2025). ChatGPT vs. Deepseek: A comparative psychometric evaluation of AI tools in generating multiple-choice questions. International Journal of Assessment Tools in Education, 12(4), 1055-1079. doi: 10.21449/ijate.1674995
Gwet, K. L. (2008). Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(1), 29–48. https://doi.org/10.21449/ijate.1674995
Gwet, K. L. (2014). Handbook of inter-rater reliability: The definitive guide to measuring the extent of agreement among raters (4th edition). Advanced Analytics, LLC.
Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education, 15(3), 309–333. https://doi.org/10.1207/S15324818AME1503_5
Karabıyık, Ü. (2024). Matematik eğitiminde yenilikçi bir yaklaşım: ChatGPT’nin rolü [An innovative approach in mathematics education: The role of ChatGPT]. Uşak Üniversitesi Eğitim Araştırmaları Dergisi [Usak University Journal of Educational Research], 10(1), 26–46. https://doi.org/10.29065/usakead.1393487
Karakaya, İ., Öç, U., & Gümüş, G. (2024). Üst düzey zihinsel becerilerin ölçülmesinde açık uçlu maddelerin yapay zekâ destekli uygulamalarda hazırlanması [Developing open-ended items supported by artificial intelligence for assessing higher-order thinking skills]. Journal of Applied Measurement and Assessment, 1(2), 59–70. https://izlik.org/JA72EM45TW
Kıyak, Y. S., & Emekli, E. (2024). ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: A literature review. Postgraduate Medical Journal, 100(1189), 858–865. https://doi.org/10.1093/postmj/qgae065
Köğce, D., & Baki, A. (2009). Matematik öğretmenlerinin yazılı sınav soruları ile ÖSS sınavlarında sorulan matematik sorularının Bloom taksonomisine göre karşılaştırılması [A comparison of high-school mathematics teachers’ examination questions and mathematics questions in university entrance examinations according to Bloom’s taxonomy]. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi [Pamukkale University Journal of Education], 26(26), 70–80. https://izlik.org/JA53FX47SB
Koman, I., Bayrakceken, S., Oktay, O., & Canpolat, N. (2023). Evaluation of science teachers' exam and high school entrance exam science questions based on the revised Bloom taxonomy. Science Insights Education Frontiers, 16(2), 2505-2536. doi: 10.15354/sief.23.or335
Kotsis, K. T. (2025). Artificial intelligence as a catalyst for changes in university-level science education. EIKI Journal of Effective Teaching Methods, 3(3). https://doi.org/10.59652/jetm.v3i3.618
Lee, J., Smith, D., Woodhead, S., & Lan, A. (2024). Math multiple choice question generation via human-large language model collaboration. ArXiv Preprint Arxiv:2405.00864.
Li, H., Fang, Y., Zhang, S., Lee, S. M., Wang, Y., Trexler, M., & Botelho, A. F. (2025). ARCHED: A human-centered framework for transparent, responsible, and collaborative AI-assisted instructional design (ArXiv:2503.08931). https://doi.org/10.48550/arXiv.2503.08931
Liu, Y., Zheng, Z., Deng, N., & Li, T. (2026). How ai-driven technological transformation advances the modernization integrated transition of Energy. Economic Analysis and Policy, 91, 206–223. https://doi.org/10.1016/j.eap.2026.03.003
Maity, S., & Deroy, A. (2024). Human-centric explainable AI in education. Arxiv Preprint Arxiv:2410.19822. https://doi.org/10.48550/arXiv.2410.19822
May, T. A., Fan, Y. K., Stone, G. E., Koskey, K. L. K., Sondergeld, C. J., Folger, T. D., Archer, J. N., Provinzano, K., & Johnson, C. C. (2025). An effectiveness study of generative artificial intelligence tools used to develop multiple-choice test items. Education Sciences, 15(2), 144. doi: 10.3390/educsci15020144
McGalliard, W., & Otten, S. (2025). AI responses to challenging problems and educator responses to AI availability. Digital Experiences in Mathematics Education, 11, 319–332. https://doi.org/10.1007/s40751-024-00167-4
McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://pmc.ncbi.nlm.nih.gov/articles/PMC3900052/
Nayıroğlu, B., & Tutak T. (2023). The effect of using Web 2.0 tools in algebra teaching on student success and attitude. International e-Journal of Educational Studies, 7(14), 416–425. https://doi.org/10.31458/iejes.1270732
Omopekunola, M. O., & Kardanova, E. Y. (2024). Automatic generation of physics items with large language models (LLMs). REID (Research and Evaluation İn Education), 10(2), 4. https://doi.org/10.21831/reid.v10i2.76864
Organisation for Economic Co-operation and Development (OECD). (2023). Education at a glance 2023: OECD indicators. OECD Publishing. https://doi.org/10.1787/e13bef63-en
Owan, V. J., Abang, K. B., Idika, D. O., Etta, E. O., & Bassey, B. A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. Eurasia Journal of Mathematics, Science and Technology Education, 19(8), https://doi.org/10.29333/ejmste/13428
Rahman, A. B. M., Yang, J., Zhan, Z., Hossain, M. K., Xu, Y., & Feng, W. (2025). Pedagogical transformation through artificial intelligence: Implementation barriers and adoption strategies in business education. ICEB 2025 Proceedings, 50, 314-322. Retrieved from https://aisel.aisnet.org/iceb2025/50
Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education, 26, 582-599. doi: 10.1007/S40593-016-0110-3
Scaria, N., Dharani Chenna, S., & Subramani, D. (2024). Automated educational question generation at different Bloom’s skill levels using large language models: Strategies and evaluation. In Artificial intelligence in education (Lecture Notes in Computer Science, Vol. 14830, pp. 165–179). Springer. https://doi.org/10.1007/978-3-031-64299-9_12
Shaunessy, E. (2000). Techniques questioning in the gifted classroom? Gifted Child Today, 23(5), 14–21. doi: 10.4219/Gct-2000-752
Siegle, D. (2024). Using artificial intelligence (AI) technology to support the three legs of talent development. Gifted Child Today, 47(3), 221–227. doi: 10.1177/10762175241242495
Tan, X., Cheng, G., & Ling, M. H. (2025). Artificial Intelligence in teaching and teacher professional development: A systematic review. Computers and Education: Artificial Intelligence, 8, 100355. doi:10.1016/j.caeai.2024.100355
Vantassel-Baska, J. (2014). Curriculum issues: Artful inquiry: the use of questions in working with the gifted. Gifted Child Today, 37(1), 48–50. doi: 10.1177/1076217513509621
Wan, X., Sun, R., Nakhost, H., Dai, H., Eisenschlos, J. M., Arık, S. Ö., & Pfister, T. (2023). Universal self-adaptive prompting. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 7437–7462). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.461
Wankhede, H. S., & Kiwelekar, A. W. (2016). Qualitative assessment of software engineering examination questions with Bloom’s taxonomy. Indian Journal of Science and Technology, 9(6). doi: 10.17485/ijst/2016/v9i6/85012
Webb, N. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. National Institute for Science Education Madison, WI. Retrieved from http://facstaff.wceruw.org/ normw/WEBBMonograph6criteria.pdf.
Wongpakaran, N., Ward, P. R., Wongpakaran, T., & Sirisengtaksin, V. (2013). A comparison of reliability methods for psychiatric diagnosis: Cohen's kappa, Gwet's AC1, and Fleiss' kappa. Annals of General Psychiatry, 12(1), Article 17. doi: 10.1186/1744-859X-12-17
Xu, W., & Ouyang, F. (2022). The application of AI technologies in STEM education: A systematic review from 2011 to 2021. International Journal of STEM Education, 9, Article 59. https://doi.org/10.1186/s40594-022-00377-5
Yaacoub, A., Assaghir, Z., Prevost, L., & Da-Rugna, J. (2025a). Analyzing feedback mechanisms in AI-generated MCQS: Insights into readability, lexical properties, and levels of challenge. Arxiv Preprint Arxiv:2504.21013. https://doi.org/10.48550/arXiv.2504.21013
Yaacoub, A., Da-Rugna, J., & Assaghir, Z. (2025b). Assessing AI-generated questions’ alignment with cognitive frameworks in educational assessment. International Journal of Computer Theory and Engineering, 17 (3), 114-125. https://doi.org/10.7763/IJCTE.2025.V17.1374
Yoşumaz, İ. (2025). Generative artificial intelligence and usage in academia. Fırat Üniversitesi Sosyal Bilimler Dergisi [Firat University Journal of Social Sciences], 35(1), 1–24. https://doi.org/10.18069/firatsbed.1423208
Zhang, X., Zhang, P., Shen, Y., Wang, Y., Li, M., & Chen, H. (2024). A systematic literature review of empirical research on applying generative artificial intelligence in education. Frontiers of Digital Education, 1(3), 223–245. doi:10.1007/s44366-024-0028-5

Details

Primary Language

English

Subjects

Instructional Technologies

Journal Section

Research Article

Authors

Gizem Ünsal ^*
0000-0002-4379-423X
Türkiye

Behice Nur Kalkan
0009-0003-5017-6346
Türkiye

Nazan Sezen Yüksel
0000-0002-0539-3785
Türkiye

Publication Date

November 1, 2026

Submission Date

March 10, 2026

Acceptance Date

July 8, 2026

Published in Issue

Year 2026 Volume: 13 Number: 6

DOI

https://doi.org/10.17275/per.26.79.13.6

IZ

https://izlik.org/JA35TP94TW

APA

Ünsal, G., Kalkan, B. N., & Sezen Yüksel, N. (2026). Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy. Participatory Educational Research, 13(6), 77-100. https://doi.org/10.17275/per.26.79.13.6

AMA

1.Ünsal G, Kalkan BN, Sezen Yüksel N. Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy. PER. 2026;13(6):77-100. doi:10.17275/per.26.79.13.6

Chicago

Ünsal, Gizem, Behice Nur Kalkan, and Nazan Sezen Yüksel. 2026. “Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy”. Participatory Educational Research 13 (6): 77-100. https://doi.org/10.17275/per.26.79.13.6.

EndNote

Ünsal G, Kalkan BN, Sezen Yüksel N (November 1, 2026) Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy. Participatory Educational Research 13 6 77–100.

IEEE

[1]G. Ünsal, B. N. Kalkan, and N. Sezen Yüksel, “Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy”, PER, vol. 13, no. 6, pp. 77–100, Nov. 2026, doi: 10.17275/per.26.79.13.6.

ISNAD

Ünsal, Gizem - Kalkan, Behice Nur - Sezen Yüksel, Nazan. “Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy”. Participatory Educational Research 13/6 (November 1, 2026): 77-100. https://doi.org/10.17275/per.26.79.13.6.

JAMA

1.Ünsal G, Kalkan BN, Sezen Yüksel N. Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy. PER. 2026;13:77–100.

MLA

Ünsal, Gizem, et al. “Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy”. Participatory Educational Research, vol. 13, no. 6, Nov. 2026, pp. 77-100, doi:10.17275/per.26.79.13.6.

Vancouver

1.Gizem Ünsal, Behice Nur Kalkan, Nazan Sezen Yüksel. Artificial Intelligence’s Geometry Exam: Competencies in Generating Geometry Questions According to Bloom’s Taxonomy. PER. 2026 Nov. 1;13(6):77-100. doi:10.17275/per.26.79.13.6

27707

Creative Commons Atıf 4.0 International Lisance

download?token=eyJhdXRoX3JvbGVzIjpbXSwiZW5kcG9pbnQiOiJqb3VybmFsIiwib3JpZ2luYWxuYW1lIjoicGVyIHNjb3B1cyBpc3RhdHRpc3Rpay5wbmciLCJwYXRoIjoiOTg5Ni9mN2E1LzU3ZDcvNmEyMmIyZmFlMGJkYjYuMDUyNjIyMDMucG5nIiwiZXhwIjoxNzgwNjYyNTM5LCJub25jZSI6IjcxYzczMjhhOTdkOWYxM2ExZjE2NWExYjQyMzMyOWJhIn0.g4uvJoi6Oy8W0gRoUNW-d5FO9m4u30pEA0l1h8DwzSE

Artificial Intelligence's Geometry Exam: Competencies in Generating Geometry Questions According to Bloom's Taxonomy

Abstract

Keywords

References

Details

Primary Language

Subjects

Journal Section

Authors

Publication Date

Submission Date

Acceptance Date

Published in Issue

DOI

IZ

Cite