Educational quality and cognitive alignment of GPT-5-generated head and neck anatomy multiple-choice questions

Abstract

Objectives: Large language models (LLMs) have emerged as promising tools for generating educational assessment items; however, their ability to produce multiple-choice questions (MCQs) that accurately reflect intended cognitive levels remains uncertain. This study evaluated the educational quality and cognitive alignment of head and neck anatomy MCQs generated by GPT-5 (Generative Pre-trained Transformer 5) using Bloom’s Revised Taxonomy. Methods: Fifty head and neck anatomy MCQs were generated by GPT-5 using structured prompts specifying the anatomical topic, the intended Bloom level, and undergraduate learning objectives. Two experienced anatomists independently evaluated each question for clarity, scientific accuracy, curriculum relevance, and suitability for its intended Bloom level using a five-point Likert scale. Inter-rater reliability was assessed using intraclass correlation coefficients (ICCs), and content validity was evaluated using item- and scale-level content validity indices (I-CVI and S-CVI). To examine cognitive alignment independently, all MCQs subsequently underwent a blinded consensus Bloom-level classification, in which two anatomists independently assigned Bloom levels before resolving disagreements through discussion. Agreement between the intended and consensus-assigned Bloom levels was assessed using Cohen’s κ and weighted κ statistics. Results: GPT-5-generated MCQs received high pooled ratings for clarity (4.42±1.02), scientific accuracy (4.55±0.97), curriculum relevance (4.51±1.11), and suitability for the intended Bloom level (4.29±1.11). Inter-rater agreement ranged from good to excellent across all domains (ICC=0.717–0.945). Content validity was high, with S-CVI/Ave values ranging from 0.82 to 0.89. Blinded consensus Bloom classification showed 92.0% exact agreement between the intended and expert-assigned Bloom levels (Cohen’s κ=0.900; linear weighted κ=0.913; quadratic weighted κ=0.926). All four discrepancies represented reassignment to a lower cognitive level; two involved adjacent Bloom categories and two spanned more than one category. Conclusion: GPT-5 generated head and neck anatomy MCQs that experts considered clear, scientifically accurate, educationally relevant, and generally appropriate for their intended cognitive level. The high level of agreement observed during blinded consensus Bloom classification is consistent with structured prompting largely preserving intended cognitive objectives, although this preservation was not uniform at higher cognitive levels. Expert review therefore remains essential, particularly for optimising higher-order cognitive complexity and distractor quality before incorporation into undergraduate assessments.

Keywords

• anatomy education
• artificial intelligence
• Bloom’s Revised Taxonomy
• GPT-5
• large language models
• medical education
• multiple-choice questions
• multiple-choice questions

References

1. Montello MB, Bezerra de Moura SA, Abreu BJ. Bloom’s taxonomy as a tool for educational objectives applied to human anatomy. Morphologie 2025;109:100943.
2. Estai M, Bunt S. Best teaching practices in anatomy education: a critical review. Ann Anat 2016;208:151–7.
3. Ghosh SK. Cadaveric dissection as an educational tool for anatomical sciences in the 21st century. Anat Sci Educ 2017;10:286–99.
4. Chytas D, Salmas M, Demesticha T, Troupis T. Technological resources in anatomy education: are there any which are worth the effort more than others? Anat Sci Educ 2023;16:187–8.
5. Shiozawa T, Butz B, Herlan S, Kramer A, Hirt B. Interactive anatomical and surgical live stream lectures improve students’ academic performance in applied clinical anatomy. Anat Sci Educ 2017;10:46–52.
6. Choi-Lundberg DL, Cuellar WA, Williams AM. Online dissection audio-visual resources for human anatomy: undergraduate medical students’ usage and learning outcomes. Anat Sci Educ 2016;9:545–54.
7. Thompson AR, Lake LPO. Relationship between learning approach, Bloom’s taxonomy, and student performance in an undergraduate human anatomy course. Adv Health Sci Educ Theory Pract 2023;28:1115–30.
8. Tekin A, Karamus NF, Çolak T. Anatomy exam model for the circulatory and respiratory systems using GPT-4: a medical school study. Surg Radiol Anat 2025;47:158.

9. Herrmann-Werner A, Festl-Wietek T, Holderried F, Herschbach L, Griewatz J, Masters K, Zipfel S, Mahling M. Assessing ChatGPT’s mastery of Bloom’s taxonomy using psychosomatic medicine exam questions: mixed-methods study. J Med Internet Res 2024;26:e52113.
10. Bharatha A, Ojeh N, Fazle Rabbi AM, Campbell MH, Krishnamurthy K, Layne-Yarde RNA, Kumar A, Springer DCR, Connell KL, Majumder MAA. Comparing the performance of ChatGPT-4 and medical students on MCQs at varied levels of Bloom’s taxonomy. Adv Med Educ Pract 2024;15:393–400.
11. Sallam M, Al-Salahat K. Below average ChatGPT performance in medical microbiology exam compared to university students. Front Educ 2023;8:1333415.
12. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, Tseng V. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health 2023;2:e0000198.
13. Kıyak YS, Coşkun Ö, Budakoğlu İİ, Uluoğlu C. ChatGPT for generating multiple-choice questions: evidence on the use of artificial intelligence in automatic item generation for a rational pharmacotherapy exam. Eur J Clin Pharmacol 2024;80:729–35.
14. Newton PM, Xiromeriti M. ChatGPT performance on multiple choice question examinations in higher education: a pragmatic scoping review. Assessment and Evaluation in Higher Education 2024;49:781–98.
15. Abouzeid E, Wassef R, Jawwad A, Harris P. Chatbots’ role in generating single best answer questions for undergraduate medical student assessment: comparative analysis. JMIR Med Educ 2025;11:e69521.
16. Balu A, Prvulovic ST, Fernandez Perez C, Kim A, Donoho DA, Keating G. Evaluating the value of AI-generated questions for USMLE Step 1 preparation: a study using ChatGPT-3.5. Med Teach 2025;47:1645–53.
17. Law AK, So J, Lui CT, Choi YF, Cheung KH, Hung KK, Graham CA. AI versus human-generated multiple-choice questions for medical education: a cohort study in a high-stakes examination. BMC Med Educ 2025;25:208.
18. So CC, Lee CH, Yang D, Loh AWK, Sze CK. Automatic generation of high-quality MCQs with large language models for artificial intelligence education. IEEE Access 2025;13:184332–47.
19. Yaacoub A, Da-Rugna J, Assaghir Z. Assessing AI-generated questions’ alignment with cognitive frameworks in educational assessment. International Journal of Computer Theory and Engineering 2025;17:114–25.
20. Indran IR, Paranthaman P, Gupta N, Mustafa N. Twelve tips to leverage AI for efficient and effective medical question generation: a guide for educators using ChatGPT. Med Teach 2024;46:1021–6.
21. Hwang K, Challagundla S, Alomair M, Chen LK, Choa FS. Towards AI-assisted multiple choice question generation and quality evaluation at scale: aligning with Bloom’s taxonomy. Proceedings of the NeurIPS Workshop on Generative AI for Education: Advances, Opportunities, and Challenges; 2023; New Orleans, LA, USA.
22. Kazemi Vanhari F, Anand C, Welch C. Analyzing interview questions via Bloom’s taxonomy to enhance the design thinking process. In: Kochmar E, Alhafni B, Bexte M, editors. Proceedings of the 20th workshop on innovative use of NLP for building educational applications (BEA 2025). Association for Computational Linguistics; 2025. p. 582–93.
23. Boufrikha W, Sallem A, Laabidi B, Mallek R, Slama N, Youssef S, Majdoub A, Salem N, Boukhris S. Evaluation of three artificial intelligence chatbots for generating clinical hematology multiple-choice questions for medical students. Sci Rep 2026;16:5802.
24. Nguyen TT, Nguyen L, Do HTN, Nguyen HTT, Tong SM. Evaluating cognitive depth of AI-generated multiple-choice questions with Bloom’s taxonomy. PLoS One 2026;21:e0341317.
25. García-Ramírez Y. Can ChatGPT-generated exams using Bloom’s taxonomy effectively assess cognitive learning outcomes in road engineering education? Proc Eng Sci 2026;8:591–6.
26. Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel) 2023;11:887.