Research Article
BibTex RIS Cite

Evaluation of ChatGPT as a Multiple-Choice Question Generator in Dental Traumatology

Year 2024, Volume: 6 Issue: 2, 235 - 238, 16.05.2024
https://doi.org/10.37990/medr.1446396

Abstract

Aim: This study aims to evaluate the ability of ChatGPT-4 to generate clinical case-based multiple-choice questions in dental traumatology.
Material and Method: International Association of Dental Traumatology guidelines for the management of traumatic dental injuries were introduced to ChatGPT-4 as an information source and ChatGPT-4 was prompted to 20 generate questions in fractures and luxations, avulsion of permanent teeth, injuries in the primary dentition. Sixty questions in total were generated by ChatGPT and the questions were evaluated by 2 endodontists using a 3-point scale. The One-way analysis of variance and Post Hoc Tukey test were used to analyze the data and the significance was P<0.05.
Results: The average time to generate 20 questions was 1 min 55 sec. It was noted that 52% of the questions were usable without modification or with minor changes, while 28% were incorrect or completely useless.
Conclusion: Despite the flaws, ChatGPT can be useful for creating multiple-choice questions in dental traumatology after a rigorous evaluation, elimination, and development procedure.

Ethical Statement

This study does not require an ethical approval

References

  • Javaeed A. Assessment of higher ordered thinking in medical education: multiple choice questions and modified essay questions. MedEdPublish. 2018;7:128.
  • Scully D. Constructing multiple-choice items to measure higher-order thinking. PARE. 2019;22:4.
  • Scouller K. The influence of assessment method on students' learning approaches: multiple choice question examination versus assignment essay. Higher Education; Dordrecht. 1998;35:453-72.
  • Rao SK, Kimball AB, Lehrhoff SR, et al. The impact of administrative burden on academic physicians: results of a hospital-wide physician survey. Acad Med. 2017;92:237-43.
  • Chen L, Chen P, Lin Z. Artificial intelligence in education: a review. IEEE Access. 2020;8:75264-78.
  • Cardoso MJ, Houssami N, Pozzi G, Séroussi B. Artificial intelligence (AI) in breast cancer care-leveraging multidisciplinary skills to improve care. Artif Intell Med. 2022;123:102215.
  • Abd-Alrazaq A, AlSaad R, Alhuwail D, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023;9:e48291.
  • Mesko B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med. 2023;6:120.
  • Fatani B. ChatGPT for future medical and dental research. Cureus. 2023;15:e37285.
  • Giannakopoulos K, Kavadella A, Stamatopoulos V, Kaklamanos E. Evaluation of generative artificial intelligence large language models ChatGPT, Google Bard, and Microsoft Bing Chat in supporting evidence-based dentistry: a comparative mixed-methods study. J Med Internet Res. 2023;25:e51580.
  • Acar AH. Can natural language processing serve as a consultant in oral surgery?. J Stomatol Oral Maxillofac Surg. 2024;125:101724.
  • Suarez A, Diaz-Flores Garcia V, Algar J, et al. Unveiling the ChatGPT phenomenon: evaluating the consistency and accuracy of endodontic question answers. Int Endod J. 2024;57:108-13.
  • Ali K, Barhom N, Tamimi F, Duggal M. ChatGPT-A double-edged sword for healthcare education? Implications for assessments of dental students. Eur J Dent Educ. 2024;28:206-11.
  • Cheung BHH, Lau GKK, Wong GTC, et al. ChatGPT versus human in generating medical graduate exam multiple choice questions-A multinational prospective study (Hong Kong S.A.R., Singapore, Ireland, and the United Kingdom). PLoS One. 2023;18:e0290691.
  • Ayub I, Hamann D, Hamann CR, Davis MJ. Exploring the potential and limitations of chat generative pre-trained transformer (ChatGPT) in generating board-style dermatology questions: a qualitative analysis. Cureus. 2023;15:e43717.
  • Kim HW, Shin DH, Kim J, et al. Assessing the performance of ChatGPT's responses to questions related to epilepsy: a cross-sectional study on natural language processing and medical information retrieval. Seizure. 2024;114:1-8.
  • Spallek S, Birrell L, Kershaw S, et al. Can we use ChatGPT for mental health and substance use education? Examining its quality and potential harms. JMIR Med Educ. 2023;9:e51243.
  • Bourguignon C, Cohenca N, Lauridsen E, et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 1. fractures and luxations. Dent Traumatol. 2020;4:314-30.
  • Levin L, Day PF, Hicks L, et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: general introduction. Dent Traumatol. 2020;4:309-13.
  • Fouad AF, Abbott PV, Tsilingaridis G, et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 2. avulsion of permanent teeth. Dent Traumatol. 2020;36:331-42.
  • Day PF, Flores MT, O'Connell AC, et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 3. injuries in the primary dentition. Dent Traumatol. 2020;36:343-59.
  • Ngo A, Gupta S, Perrine O, et al. ChatGPT 3.5 fails to write appropriate multiple choice practice exam questions. Acad Pathol. 2023;11:100099.
Year 2024, Volume: 6 Issue: 2, 235 - 238, 16.05.2024
https://doi.org/10.37990/medr.1446396

Abstract

References

  • Javaeed A. Assessment of higher ordered thinking in medical education: multiple choice questions and modified essay questions. MedEdPublish. 2018;7:128.
  • Scully D. Constructing multiple-choice items to measure higher-order thinking. PARE. 2019;22:4.
  • Scouller K. The influence of assessment method on students' learning approaches: multiple choice question examination versus assignment essay. Higher Education; Dordrecht. 1998;35:453-72.
  • Rao SK, Kimball AB, Lehrhoff SR, et al. The impact of administrative burden on academic physicians: results of a hospital-wide physician survey. Acad Med. 2017;92:237-43.
  • Chen L, Chen P, Lin Z. Artificial intelligence in education: a review. IEEE Access. 2020;8:75264-78.
  • Cardoso MJ, Houssami N, Pozzi G, Séroussi B. Artificial intelligence (AI) in breast cancer care-leveraging multidisciplinary skills to improve care. Artif Intell Med. 2022;123:102215.
  • Abd-Alrazaq A, AlSaad R, Alhuwail D, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023;9:e48291.
  • Mesko B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med. 2023;6:120.
  • Fatani B. ChatGPT for future medical and dental research. Cureus. 2023;15:e37285.
  • Giannakopoulos K, Kavadella A, Stamatopoulos V, Kaklamanos E. Evaluation of generative artificial intelligence large language models ChatGPT, Google Bard, and Microsoft Bing Chat in supporting evidence-based dentistry: a comparative mixed-methods study. J Med Internet Res. 2023;25:e51580.
  • Acar AH. Can natural language processing serve as a consultant in oral surgery?. J Stomatol Oral Maxillofac Surg. 2024;125:101724.
  • Suarez A, Diaz-Flores Garcia V, Algar J, et al. Unveiling the ChatGPT phenomenon: evaluating the consistency and accuracy of endodontic question answers. Int Endod J. 2024;57:108-13.
  • Ali K, Barhom N, Tamimi F, Duggal M. ChatGPT-A double-edged sword for healthcare education? Implications for assessments of dental students. Eur J Dent Educ. 2024;28:206-11.
  • Cheung BHH, Lau GKK, Wong GTC, et al. ChatGPT versus human in generating medical graduate exam multiple choice questions-A multinational prospective study (Hong Kong S.A.R., Singapore, Ireland, and the United Kingdom). PLoS One. 2023;18:e0290691.
  • Ayub I, Hamann D, Hamann CR, Davis MJ. Exploring the potential and limitations of chat generative pre-trained transformer (ChatGPT) in generating board-style dermatology questions: a qualitative analysis. Cureus. 2023;15:e43717.
  • Kim HW, Shin DH, Kim J, et al. Assessing the performance of ChatGPT's responses to questions related to epilepsy: a cross-sectional study on natural language processing and medical information retrieval. Seizure. 2024;114:1-8.
  • Spallek S, Birrell L, Kershaw S, et al. Can we use ChatGPT for mental health and substance use education? Examining its quality and potential harms. JMIR Med Educ. 2023;9:e51243.
  • Bourguignon C, Cohenca N, Lauridsen E, et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 1. fractures and luxations. Dent Traumatol. 2020;4:314-30.
  • Levin L, Day PF, Hicks L, et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: general introduction. Dent Traumatol. 2020;4:309-13.
  • Fouad AF, Abbott PV, Tsilingaridis G, et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 2. avulsion of permanent teeth. Dent Traumatol. 2020;36:331-42.
  • Day PF, Flores MT, O'Connell AC, et al. International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 3. injuries in the primary dentition. Dent Traumatol. 2020;36:343-59.
  • Ngo A, Gupta S, Perrine O, et al. ChatGPT 3.5 fails to write appropriate multiple choice practice exam questions. Acad Pathol. 2023;11:100099.
There are 22 citations in total.

Details

Primary Language English
Subjects Endodontics
Journal Section Original Articles
Authors

Yağız Özbay 0000-0003-2028-8120

Publication Date May 16, 2024
Submission Date March 3, 2024
Acceptance Date April 19, 2024
Published in Issue Year 2024 Volume: 6 Issue: 2

Cite

AMA Özbay Y. Evaluation of ChatGPT as a Multiple-Choice Question Generator in Dental Traumatology. Med Records. May 2024;6(2):235-238. doi:10.37990/medr.1446396

17741

Chief Editors

Assoc. Prof. Zülal Öner
Address: İzmir Bakırçay University, Department of Anatomy, İzmir, Türkiye

Assoc. Prof. Deniz Şenol
Address: Düzce University, Department of Anatomy, Düzce, Türkiye

E-mail: medrecsjournal@gmail.com

Publisher:
Medical Records Association (Tıbbi Kayıtlar Derneği)
Address: Düzce / Türkiye

Publication Support:

Effect Publishing & Agency
Phone: + 90 (553) 610 67 80
E-mail: info@effectpublishing.com