Evaluating ChatGPT's Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon

Hacer Eberliköse; Arif Yiğit Güler; Raha Akbarihamed; Caner Öztürk; Hakan Alpay Karasu

doi:10.52037/eads.2025.0014

Evaluating ChatGPT's Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon

Abstract

Objective: Artificial intelligence (AI) and profound learning algorithms have been increasingly used for computerized decision-making in various complex tasks in recent years. This study aimed to compare ChatGPT (OpenAI, San Francisco, California, U.S.) with a maxillofacial surgeon to diagnose and find differential diagnoses of oral mucosal lesions and evaluate their usefulness. Material and Methods: A maxillofacial surgeon with five years of experience and ChatGPT answered questions about twenty-three oral mucosal lesions. The lesion diagnosis is labeled as diagnosed and incapable of providing a diagnosis, and one point is awarded for each accurate differential diagnosis. Results: While the clinician correctly diagnosed all twenty-three oral mucosal lesions included in the study, ChatGPT correctly diagnosed nineteen, and there was no statistically significant difference (P = 0.109). When the differential diagnosis results of the clinician and ChatGPT were compared, no statistically significant difference was found (P = 0.500). Conclusion: Our study showed that a maxillofacial surgeon with five years of experience and ChatGPT showed similar results in the diagnosis and differential diagnosis of oral mucosal lesions. It will be speculated that ChatGPT can act as a new tool that provides information for patients with oral mucosal lesions. Hence, it possesses the capacity to function as a supplementary apparatus, thereby mitigating the workload encountered within the healthcare domain and enabling patients to reach preliminary evaluation from home.

Keywords

References

Gonsalves WC, Chi AC, Neville BW. Common oral lesions: Part I. Superficial mucosal lesions. AFP. 2007;75(4):501–7.
Khurana D, Koli A, Khatter K, Singh S. Natural language processing: state of the art, current trends and challenges. Multimed Tools Appl. 2023;82(3):3713–3744. doi:10.1007/s11042-022-13428-4.
Kılıc MC, Bayrakdar IS, Çelik O, Bilgir E, Orhan K, Aydın OB, et al. Artificial intelligence system for automatic deciduous tooth detection and numbering in panoramic radiographs. Dentomaxillofac Radiol. 2021;50(6):20200172. doi:10.1259/dmfr.20200172.
Lewis DD, Jones KS. Natural language processing for information retrieval. Communications of the ACM. 1996;39(1):92–101.
Thakare AD, Laddha S, Pawar A. Hybrid Intelligent Systems for Information Retrieval. Chapman and Hall/CRC; 2022.
Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How does ChatGPT perform on the United States Medical Licensing Examination (USMLE)? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9(1):e45312. doi:10.2196/45312.
Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. doi:10.1371/journal.pdig.0000198.
Alotaibi G, Awawdeh M, Farook FF, Aljohani M, Aldhafiri RM, Aldhoayan M. Artificial intelligence (AI) diagnostic tools: utilizing a convolutional neural network (CNN) to assess periodontal bone level radiographically—a retrospective study. BMC Oral Health. 2022;22(1):399. doi:10.1186/s12903-022-02436-3.

Veerabhadrappa SK, Vengusamy S, Padarha S, Iyer K, Yadav S. Fully automated deep learning framework for detection and classification of impacted mandibular third molars in panoramic radiographs. JOMOS. 2025;31(1):7. doi:10.1051/mbcb/2025008.
Goyal R, Jadia S, Jain L, Agarawal C. A clinical study of oral mucosal lesions in patients visiting a tertiary care centre in central India. Indian J Otolaryngol Head Neck Surg. 2016;68(4):413–416. doi:10.1007/s12070-015-0868-x.
Gonsalves WC, Chi AC, Neville BW. Common oral lesions: Part II. Masses and neoplasia. AFP. 2007;75(4):509–512.
Prabhu S. Handbook of oral pathology and oral medicine. John Wiley & Sons; 2021.
Radwan-Oczko M, Sokół I, Babuśka K, Owczarek-Drabińska JE. Prevalence and characteristic of oral mucosa lesions. Symmetry. 2022;14(2):307. doi:10.3390/sym14020307.
Krishna AB, Tanveer A, Bhagirath PV, Gannepalli A. Role of artificial intelligence in diagnostic oral pathology – A modern approach. J Oral Maxillofac Pathol. 2020;24(1):152–156. doi:10.4103/jomfp.JOMFP_215_19.
Orhan K, Bilgir E, Bayrakdar IS, Ezhov M, Gusarev M, Shumilov E. Evaluation of artificial intelligence for detecting impacted third molars on cone-beam computed tomography scans. J Stomatol Oral Maxillofac Surg. 2021;122(4):333–337. doi:10.1016/j.jormas.2020.12.006.
Huh S. Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study. J Educ Eval Health Prof. 2023;20(1):1. doi:10.3352/jeehp.2023.20.1.
Warin K, Limprasert W, Suebnukarn S, Jinaporntham S, Jantana P, Vicharueang S. AI-based analysis of oral lesions using novel deep convolutional neural networks for early detection of oral cancer. PLoS One. 2022;17(8):e0273508. doi:10.1371/journal.pone.0273508.
Dubuc A, Zitouni A, Thomas C, Kemoun P, Cousty S, Monsarrat P, et al. Improvement of mucosal lesion diagnosis with machine learning based on medical and semiological data: an observational study. J Clin Med. 2022;11(21):6596. doi:10.3390/jcm11216596.

Details

Primary Language

English

Subjects

Surgery (Other), Oral Medicine and Pathology

Journal Section

Research Article

Authors

Hacer Eberliköse ^*
0000-0003-1450-1968
Türkiye

Arif Yiğit Güler
0000-0002-9962-4926
Türkiye

Raha Akbarihamed
0000-0003-3401-2117
Türkiye

Caner Öztürk
0000-0001-9549-2770
Türkiye

Hakan Alpay Karasu
0000-0003-1867-0347
Türkiye

Early Pub Date

August 30, 2025

Publication Date

August 31, 2025

Submission Date

December 16, 2024

Acceptance Date

June 26, 2025

Published in Issue

Year 2025 Volume: 52 Number: 2

DOI

https://doi.org/10.52037/eads.2025.0014

IZ

https://izlik.org/JA85LU49LE

Cite

RIS / Bibtex

APA

Eberliköse, H., Güler, A. Y., Akbarihamed, R., Öztürk, C., & Karasu, H. A. (2025). Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon. European Annals of Dental Sciences, 52(2), 92-96. https://doi.org/10.52037/eads.2025.0014

AMA

1.Eberliköse H, Güler AY, Akbarihamed R, Öztürk C, Karasu HA. Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon. EADS. 2025;52(2):92-96. doi:10.52037/eads.2025.0014

Chicago

Eberliköse, Hacer, Arif Yiğit Güler, Raha Akbarihamed, Caner Öztürk, and Hakan Alpay Karasu. 2025. “Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study With a Maxillofacial Surgeon”. European Annals of Dental Sciences 52 (2): 92-96. https://doi.org/10.52037/eads.2025.0014.

EndNote

Eberliköse H, Güler AY, Akbarihamed R, Öztürk C, Karasu HA (August 1, 2025) Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon. European Annals of Dental Sciences 52 2 92–96.

IEEE

[1]H. Eberliköse, A. Y. Güler, R. Akbarihamed, C. Öztürk, and H. A. Karasu, “Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon”, EADS, vol. 52, no. 2, pp. 92–96, Aug. 2025, doi: 10.52037/eads.2025.0014.

ISNAD

Eberliköse, Hacer - Güler, Arif Yiğit - Akbarihamed, Raha - Öztürk, Caner - Karasu, Hakan Alpay. “Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study With a Maxillofacial Surgeon”. European Annals of Dental Sciences 52/2 (August 1, 2025): 92-96. https://doi.org/10.52037/eads.2025.0014.

JAMA

1.Eberliköse H, Güler AY, Akbarihamed R, Öztürk C, Karasu HA. Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon. EADS. 2025;52:92–96.

MLA

Eberliköse, Hacer, et al. “Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study With a Maxillofacial Surgeon”. European Annals of Dental Sciences, vol. 52, no. 2, Aug. 2025, pp. 92-96, doi:10.52037/eads.2025.0014.

Vancouver

1.Hacer Eberliköse, Arif Yiğit Güler, Raha Akbarihamed, Caner Öztürk, Hakan Alpay Karasu. Evaluating ChatGPT’s Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon. EADS. 2025 Aug. 1;52(2):92-6. doi:10.52037/eads.2025.0014