Evaluating ChatGPT's Diagnostic Accuracy in Oral Mucosal Lesions: A Comparative Study with a Maxillofacial Surgeon

Year 2025, Volume: 52 Issue: 2, 92 - 96, 31.08.2025

Hacer Eberliköse , Arif Yiğit Güler , Raha Akbarihamed , Caner Öztürk , Hakan Alpay Karasu

https://doi.org/10.52037/eads.2025.0014

Abstract

Objective: Artificial intelligence (AI) and profound learning algorithms have been increasingly used for computerized decision-making in various complex tasks in recent years. This study aimed to compare ChatGPT (OpenAI, San Francisco, California, U.S.) with a maxillofacial surgeon to diagnose and find differential diagnoses of oral mucosal lesions and evaluate their usefulness.
Material and Methods: A maxillofacial surgeon with five years of experience and ChatGPT answered questions about twenty-three oral mucosal lesions. The lesion diagnosis is labeled as diagnosed and incapable of providing a diagnosis, and one point is awarded for each accurate differential diagnosis.
Results: While the clinician correctly diagnosed all twenty-three oral mucosal lesions included in the study, ChatGPT correctly diagnosed nineteen, and there was no statistically significant difference (P = 0.109). When the differential diagnosis results of the clinician and ChatGPT were compared, no statistically significant difference was found (P = 0.500).
Conclusion: Our study showed that a maxillofacial surgeon with five years of experience and ChatGPT showed similar results in the diagnosis and differential diagnosis of oral mucosal lesions. It will be speculated that ChatGPT can act as a new tool that provides information for patients with oral mucosal lesions. Hence, it possesses the capacity to function as a supplementary apparatus, thereby mitigating the workload encountered within the healthcare domain and enabling patients to reach preliminary evaluation from home.

Keywords

Computer-Assisted Diagnosis; , Differential Diagnosis; , Artificial intelligence

References

• Gonsalves WC, Chi AC, Neville BW. Common oral lesions: Part I. Superficial mucosal lesions. AFP. 2007;75(4):501–7.
• Khurana D, Koli A, Khatter K, Singh S. Natural language processing: state of the art, current trends and challenges. Multimed Tools Appl. 2023;82(3):3713–3744. doi:10.1007/s11042-022-13428-4.
• Kılıc MC, Bayrakdar IS, Çelik O, Bilgir E, Orhan K, Aydın OB, et al. Artificial intelligence system for automatic deciduous tooth detection and numbering in panoramic radiographs. Dentomaxillofac Radiol. 2021;50(6):20200172. doi:10.1259/dmfr.20200172.
• Lewis DD, Jones KS. Natural language processing for information retrieval. Communications of the ACM. 1996;39(1):92–101.
• Thakare AD, Laddha S, Pawar A. Hybrid Intelligent Systems for Information Retrieval. Chapman and Hall/CRC; 2022.
• Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How does ChatGPT perform on the United States Medical Licensing Examination (USMLE)? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9(1):e45312. doi:10.2196/45312.
• Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. doi:10.1371/journal.pdig.0000198.
• Alotaibi G, Awawdeh M, Farook FF, Aljohani M, Aldhafiri RM, Aldhoayan M. Artificial intelligence (AI) diagnostic tools: utilizing a convolutional neural network (CNN) to assess periodontal bone level radiographically—a retrospective study. BMC Oral Health. 2022;22(1):399. doi:10.1186/s12903-022-02436-3.
• Veerabhadrappa SK, Vengusamy S, Padarha S, Iyer K, Yadav S. Fully automated deep learning framework for detection and classification of impacted mandibular third molars in panoramic radiographs. JOMOS. 2025;31(1):7. doi:10.1051/mbcb/2025008.
• Goyal R, Jadia S, Jain L, Agarawal C. A clinical study of oral mucosal lesions in patients visiting a tertiary care centre in central India. Indian J Otolaryngol Head Neck Surg. 2016;68(4):413–416. doi:10.1007/s12070-015-0868-x.
• Gonsalves WC, Chi AC, Neville BW. Common oral lesions: Part II. Masses and neoplasia. AFP. 2007;75(4):509–512.
• Prabhu S. Handbook of oral pathology and oral medicine. John Wiley & Sons; 2021.
• Radwan-Oczko M, Sokół I, Babuśka K, Owczarek-Drabińska JE. Prevalence and characteristic of oral mucosa lesions. Symmetry. 2022;14(2):307. doi:10.3390/sym14020307.
• Krishna AB, Tanveer A, Bhagirath PV, Gannepalli A. Role of artificial intelligence in diagnostic oral pathology – A modern approach. J Oral Maxillofac Pathol. 2020;24(1):152–156. doi:10.4103/jomfp.JOMFP_215_19.
• Orhan K, Bilgir E, Bayrakdar IS, Ezhov M, Gusarev M, Shumilov E. Evaluation of artificial intelligence for detecting impacted third molars on cone-beam computed tomography scans. J Stomatol Oral Maxillofac Surg. 2021;122(4):333–337. doi:10.1016/j.jormas.2020.12.006.
• Huh S. Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study. J Educ Eval Health Prof. 2023;20(1):1. doi:10.3352/jeehp.2023.20.1.
• Warin K, Limprasert W, Suebnukarn S, Jinaporntham S, Jantana P, Vicharueang S. AI-based analysis of oral lesions using novel deep convolutional neural networks for early detection of oral cancer. PLoS One. 2022;17(8):e0273508. doi:10.1371/journal.pone.0273508.
• Dubuc A, Zitouni A, Thomas C, Kemoun P, Cousty S, Monsarrat P, et al. Improvement of mucosal lesion diagnosis with machine learning based on medical and semiological data: an observational study. J Clin Med. 2022;11(21):6596. doi:10.3390/jcm11216596.