İmplant Hastalıkları için Risk Değerlendirme Sistemi (IDRA), ChatGPT ve DeepSeek’in Peri-implant Hastalık Riski Değerlendirmesindeki Uyumunun Karşılaştırılması: 500 Vaka Senaryosu Çalışması

Öz

Amaç Bu çalışmanın amacı, İmplant Hastalıkları için Risk Değerlendirme Sistemi (IDRA) ile yapay zeka tabanlı dil modellerinin, bireyleri düşük, orta ve yüksek peri-implant hastalık riski kategorilerine sınıflandırmadaki uyum düzeyini değerlendirmektir. Gereç ve Yöntemler Bu kesitsel karşılaştırmalı çalışmada, literatüre dayalı olarak farklı klinik durumları temsil edecek şekilde oluşturulan toplam 500 standartlaştırılmış implant vaka senaryosu analiz edilmiştir. Her bir senaryo; yaş, mevcut diş ve implant sayısı, sondalama derinliği, sondalamada kanama, kemik kaybı, periodontitis öyküsü ve destekleyici periodontal tedaviye uyum gibi klinik parametreleri içermektedir. Tüm vakalar, standart bir veri giriş formatı kullanılarak IDRA, ChatGPT ve DeepSeek platformlarında değerlendirmeye tabi tutulmuştur. Elde edilen risk sınıflandırmaları düşük, orta ve yüksek olarak kaydedilmiştir. Sistemler arasındaki uyum, lineer ağırlıklı Cohen’s kappa (κw) istatistiği ile değerlendirilmiştir. Bulgular IDRA, vakaların %95.8’ini yüksek risk olarak sınıflandırırken, ChatGPT ve DeepSeek daha heterojen bir dağılım sergilemiştir (sırasıyla %72.4 ve %88.6 yüksek risk). ChatGPT ile DeepSeek arasındaki uyum orta düzeyde bulunmuştur (κw = 0.442). IDRA ile DeepSeek arasında daha yüksek düzeyde uyum saptanırken (κw = 0.532), en düşük uyum IDRA ile ChatGPT arasında gözlenmiştir (κw = 0.258). Sonuç Yapay zeka tabanlı dil modelleri ile yapılandırılmış peri-implant risk değerlendirme sistemleri arasında değişken düzeylerde uyum bulunmaktadır. Sınıflandırma eğilimlerindeki farklılıklar ve kategori dağılımlarındaki dengesizlikler, uyum sonuçlarını etkileyebilmektedir. Bu bulgular, sınıflandırma dağılımındaki farklılıkları yansıtmakta olup, tanısal doğruluk veya klinik geçerliliği göstermemektedir. Klinik uygulamaya yönelik çıkarımlar yapılmadan önce gerçek hasta verileri ile ileri doğrulama çalışmalarına ihtiyaç vardır.

Anahtar Kelimeler

• yapay zeka
• Chat-GPT
• DeepSeek
• risk değerlendirmesi
• dental implantlar

Comparative Agreement of the Online Implant Disease Risk Assessment (IDRA) Tool, ChatGPT and DeepSeek in Peri-implant Disease Risk Classification: A 500-Case Scenario Study

Abstract

Objectives The aim of this study was to evaluate the level of agreement between an online Implant Disease Risk Assessment (IDRA) tool and large language models in classifying patients into low, moderate, and high peri-implant disease risk categories. Material and Methods A total of 500 standardized implant case scenarios were generated based on established different clinical scenarios. Each scenario included clinical variables such as age, number of teeth and implants, probing depth, bleeding on probing, bone loss, history of periodontitis, and maintenance compliance. All cases were entered into IDRA (perio-tools.com), ChatGPT, and DeepSeek using an identical input format. Risk classifications were recorded as low, moderate, or high. Agreement between systems was assessed using linearly weighted Cohen’s kappa statistics. Results IDRA classified 95.8% of cases as high risk, whereas ChatGPT and DeepSeek showed more heterogeneous distributions (72.4% and 88.6% high risk, respectively). Agreement between ChatGPT and DeepSeek was moderate (κw = 0.442), while higher agreement was observed between IDRA and DeepSeek (κw = 0.532). The lowest agreement was found between IDRA and ChatGPT (κw = 0.258). Conclusion Large language models show variable agreement with structured peri-implant risk assessment systems. Differences in classification tendencies and class distribution may influence agreement outcomes. These findings reflect differences in classification pattern rather than diagnostic accuracy or clinical validity. Further validation using clinical data is required before any potential clinical application.

Keywords

• Chat-GPT
• artificial intelligence
• DeepSeek
• risk assessment
• dental implants

References

1. 1. Temsah O, Khan SA, Chaiah Y, et al. Overview of early ChatGPT's presence in medical literature: Insights from a hybrid literature review by ChatGPT and human experts. Cureus. 2023; 15: E37281. doi: 10.7759/cureus.37281
2. 2. Israni ST, Verghese A. Humanizing artificial intelligence. JAMA. 2019; 321: 29-30. doi: 10.1001/jama.2018.19398
3. 3. Berglundh T, Armitage G, Araujo MG, et al. Peri-implant diseases and conditions: consensus report of workgroup 4 of the 2017 World Workshop on the classification of periodontal and peri-implant diseases and conditions. J Periodontol. 2018; 89: S313-8. doi: 10.1002/JPER.17-0739
4. 4. Schwarz F, Derks J, Monje A, et al. Peri-implantitis. J Clin Periodontol. 2018; 45: S246-66. doi: 10.1111/jcpe.12954
5. 5. Heitz-Mayfield LJ, Heitz F, Lang NP. Implant disease risk assessment: a systematic approach to evaluate the risk for peri-implant diseases. Clin Oral Implants Res. 2020; 31: 397-403. doi: 10.1111/clr.13585
6. 6. Lang NP, Tonetti MS. Periodontal risk assessment (PRA) for patients in supportive periodontal therapy (SPT). Oral Health Prev Dent. 2003; 1: 7-16.
7. 7. Ramseier CA. Diagnostic measures for monitoring and follow-up in periodontology and implant dentistry. Periodontol 2000. 2024; 95: 129-55. doi: 10.1111/prd.12588
8. 8. Schwendicke F, Samek W, Krois J. Artificial intelligence in dentistry: chances and challenges. J Dent Res. 2020; 99: 769-74. doi: 10.1177/0022034520915714

9. 9. Hung K, Montalvao C, Tanaka R, et al. The use and performance of artificial intelligence applications in dental and maxillofacial radiology: a systematic review. Dentomaxillofac Radiol. 2020; 49: 20190107. doi:10.1259/dmfr.20190107
10. 10. Kaplan TT, Akyol GE. The role of large language models in dental diagnosis, decision-making, and communication: a systematic review. Jpn Dent Sci Rev. 2026; 62: 57-67. doi:10.1016/j.jdsr.2026.01.002
11. 11. Wu Z, Li J, Wang X, et al. Application of artificial intelligence in dental implant prognosis: a scoping review. J Dent. 2024; 144: 104924. doi: 10.1016/j.jdent.2024.104924
12. 12. Khan S, Siddique A, Khan A, et al. Artificial in telligence in periodontology and implantology-anarrative review. J Med Artif Intell. 2024; 7. doi:10.21037/jmai-23-186
13. 13. Kaygisiz ÖF, Teke MT. Can DeepSeek and ChatGPT be used in the diagnosis of oral pathologies? BMC Oral Health. 2025; 25: 638. doi: 10.1186/s12903-025-06034-x
14. 14. Hassanein FE, El Barbary A, Hussein RR, et al. Diagnostic performance of ChatGPT-4o and DeepSeek-V3 in differential diagnosis of complex oral lesions: A multimodal imaging and case difficulty analysis. Oral Dis. 2025; 31: 3361-71. doi: 10.1111/odi.70007
15. 15. He R, Sarwal V, Qiu X, et al. Generative AI models in time-varying biomedical data: Scoping review. JMIR Med Inform. 2025; 27: E59792. doi: 10.2196/59792
16. 16. Huang H, Zheng O, Wang D, et al. ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model. Int J Oral Sci. 2023; 15: 29. doi: 10.1038/s41368-023-00239-y
17. 17. Wu X, Li J, Wang X, et al. A multi-dimensional performance evaluation of large language models in dental implantology: comparison of ChatGPT, DeepSeek, Grok, Gemini and Qwen across diverse clinical scenarios. BMC Oral Health. 2025; 25: 1272. doi: 10.1186/s12903-025-06619-6
18. 18. Özcivelek T, Özcan B. Comparative evaluation of responses from DeepSeek-R1, ChatGPT-o1, ChatGPT-4, and dental GPT chatbots to patient inquiries about dental and maxillofacial prostheses. BMC Oral Health. 2025; 25: 871. doi: 10.1186/s12903-025-06267-w
19. 19. Goodman RS, Patrinely JR, Stone CA, et al. Accuracy and reliability of Chatbot responses to physician questions. JAMA Netw Open. 2023; 6: E2336483. doi: 10.1001/jamanetworkopen.2023.36483
20. 20. Pan Q, Huang L, Liu N, et al. Evaluation of the accuracy of large language models in answering bone cancer-related questions. Front Public Health. 2025; 13: 1719047. doi:10.3389/fpubh.2025.1719047
21. 21. Eggmann F, Weiger R, Zitzmann NU, et al. Implications of large language models such as ChatGPT for dental medicine. J Esthet Restor Dent. 2023; 35: 1098-102. doi: 10.1111/jerd.13046
22. 22. Sim J, Wright CC. The Kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005; 85: 257-68.
23. 23. Hancı V, Ergün B, Gül Ş, et al. Assessment of readability, reliability, and quality of ChatGPT®, BARD®, Gemini®, Copilot®, Perplexity® responses on palliative care. Medicine (Baltimore). 2024; 103: E39305. doi: 10.1097/MD.0000000000039305
24. 24. Metin U, Goymen M. Information from digital and humansources: a comparison of chatbot and clinician responses to orthodontic questions. Am J Orthod Dentofacial Orthop. 2025; 168: 348-57. doi: 10.1016/j.ajodo.2025.04.008
25. 25. Chan L, Xu X, Lv K. DeepSeek-R1 and GPT-4 are comparable in a complex diagnostic challenge: a historical control study. Int J Surg. 2025; 111: 4056-9. doi: 10.1097/JS9.0000000000002386
26. 26. Zhou M, Pan Y, Zhang Y, et al. Evaluating AI-generated patient education materials for spinal surgeries: Comparative analysis of readability and DISCERN quality across ChatGPT and deepseek models. Int J Med Inform. 2025; 198: 105871. doi: 10.1016/j.ijmedinf.2025.105871
27. 27. Beheshti M, Toubal IE, Alaboud K, et al. Evaluating the reliability of ChatGPT for health-related questions: a systematic review. Informatics. 2025; 12: 9. doi:10.3390/informatics12010009