Diyabetik Ayak Ülserlerinde Yapay Zeka Tarafından Üretilen Hekim Düzeyindeki Eğitsel İçeriklerin Karşılaştırmalı Değerlendirilmesi: Kalite, Kılavuz Uyumu ve Klinik Güvenilirlik

Öz

Çalışmanın amacı, diyabetik ayak ülseri yönetimine ilişkin yapay zeka tarafından üretilen hekim düzeyindeki eğitsel yanıtların kalite, klinik güvenilirlik ve kılavuz uyumunu değerlendirmektir. Çift kör, karşılaştırmalı analiz, üç yaygın kullanılan büyük dil modeli—ChatGPT, Gemini ve Claude—kullanılarak gerçekleştirilmiş ve bu modeller önleme, tanı ve sınıflandırma ile tedaviyi kapsayan on iki standart klinik soru üzerinden değerlendirilmiştir. Tüm sorular, Uluslararası Diyabetik Ayak Çalışma Grubu (IWGDF) tarafından yayımlanan mevcut bir klinik kılavuz temel alınarak geliştirilmiştir. Yanıtlar, klinik derinlik, teknik doğruluk, karar verme yararlılığı, genel kalite ve potansiyel olarak zararlı veya yanıltıcı bilgi varlığı dahil olmak üzere önceden tanımlanmış kriterlere göre altı uzman değerlendirici tarafından bağımsız olarak incelenmiştir. Uluslararası kabul gören kılavuz ile uyumu, yerleşik önerilere dayalı yapılandırılmış kontrol listeleri kullanılarak değerlendirilmiştir. Tüm değerlendirme alanlarında modeller arasında anlamlı farklılıklar saptanmıştır. Gemini, klinik derinlik, teknik doğruluk, karar verme yararlılığı ve genel kalite açısından daha yüksek performans göstermiş ve aynı zamanda kılavuz temelli önerilerle daha yüksek düzeyde uyum sergilemiştir. Buna karşılık ChatGPT ve Claude daha düşük skorlar elde etmiş ve yanıltıcı bilgi üretme eğilimlerinin daha yüksek olduğu gözlenmiştir. Değerlendiriciler arası uyum analizinde orta-iyi düzeyde tutarlılık saptanmıştır. Bu bulgular, yapay zeka sistemlerinin klinik açıdan anlamlı bilgi üretme potansiyeline sahip olduğunu göstermekle birlikte, performanslarının değişken olduğunu ve yüksek riskli klinik karar verme süreçlerinde her zaman gerekli standartları karşılamayabileceğini ortaya koymaktadır. Bu nedenle bu sistemlerin rutin klinik uygulamalara entegrasyonu öncesinde dikkatli değerlendirme ve doğrulama gereklidir.

Anahtar Kelimeler

• Diyabetik Ayak
• Yapay Zeka
• Klinik Karar Destek Sistemleri
• Kılavuzlar Konusu
• Sağlık Eğitimi

Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability

Öz

The study aimed to evaluate the quality, clinical reliability, and guideline concordance of physician-level educational responses generated by artificial intelligence (AI) in diabetic foot ulcer management. A blinded, comparative analysis was conducted using three widely used large language models—ChatGPT, Gemini, and Claude—which were assessed across twelve standardized clinical questions covering prevention, diagnosis and classification, and treatment. All questions were developed based on an established clinical guideline (International Working Group on the Diabetic Foot, IWGDF). Responses were independently evaluated by six expert raters using predefined criteria, including clinical depth, technical accuracy, decision-making utility, overall quality, and the presence of potentially harmful or misleading information. Guideline concordance was assessed using structured checklists derived from established recommendations. Significant differences were observed among the evaluated models across all domains. Gemini consistently demonstrated higher performance in clinical depth, technical accuracy, decision-making utility, and overall quality, while also showing greater alignment with guideline-based recommendations. ChatGPT and Claude demonstrated comparatively lower scores and a higher tendency to produce responses containing potentially misleading information. Inter-rater reliability analysis indicated moderate to good agreement among evaluators. These findings suggest that, although artificial intelligence systems show considerable potential in generating clinically relevant information, their performance remains variable and may not consistently meet the standards required for high-risk clinical decision-making. Careful evaluation and validation are therefore essential before integrating such systems into routine clinical practice.

Anahtar Kelimeler

• Diabetic Foot
• Artificial Intelligence
• Clinical Decision Support Systems
• Practice Guidelines as Topic
• Health Education

Kaynakça

1. 1. Wang L, Wan Z, Ni C, et al. Applications and concerns of ChatGPT and other conversational large language models in health care: systematic review. J Med Internet Res. 2024;26:e22769. doi:10.2196/22769
2. 2. Alowais SA, Alghamdi SS, Alsuhebany N, et al. Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Med Educ. 2023;23(1):689. doi:10.1186/s12909-023-04698-z
3. 3. Edmonds M, Manu C, Vas P. The current burden of diabetic foot disease. J Clin Orthop Trauma. 2021;17:88-93. doi:10.1016/j.jcot.2021.01.017
4. 4. Schaper NC, van Netten JJ, Apelqvist J, et al. Practical guidelines on the prevention and management of diabetes-related foot disease (IWGDF 2023 update). Diabetes Metab Res Rev. 2024;40(3):e3657. doi:10.1002/dmrr.3657
5. 5. Sauerbrei A, Kerasidou A, Lucivero F, Hallowell N. The impact of artificial intelligence on the person-centred, doctor-patient relationship: some problems and solutions. BMC Med Inform Decis Mak. 2023;23(1):73. doi:10.1186/s12911-023-02162-y
6. 6. Young AT, Amara D, Bhattacharya A, Wei ML. Patient and general public attitudes towards clinical artificial intelligence: a mixed methods systematic review. Lancet Digit Health. 2021;3(9):e599-e611. doi:10.1016/S2589-7500(21)00132-1
7. 7. American Diabetes Association Professional Practice Committee. 12. Retinopathy, Neuropathy, and Foot Care: Standards of Care in Diabetes—2026. Diabetes Care. 2026;49(suppl 1):S261-S276. doi:10.2337/dc26-S012
8. 8. Roberts SB, Löffler A, Ramachandram D, et al. Development, validation, and pilot implementation of a prognostic model to predict future foot complications among people with diabetes recently discharged from hospital. J Clin Epidemiol. 2026;28:112295. doi:10.1016/j.jclinepi.2026.112295

9. 9. Wu H, Deng J, Qiu X, et al. Comparative assessment of large language models in diabetic foot infection management: alignment with IWGDF/IDSA guidelines. Front Endocrinol (Lausanne). 2026;17:1667159. doi:10.3389/fendo.2026.1667159
10. 10. Tao Y, Hou J, Zhou G, Zhang D. Artificial intelligence applied to diabetes complications: a bibliometric analysis. Front Artif Intell. 2025;8:1455341. doi:10.3389/frai.2025.1455341
11. 11. Mondal A, Naskar A, Roy Choudhury B, et al. Evaluating the performance and safety of large language models in generating type 2 diabetes mellitus management plans: a comparative study with physicians using real patient records. Cureus. 2025;17(3):e80737. doi:10.7759/cureus.80737
12. 12. Sheng B, Pushpanathan K, Guan Z, et al. Artificial intelligence for diabetes care: current and future prospects. Lancet Diabetes Endocrinol. 2024;12(8):569-595. doi:10.1016/S2213-8587(24)00154-2
13. 13. Wang Z, Li X, Tao X, et al. Challenges of using generative artificial intelligence for diabetes patient education: a cross-platform analysis of the quality, readability, and actionability of text generated by large language models. Front Public Health. 2026;14:1804524. doi:10.3389/fpubh.2026.1804524
14. 14. Martini N, Dhaliwal NK, Alipour E, Scahill S, Sajtos L. Patient perceptions of artificial intelligence in diabetes self-management: cross-sectional survey study. JMIR Form Res. 2026;10:e83030. doi:10.2196/83030
15. 15. Taheri Z, Tajdari M, Ghadikolaei YH, et al. The transformative role of artificial intelligence in gestational diabetes mellitus: advancements in screening, management, and long-term outcome prediction. J Diabetes Metab Disord. 2026;25(1):120. doi:10.1007/s40200-025-01817-z
16. 16. Abuzar M, Rai S, Pathan T, Fakki F, Syed S. ChatGPT for diabetes education: potential, accuracy, and accessibility in patient support. Front Digit Health. 2026;8:1768843. doi:10.3389/fdgth.2026.1768843
17. 17. Johansen AR, Laursen IK, Jacobsen V, Møller ZH, Cichosz SL. Evaluating accuracy of ChatGPT-4o in automated carbohydrate estimation from images as a self-management tool for adolescents with type 1 diabetes. J Diabetes Sci Technol. 2026;20:19322968261424270. doi:10.1177/19322968261424270
18. 18. Kim JM, Kang SM, Jung JH, et al. Comprehensive management of diabetic foot ulcers: current evidence and practice. J Diabetes. 2026;18(4):e70221. doi:10.1111/1753-0407.70221
19. 19. Terzi I, Akinosoglou K. More detection, better decisions? Rethinking pathogen-driven therapy in diabetic foot infections. Int J Low Extrem Wounds. 2026;25:15347346261447707. doi:10.1177/15347346261447707
20. 20. Armstrong DG, Tan TW, Boulton AJM, Bus SA. Diabetic foot ulcers: a review. JAMA. 2023;330(1):62-75. doi:10.1001/jama.2023.10578
21. 21. Hingorani A, LaMuraglia GM, Henke P, et al. The management of diabetic foot: a clinical practice guideline by the Society for Vascular Surgery in collaboration with the American Podiatric Medical Association and the Society for Vascular Medicine. J Vasc Surg. 2016;63(2 suppl):3S-21S. doi:10.1016/j.jvs.2015.10.003