Research Article

Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability

Volume: 52 September 8, 2026
TR EN

Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability

Abstract

The study aimed to evaluate the quality, clinical reliability, and guideline concordance of physician-level educational responses generated by artificial intelligence (AI) in diabetic foot ulcer management. A blinded, comparative analysis was conducted using three widely used large language models—ChatGPT, Gemini, and Claude—which were assessed across twelve standardized clinical questions covering prevention, diagnosis and classification, and treatment. All questions were developed based on an established clinical guideline (International Working Group on the Diabetic Foot, IWGDF). Responses were independently evaluated by six expert raters using predefined criteria, including clinical depth, technical accuracy, decision-making utility, overall quality, and the presence of potentially harmful or misleading information. Guideline concordance was assessed using structured checklists derived from established recommendations. Significant differences were observed among the evaluated models across all domains. Gemini consistently demonstrated higher performance in clinical depth, technical accuracy, decision-making utility, and overall quality, while also showing greater alignment with guideline-based recommendations. ChatGPT and Claude demonstrated comparatively lower scores and a higher tendency to produce responses containing potentially misleading information. Inter-rater reliability analysis indicated moderate to good agreement among evaluators. These findings suggest that, although artificial intelligence systems show considerable potential in generating clinically relevant information, their performance remains variable and may not consistently meet the standards required for high-risk clinical decision-making. Careful evaluation and validation are therefore essential before integrating such systems into routine clinical practice.

Keywords

References

  1. 1. Wang L, Wan Z, Ni C, et al. Applications and concerns of ChatGPT and other conversational large language models in health care: systematic review. J Med Internet Res. 2024;26:e22769. doi:10.2196/22769
  2. 2. Alowais SA, Alghamdi SS, Alsuhebany N, et al. Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Med Educ. 2023;23(1):689. doi:10.1186/s12909-023-04698-z
  3. 3. Edmonds M, Manu C, Vas P. The current burden of diabetic foot disease. J Clin Orthop Trauma. 2021;17:88-93. doi:10.1016/j.jcot.2021.01.017
  4. 4. Schaper NC, van Netten JJ, Apelqvist J, et al. Practical guidelines on the prevention and management of diabetes-related foot disease (IWGDF 2023 update). Diabetes Metab Res Rev. 2024;40(3):e3657. doi:10.1002/dmrr.3657
  5. 5. Sauerbrei A, Kerasidou A, Lucivero F, Hallowell N. The impact of artificial intelligence on the person-centred, doctor-patient relationship: some problems and solutions. BMC Med Inform Decis Mak. 2023;23(1):73. doi:10.1186/s12911-023-02162-y
  6. 6. Young AT, Amara D, Bhattacharya A, Wei ML. Patient and general public attitudes towards clinical artificial intelligence: a mixed methods systematic review. Lancet Digit Health. 2021;3(9):e599-e611. doi:10.1016/S2589-7500(21)00132-1
  7. 7. American Diabetes Association Professional Practice Committee. 12. Retinopathy, Neuropathy, and Foot Care: Standards of Care in Diabetes—2026. Diabetes Care. 2026;49(suppl 1):S261-S276. doi:10.2337/dc26-S012
  8. 8. Roberts SB, Löffler A, Ramachandram D, et al. Development, validation, and pilot implementation of a prognostic model to predict future foot complications among people with diabetes recently discharged from hospital. J Clin Epidemiol. 2026;28:112295. doi:10.1016/j.jclinepi.2026.112295

Details

Primary Language

English

Subjects

Surgery (Other), Endocrinology

Journal Section

Research Article

Publication Date

September 8, 2026

Submission Date

May 11, 2026

Acceptance Date

August 6, 2026

Published in Issue

Year 2026 Volume: 52

APA
Işıktekin, E., & Hocaoğlu, E. (2026). Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability. Journal of Uludağ University Medical Faculty, 52. https://doi.org/10.32708/uutfd.1949030
AMA
1.Işıktekin E, Hocaoğlu E. Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability. J Uludag Univ Med Fac. 2026;52. doi:10.32708/uutfd.1949030
Chicago
Işıktekin, Emrah, and Erhan Hocaoğlu. 2026. “Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability”. Journal of Uludağ University Medical Faculty 52 (September). https://doi.org/10.32708/uutfd.1949030.
EndNote
Işıktekin E, Hocaoğlu E (September 1, 2026) Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability. Journal of Uludağ University Medical Faculty 52
IEEE
[1]E. Işıktekin and E. Hocaoğlu, “Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability”, J Uludag Univ Med Fac, vol. 52, Sept. 2026, doi: 10.32708/uutfd.1949030.
ISNAD
Işıktekin, Emrah - Hocaoğlu, Erhan. “Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability”. Journal of Uludağ University Medical Faculty 52 (September 1, 2026). https://doi.org/10.32708/uutfd.1949030.
JAMA
1.Işıktekin E, Hocaoğlu E. Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability. J Uludag Univ Med Fac. 2026;52. doi:10.32708/uutfd.1949030.
MLA
Işıktekin, Emrah, and Erhan Hocaoğlu. “Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability”. Journal of Uludağ University Medical Faculty, vol. 52, Sept. 2026, doi:10.32708/uutfd.1949030.
Vancouver
1.Emrah Işıktekin, Erhan Hocaoğlu. Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability. J Uludag Univ Med Fac. 2026 Sep. 1;52. doi:10.32708/uutfd.1949030

ISSN: 1300-414X, e-ISSN: 2645-9027

Creative Commons License
Journal of Uludag University Medical Faculty is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
2023