Patient Education in Hand Surgery: A Blinded Comparison of GPT-5.5 and Claude Opus 4.8

Öz

Patients increasingly consult conversational artificial intelligence about hand and wrist conditions, but the performance of proprietary large language models remains unclear. We compared the reliability, overall quality, and readability of patient-facing texts generated by GPT-5.5 and Claude Opus 4.8. Thirty hand surgery topics were selected from orthopedic education resources and the authors' outpatient practice. Each model received the same prompt and generated one response per topic (60 texts). Two fellowship-trained, certified hand surgeons independently evaluated source-masked outputs. DISCERN assessed reliability and treatment information, the Global Quality Score (GQS) overall quality, and the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) readability. Platforms were compared with the Mann-Whitney U test and interrater agreement with the intraclass correlation coefficient (ICC). Median DISCERN scores were 66.5 for GPT-5.5 and 63.5 for Claude Opus 4.8 (p = 0.143); median GQS was 4.0 for both (p = 0.389). Claude Opus 4.8 had a lower median FKGL (8.75 vs 10.4; p < 0.001) and higher median FRE (50.35 vs 43.6; p = 0.009). ICCs (95% CIs) were 0.755 (0.589-0.853) for DISCERN and 0.676 (0.458-0.807) for GQS. Expert-rated reliability and overall quality did not differ significantly, whereas Claude Opus 4.8 produced more favorable formula-based readability scores; median FKGL for both models nevertheless remained above the recommended sixth-grade level. Because FKGL and FRE quantify surface linguistic complexity rather than understanding, these findings describe written outputs only; they do not establish better patient comprehension, clinical effectiveness, factual equivalence, or safety.

Anahtar Kelimeler

• Artificial Intelligence
• Large Language Models
• Patient Education as Topic
• Health Literacy
• Hand Surgery

Destekleyen Kurum

The authors declared that this study received no financial support.

Proje Numarası

Yok

Etik Beyan

This study did not involve human participants, patient data, identifiable personal information, biological materials, animals, or any diagnostic or therapeutic intervention. Therefore, ethics committee approval and informed consent were not required. The study was conducted in accordance with research and publication ethics.

Teşekkür

None.

El Cerrahisinde Hasta Eğitimi: GPT-5.5 ile Claude Opus 4.8'in Kör Karşılaştırması

Öz

Hastalar el ve el bileği hastalıkları hakkında bilgi edinmek için giderek daha fazla konuşmaya dayalı yapay zekâ araçlarına başvurmaktadır. Bununla birlikte, mevcut büyük dil modellerinin performansı yeterince tanımlanmamıştır. Bu çalışmada GPT-5.5 ve Claude Opus 4.8 tarafından üretilen hasta eğitimi metinlerinin güvenilirliği, genel kalitesi ve okunabilirliği karşılaştırıldı. Ortopedik eğitim kaynakları ve yazarların poliklinik pratiğine dayanarak 30 el cerrahisi konusu seçildi. Her model aynı istemle konu başına bir yanıt üretti (toplam 60 metin). Yan dal eğitimi almış iki el cerrahı, kaynak kimliği maskelenmiş metinleri bağımsız değerlendirdi. Güvenilirlik ve tedavi bilgisi DISCERN, genel kalite Global Kalite Skoru (GQS), okunabilirlik Flesch-Kincaid Sınıf Düzeyi (FKGL) ve Flesch Okuma Kolaylığı (FRE) ile değerlendirildi. Platformlar Mann-Whitney U testiyle, değerlendiriciler arası uyum sınıf içi korelasyon katsayısı (ICC) ile analiz edildi. Ortanca DISCERN puanı GPT-5.5 için 66,5, Claude Opus 4.8 için 63,5 idi (p = 0,143); ortanca GQS her iki modelde 4,0 bulundu (p = 0,389). Claude Opus 4.8 daha düşük FKGL (8,75'e karşı 10,4; p < 0,001) ve daha yüksek FRE (50,35'e karşı 43,6; p = 0,009) sağladı. ICC nokta tahminleri (%95 GA) DISCERN için 0,755 (0,589-0,853), GQS için 0,676 (0,458-0,807) idi. Uzmanlarca puanlanan güvenilirlik ve genel kalite bakımından anlamlı fark saptanmazken Claude Opus 4.8 daha olumlu formüle dayalı okunabilirlik göstergeleri ortaya koydu; her iki modelde de ortanca FKGL, önerilen altıncı sınıf okuma düzeyinin üzerinde bulundu. FKGL ve FRE yüzeysel dilsel karmaşıklığı ölçtüğünden bu bulgular daha iyi hasta anlayışının, klinik etkinliğin, olgusal eşdeğerliğin veya güvenliğin kanıtı olarak yorumlanmamalıdır.

Anahtar Kelimeler

• Yapay Zekâ
• Büyük Dil Modelleri
• Hasta Eğitimi
• Sağlık Okuryazarlığı
• El Cerrahisi

Destekleyen Kurum

Yazarlar bu çalışma için herhangi bir finansal destek almadıklarını beyan etmişlerdir.

Proje Numarası

Yok

Etik Beyan

Bu çalışma; insan katılımcı, hasta verisi, kimliği belirlenebilir kişisel bilgi, biyolojik materyal, hayvan veya herhangi bir tanısal ya da terapötik müdahale içermemektedir. Bu nedenle etik kurul onayı ve bilgilendirilmiş onam gerekmemiştir. Çalışmada araştırma ve yayın etiği ilkelerine uyulmuştur.

Teşekkür

Bulunmamaktadır.

Kaynakça

1. 1. Ozkan S, Mellema JJ, Nazzal A, Lee SG, Ring D. Online health information seeking in hand and upper extremity surgery. J Hand Surg Am 2016;41(12):e469-e475.
2. 2. Daraz L, Morrow AS, Ponce OJ, et al. Readability of online health information: a meta-narrative systematic review. Am J Med Qual 2018;33(5):487-492.
3. 3. Aydin S, Karabacak M, Vlachos V, Margetis K. Large language models in patient education: a scoping review of applications in medicine. Front Med (Lausanne) 2024;11:1477898.
4. 4. Jagiella-Lodise O, Suh N, Zelenski NA. Can patients rely on ChatGPT to answer hand pathology-related medical questions? Hand (N Y) 2025;20(5):801-809.
5. 5. Croen BJ, Abdullah MS, Berns E, et al. Evaluation of patient education materials from large-language artificial intelligence models on carpal tunnel release. Hand (N Y) 2025;20(6):893-899.
6. 6. Parmar RP, Daulat SR, Shah R, Brady TT, Montague M, Roth C. Readability, accuracy, and lexical diversity of new ChatGPT models for common carpal tunnel syndrome questions. Hand (N Y). Published online February 8, 2026.
7. 7. Pohl NB, Derector E, Rivlin M, et al. A quality and readability comparison of artificial intelligence and popular health website education materials for common hand surgery procedures. Hand Surg Rehabil 2024;43(3):101723.
8. 8. Gorgos P, Ternell KH, Hammarstrand C, et al. ChatGPT and Claude in hand surgery: an explanatory evaluation of clinical decision support on common surgical cases. Hand Surg Rehabil 2025;44(6):102530.

9. 9. Bayrak HC, Karagöz B, Bayrak Ö. Comparative evaluation of large language model-based chatbots in a septic arthritis scenario: ChatGPT, Claude, and Perplexity. Acta Orthop Traumatol Turc 2025;59(6):415-420.
10. 10. American Academy of Orthopaedic Surgeons. OrthoInfo: hand and wrist conditions. https://orthoinfo.aaos.org/diseases--conditions/?bodyPart=HandWrist. Accessed June 1, 2026.
11. 11. Orthobullets. Hand specialty dashboard. https://www.orthobullets.com/topic/dashboard?id=6&specialty=6. Accessed June 1, 2026.
12. 12. OpenAI. Introducing GPT-5.5. https://openai.com/index/introducing-gpt-5-5/. Accessed July 15, 2026.
13. 13. Anthropic. Introducing Claude Opus 4.8. https://www.anthropic.com/news/claude-opus-4-8. Accessed July 15, 2026.
14. 14. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53(2):105-111.
15. 15. Bernard A, Langille M, Hughes S, Rose C, Leddin D, Veldhuyzen van Zanten S. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol 2007;102(9):2070-2077.
16. 16. Kincaid JP, Fishburne RP Jr, Rogers RL, Chissom BS. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report 8-75. Memphis, TN: Chief of Naval Technical Training, Naval Air Station Memphis; 1975.
17. 17. Flesch R. A new readability yardstick. J Appl Psychol 1948;32(3):221-233.
18. 18. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15(2):155-163.
19. 19. Gezer MC, Armangil M. Assessing the quality of ChatGPT's responses to commonly asked questions about trigger finger treatment. Ulus Travma Acil Cerrahi Derg 2025;31(4):389-393.
20. 20. Casey JC, Wang L, Milner JD, et al. ChatGPT is a concise, comprehensive, and accurate alternative to Google for answering common hand therapy questions. Hand Ther. Published online April 21, 2026.
21. 21. Eltorai AEM, Sharma P, Wang J, Daniels AH. Most American Academy of Orthopaedic Surgeons' online patient education material exceeds average patient reading level. Clin Orthop Relat Res 2015;473(4):1181-1186.
22. 22. Ghanem D, Covarrubias O, Maxson R, Sabharwal S, Shafiq B. Readability of trauma-related patient education materials from the American Academy of Orthopaedic Surgeons and Orthopaedic Trauma Association websites. J Am Acad Orthop Surg 2024;32(13):e642-e650.
23. 23. Yuncu M, Kose O. Comparative evaluation of large language models for patient education after total knee arthroplasty. Knee 2026;61:104445.
24. 24. Travis LM, Prasad S, Deiparine S, Marmor WA, Rizzo MG. Evaluation of ChatGPT-4o as a patient information tool for common orthopaedic surgeries: accuracy, completeness, and clinical utility. JAAOS Glob Res Rev 2025;9(12):e25.00341.
25. 25. Fiedler B, Ghilzai U, Ghali A, et al. A supplement, not a substitute: accuracy and completeness of ChatGPT responses for common elbow pathology. Shoulder Elbow 2026;18(2):379-385.
26. 26. Olszewski R, Brzezinski J, Watros K, Rysz J. Quantifying readability in chatbot-generated medical texts using classical linguistic indices: a review. Appl Sci 2026;16(3):1423.
27. 27. Will J, Gupta M, Zaretsky J, Dowlath A, Testa P, Feldman J. Enhancing the readability of online patient education materials using large language models: cross-sectional study. J Med Internet Res 2025;27:e69955.
28. 28. Pohl NB, Tarawneh OH, Johnson E, Aita D, Tadley M, Fletcher DJ. Patient preferences for carpal tunnel release education: a comparison of education materials from popular healthcare websites and ChatGPT. Hand Surg Rehabil 2025;44(1):102073.