Klinik optik çoktan seçmeli sorularda yapay zekâ sohbet botları: ChatGPT-5, Copilot ve Gemini’nin Türkçe-İngilizce karşılaştırması

Abstract

Amaç: Klinik optik alanındaki çoktan seçmeli sorularda büyük dil modellerinin (BDM) doğruluğunu ve dil (Türkçe–İngilizce) etkisini değerlendirmek. Gereç ve Yöntem: AAO 2024–2025 BCSC Clinical Optics and Vision Rehabilitation kitabındaki dört seçenekli 88 soru seçildi; İngilizce özgün sorular sertifikalı çevirmen tarafından Türkçeye çevrildi. ChatGPT-5, Copilot ve Gemini’ye standart yönerge ile her soru ayrı oturumda yöneltildi. Yanıtlar resmi cevap anahtarıyla karşılaştırıldı. Birincil sonuç, modellerin her dildeki doğruluğu; ikincil sonuç, aynı soruların iki dildeki yanıt tutarlılığıydı. Karşılaştırmalarda Pearson ki-kare, eşleştirilmiş dil içi değerlendirmede McNemar testi kullanıldı (p < 0,05). Bulgular: İngilizce sorularda doğruluk oranları sırasıyla ChatGPT-5 %71,6 (63/88), Copilot %70,5 (62/88) ve Gemini %65,9 (58/88) olup modeller arasında fark yoktu (p = 0,354). Türkçe sürümlerde doğruluk oranları ChatGPT-5 %68,2 (60/88), Copilot %69,3 (61/88) ve Gemini %63,6 (56/88) olarak bulundu; aradaki fark istatistiksel olarak anlamlı değildi (p = 0,126). Dil içi tutarlılık yüksek seyretti: ChatGPT-5 aynı soruların %93,2’sinde (82/88), Copilot %95,5’inde (84/88) ve Gemini %92,1’inde (81/88) aynı seçeneği verdi; hiçbir modelde Türkçe–İngilizce doğruluk farkı anlamlı değildi (ChatGPT-5 p = 0,435; Copilot p = 0,674; Gemini p = 0,481). Sonuç: Üç BDM, klinik optik sorularında orta düzey doğruluk (%63–%72) ve iki dil arasında büyük ölçüde benzer performans sergilemiştir. Bu durum, klinik optik eğitiminde BDM’lerin dil açısından tutarlı bir yardımcı araç olabileceğini düşündürmekle birlikte, mutlak doğruluğun sınırlı oluşu modellerin tek başına otorite olarak kullanılmaması gerektiğine işaret etmektedir.

Keywords

• Büyük dil modelleri
• ChatGPT-5
• Copilot
• Gemini
• Klinik optik
• Oftalmoloji eğitimi
• Yapay zekâ sohbet botları

AI Chatbots on Clinical Optics Multiple-Choice Questions: A Turkish–English Comparison of ChatGPT-5, Copilot, and Gemini

Abstract

Objective: To evaluate the accuracy of large language models (LLMs) on multiple-choice questions (MCQs) in clinical optics and to assess the effect of language (Turkish vs English). Materials and Methods: We selected 88 four-option items from the AAO 2024–2025 BCSC Clinical Optics and Vision Rehabilitation book; the original English items were translated into Turkish by a certified translator. The same questions, in both languages, were posed to ChatGPT-5, Copilot, and Gemini using a standardized prompt, with a fresh session for each question. Model outputs were compared with the official answer key. The primary outcome was model accuracy in each language; the secondary outcome was within-question agreement across the two languages. Group comparisons used the Pearson chi-square test, and within-question language comparisons used the McNemar test (p < 0.05). Results: On the English set, accuracies were 71.6% (63/88) for ChatGPT-5, 70.5% (62/88) for Copilot, and 65.9% (58/88) for Gemini, with no significant difference among models (p = 0.354). On the Turkish set, accuracies were 68.2% (60/88), 69.3% (61/88), and 63.6% (56/88), respectively; differences were not significant (p = 0.126). Cross-language consistency was high: ChatGPT-5 gave the same option on 93.2% (82/88) of paired questions, Copilot on 95.5% (84/88), and Gemini on 92.1% (81/88). For none of the models was the English–Turkish accuracy difference significant (ChatGPT-5 p = 0.435; Copilot p = 0.674; Gemini p = 0.481). Conclusion: Across clinical optics MCQs, the three LLMs achieved moderate accuracy (63%–72%) and largely comparable performance in Turkish and English. While this cross-lingual stability supports their use as language-consistent adjuncts in clinical optics education, the limited absolute accuracy argues against relying on them as stand-alone authorities.

Keywords

• ChatGPT-5
• Clinical optics
• Conversational AI chatbots
• Copilot
• Gemini
• Large language models
• Ophthalmology education

References

1. 1. Lee P, Bubeck S, Petro J. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. N Engl J Med. 2023;388(13):1233-1239. doi:10.1056/NEJMsr2214184.
2. 2. Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. doi:10.1371/journal.pdig.0000198.
3. 3. Mihalache A, Popovic MM, Muni RH. Performance of an artificial intelligence chatbot in ophthalmic knowledge assessment. JAMA Ophthalmol. 2023;141(6):589-597. doi:10.1001/jamaophthalmol.2023.1144.
4. 4. Vrdoljak J, Boban Z, Vilović M, et al. A review of large language models in medical education, clinical decision support, and healthcare administration. Healthcare (Basel). 2025;13(6):603. doi:10.3390/healthcare13060603.
5. 5. Wu JH, Nishida T, Liu TYA. Accuracy of large language models in answering ophthalmology board-style questions: A meta-analysis. Asia Pac J Ophthalmol (Phila). 2024;13(5):100106. doi:10.1016/j.apjo.2024.100106.
6. 6. Antaki F, Touma S, Milad D, et al. Evaluating the performance of ChatGPT in ophthalmology: An analysis of its successes and shortcomings. Ophthalmol Sci. 2023;3(4):100324. doi:10.1016/j.xops.2023.100324.
7. 7. Alipour F, Samavati K, Abdi P, et al. An innovative optics lab design for residency training in ophthalmology. Sci Rep. 2023;13(1):21844. doi:10.1038/s41598-023-44688-1.
8. 8. Tu DC, Park DW, Krisciunas T, et al. Teaching ophthalmology residents clinical optics via a flipped classroom curriculum. J Acad Ophthalmol. 2019;11(1):e16-e21.

9. 9. American Academy of Ophthalmology. Clinical Optics and Vision Rehabilitation. Basic and Clinical Science Course, Section 3, 2024–2025. American Academy of Ophthalmology; 2024.
10. 10. Şensoy E, Şensoy M, Çıtırık M. Dil farklılıklarının ChatGPT-3.5, Copilot ve Gemini’nin pediatrik oftalmoloji ve şaşılık çoktan seçmeli sorulardaki etkinliğinin değerlendirilmesi. Anadolu Klin. 2025;30(2):262-267.
11. 11. Dhiman S, Mishra C, Kaur S, et al. Comparative evaluation of five AI chatbots in pediatric ophthalmology: A multidomain expert-based appraisal. AJO Int. 2025;100188.
12. 12. Cohen SA, Fisher AC, Xu BY, et al. Comparing the accuracy and readability of glaucoma-related question responses and educational materials by Google and ChatGPT. J Curr Glaucoma Pract. 2024;18(3):110-116. doi:10.5005/jp-journals-10078-1448.
13. 13. Balas M, Mandelcorn ED, Yan P, et al. ChatGPT and retinal disease: A cross-sectional study on AI comprehension of clinical guidelines. Can J Ophthalmol. 2025;60(1):e117-e123. doi:10.1016/j.jcjo.2024.06.001.
14. 14. Das D, Narayan A, Mishra V, et al. AI chatbots in answering questions related to ocular oncology: A comparative study between DeepSeek v3, ChatGPT-4o, and Gemini 2.0. Cureus. 2025;17(8):e90773. doi:10.7759/cureus.90773.
15. 15. Güler MS, Baydemir EE. Evaluation of ChatGPT-4 responses to glaucoma patients' questions: Can artificial intelligence become a trusted advisor between doctor and patient? Clin Exp Ophthalmol. 2024;52(9):1016-1019. doi:10.1111/ceo.14451.
16. 16. David D, Zloto O, Katz G, et al. The use of artificial intelligence-based chatbots in ophthalmology triage. Eye (Lond). 2025;39(4):785-789. doi:10.1038/s41433-024-03488-1.
17. 17. Carlà MM, Gambini G, Baldascino A, et al. Exploring AI-chatbots' capability to suggest surgical planning in ophthalmology: ChatGPT versus Google Gemini analysis of retinal detachment cases. Br J Ophthalmol. 2024;108(10):1457-1469. doi:10.1136/bjo-2023-325143.
18. 18. Sabaner MC, Yozgat Z. Performance of ChatGPT-4 Omni and Gemini 1.5 Pro on ophthalmology-related questions in the Turkish Medical Specialty Exam. Turk J Ophthalmol. 2025;55(4):177-185. doi:10.4274/tjo.galenos.2025.27895.
19. 19. Oztermeli AD, Oztermeli A. ChatGPT performance in the medical specialty exam: An observational study. Medicine (Baltimore). 2023;102(32):e34673. doi:10.1097/MD.0000000000034673.
20. 20. Yavrum F, Kocabas Ozkoyuncu D. Is ChatGPT a useful tool for ophthalmology practice? Acta Med Alanya. 2024;8(3):221-227. doi:10.30565/medalanya.1531790.
21. 21. Panthier C, Gatinel D. Success of ChatGPT, an AI language model, in taking the French language version of the European Board of Ophthalmology examination: A novel approach to medical knowledge assessment. J Fr Ophtalmol. 2023;46(7):706-711. doi:10.1016/j.jfo.2023.05.006.
22. 22. Alonso I, Oronoz M, Agerri R. MedExpQA: Multilingual benchmarking of large language models for medical question answering. Artif Intell Med. 2024;155:102938. doi:10.1016/j.artmed.2024.102938.