Araştırma Makalesi

Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology

Cilt: 36 Sayı: 2026 22 Ekim 2026
PDF İndir
EN TR

Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology

Öz

Aim: Global Large Language Models (LLMs) often struggle with cultural nuances. This pilot study investigates the feasibility of “Kumru,” a developing native Turkish LLM, in interpreting culturally specific emergency triage vignettes compared to standard medical terminology.
Materials and Methods: A prospective in-silico study was conducted using a pilot dataset of 50 clinical vignettes based on Turkish Ministry of Health (Annex-4) Guidelines. The dataset was stratified into standard medical terminology (n=31) and cultural idioms (n=19) to assess domain-specific linguistic processing.
Results: The model achieved 93.7% sensitivity for high-acuity cases but exhibited a specificity of 0% and an over-triage rate of 52%. While overall diagnostic accuracy was moderate (44.0%), the model demonstrated statistically superior performance in decoding cultural idioms (84.2%) compared to standard terminology (19.3%, p < .001).
Conclusions: Although the current native model demonstrates a high capability for understanding local idioms, its low specificity renders it operationally inefficient for autonomous clinical decision-making. These findings present a proof-of-concept for linguistic localization but highlight the necessity for extensive fine-tuning before clinical deployment.

Anahtar Kelimeler

Cultural Sensitivity, Data Sovereignty, Emergency Triage, Large Language Models, Native AI, Natural Language Processing.

Destekleyen Kurum

The authors received no specific funding for this work.

Etik Beyan

Not Applicable. This study is an in-silico evaluation using theoretical, author-generated clinical vignettes created based on public guidelines (Ministry of Health Annex-4). It involves no human subjects, no real patient records, and no animal data. Therefore, Institutional Review Board (IRB) approval is not required for this study design.

Teşekkür

This study was conducted using the public research preview of Kumru (v0.2.1). The authors acknowledge that the model is in a developmental stage and, in accordance with its Terms of Use, the outputs were evaluated solely for research purposes on simulated data and not applied to real clinical decision-making. Furthermore, the authors would like to express their sincere gratitude to the independent emergency medicine specialists for their valuable time and expert contribution in blindly validating the clinical vignettes and establishing the reference triage categories.

Kaynakça

  1. 1. Eidstø A, Ylä-Mattila J, Tuominen J, Huhtala H, Palomäki A, Koivistoinen T. Emergency department crowding increases 10-day mortality for non-critical patients: a retrospective observational study. Intern Emerg Med 2023;19(1):175.
  2. 2. Alshiakh S, Abdulwahab F, Baz A, et al. Factors and solutions addressing overcrowding in Emergency Department. J Healthc Sci. 2023;3(1):1.
  3. 3. Schulson L, Novack V, Smulowitz PB, Dechen T, Landon BE. Emergency department care for patients with limited English proficiency: a retrospective cohort study. J Gen Intern Med 2018;33(12):2113.
  4. 4. Portillo EN, Stack AM, Monuteaux MC, Curt A, Perron C, Lee LK. Association of limited English proficiency and increased pediatric emergency department revisits. Acad Emerg Med 2021;28(9):1001.
  5. 5. Chu JN, Wong J, Bardach NS, et al. Association between language discordance and unplanned hospital readmissions or emergency department revisits: a systematic review and meta-analysis. BMJ Qual Saf 2023;33(7):456.
  6. 6. Masanneck L, Schmidt L, Seifert A, et al. Triage performance across Large Language Models, ChatGPT, and untrained doctors in emergency medicine: comparative study. J Med Internet Res 2024;26:e53297.
  7. 7. Williams CYK, Zack T, Miao BY, et al. Use of a Large Language Model to assess clinical acuity of adults in the Emergency Department. JAMA Netw Open 2024;7(5):e248895.
  8. 8. Williams CYK, Miao BY, Kornblith AE, Butte AJ. Evaluating the use of large language models to provide clinical recommendations in the Emergency Department. Nat Commun 2024;15(1):1690.
  9. 9. Esmaeilzadeh P. Ethical implications of using general-purpose LLMs in clinical settings: a comparative analysis of prompt engineering strategies and their impact on patient safety. BMC Med Inform Decis Mak 2025;25(1):1.
  10. 10. Mwaniki P, Musau W, Isaaka L, et al. Benchmarking Large Language Models and clinicians using locally generated primary healthcare vignettes in Kenya. bioRxiv. Preprint. Posted October 25, 2025.

Kaynak Göster

APA
İnan, M., & Aypak, C. (2026). Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology. Genel Tıp Dergisi, 36(2026), 1-6. https://doi.org/10.54005/geneltip.1861837
AMA
1.İnan M, Aypak C. Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology. Genel Tıp Derg. 2026;36(2026):1-6. doi:10.54005/geneltip.1861837
Chicago
İnan, Muhammed, ve Cenk Aypak. 2026. “Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology”. Genel Tıp Dergisi 36 (2026): 1-6. https://doi.org/10.54005/geneltip.1861837.
EndNote
İnan M, Aypak C (01 Ekim 2026) Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology. Genel Tıp Dergisi 36 2026 1–6.
IEEE
[1]M. İnan ve C. Aypak, “Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology”, Genel Tıp Derg, c. 36, sy 2026, ss. 1–6, Eki. 2026, doi: 10.54005/geneltip.1861837.
ISNAD
İnan, Muhammed - Aypak, Cenk. “Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology”. Genel Tıp Dergisi 36/2026 (01 Ekim 2026): 1-6. https://doi.org/10.54005/geneltip.1861837.
JAMA
1.İnan M, Aypak C. Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology. Genel Tıp Derg. 2026;36:1–6.
MLA
İnan, Muhammed, ve Cenk Aypak. “Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology”. Genel Tıp Dergisi, c. 36, sy 2026, Ekim 2026, ss. 1-6, doi:10.54005/geneltip.1861837.
Vancouver
1.Muhammed İnan, Cenk Aypak. Preliminary evaluation of a native Turkish large language model in emergency triage: a pilot study on cultural vs. standard terminology. Genel Tıp Derg. 01 Ekim 2026;36(2026):1-6. doi:10.54005/geneltip.1861837