The reliability of an artificial intelligence tool, ‘Decision Trees’, in emergency medicine triage

Abstract

Abstract Purpose: Emergency department (ED) overcrowding is a significant challenge worldwide. Triage systems categorize patients according to the urgency of their clinical conditions to ensure patient safety and minimize the risk of clinical errors. The aim of this study was to evaluate the utility of decision trees, an artificial intelligence-based model, in emergency medicine triage. Materials and methods: This prospective, cross-sectional study was conducted in the ED of a tertiary care hospital. A total of 1,999 patients aged >17 years were included. Prior to the study, triage staff received training on the Australasian Triage Scale (ATS). Two independent observers assessed the final triage category of each patient. A new triage algorithm was developed using a decision tree model. Results: Mean age of the study population was 41.1±17.2 years, and 49.1% (n=981) were male. A total of 867 patients (43.3%) were assigned to triage category 5, whereas 14 patients (0.7%) were assigned to category 1. The most common clinical descriptors were minimal pain with no high-risk features (n=409, 20.5%) and minor symptoms of low-risk conditions (n=362, 18.1%). There was excellent agreement between the independent observers (κ=0.997). The decision tree algorithm incorrectly classified only one patient, yielding an accuracy rate of 99.9%. Agreement between the ATS and the decision tree algorithm was also excellent (κ=0.999). Agreement between physicians and paramedics was moderate (κ=0.541). Conclusion: Decision tree models may be used to develop practical triage algorithms as decision-support tools in ED triage. Artificial intelligence-based decision trees have the potential to support triage practices.

Keywords

• artificial intelligence
• decision trees
• emergency department
• triage

Supporting Institution

None

Ethical Statement

Ethical approval for the study was obtained from the Clinical Research Ethics Committee of the Faculty of Medicine, Akdeniz University (Approval Date-No: 30/05/2011-193). This study was conducted in accordance with the ethical principles outlined in the Declaration of Helsinki

Thanks

None

Acil tıp triyajında yapay zeka aracı olarak 'Karar Ağaçları'nın güvenilirliği

Abstract

Öz Amaç: Acil servis (AS) kalabalıklığı dünya çapında önemli bir sağlık hizmeti sorunudur. Triyaj sistemleri, hasta güvenliğini sağlamak ve klinik hata riskini en aza indirmek amacıyla hastaları klinik durumlarının aciliyetine göre sınıflandırmaktadır. Bu çalışmanın amacı, yapay zeka tabanlı bir model olan Karar Ağaçlarının acil tıp triyajındaki kullanımını değerlendirmektir. Gereç ve yöntem: Bu prospektif, kesitsel çalışma, üçüncü basamak bir hastanenin acil servisinde gerçekleştirilmiştir. Çalışmaya 17 yaşından büyük toplam 1.999 hasta dahil edilmiştir. Çalışma öncesinde triyaj personeline, Avustralasya Triyaj Ölçeği (ATS) konusunda eğitim verilmiştir. İki bağımsız gözlemci, her hastanın nihai triyaj kategorisini değerlendirmiştir. Karar Ağacı modeli kullanılarrk yeni bir triyaj algoritması oluşturulmuştur. Bulgular: Çalışma popülasyonunun yaş ortalaması 41,1±17,2 yıl olup, hastaların %49,1'i (n=981) erkekti. Toplam 867 hasta (%43,3) triyaj kategorisi 5’e, 14 hasta (%0,7) ise kategori 1’e atanmıştır. En sık karşılaşılan klinik tanımlayıcılar, yüksek riskli özellikler bulunmaksızın minimal ağrı (n=409, %20,5) ve düşük riskli durumlara ait minör semptomlar (n=362, %18,1) olmuştur. İki bağımsız gözlemci arasındaki uyum mükemmel düzeyde bulunmuştur (κ=0,997). 'Karar Ağacı' algoritması yalnızca bir hastayı yanlış sınıflandırmış ve %99,9 doğruluk oranına ulaşmıştır. ATS ile 'Karar Ağacı' algoristması arasındaki uyum da mükemmel düzeyde bulunmuştur (κ=0,999). Buna karşılık, hekimler ve paramedikler arasındaki uyum orta düzeydeydi (κ=0,541). Sonuç: Yapay zeka modeli olarak “Karar Ağaçları”, acil servis triyajında karar destek aracı olarak kullanılabilecek pratik triyaj algoritmalarının geliştirilmesinde kullanılabilir. Bu bulgular, yapay zekâ tabanlı karar ağaçlarının acil sağlık hizmetlerinde standartlaştırılmış ve doğru triyaj uygulamalarını destekleme potansiyeline sahip olduğunu göstermektedir.

Keywords

• acil servis
• karar ağaçları
• triyaj
• yapay zeka

Supporting Institution

Bulunmamaktadır

Ethical Statement

Çalışma için etik onay, Akdeniz Üniversitesi Tıp Fakültesi Klinik Araştırmalar Etik Kurulu’ndan alınmıştır (Onay Tarihi ve No: 30/05/2011-193). Çalışma, Helsinki Bildirgesi’nde belirtilen etik ilkelere uygun olarak yürütülmüştür.

Thanks

Bulunmamaktadır

References

1. Shiber, J. R., Longley, M. B., & Brewer, K. L. (2009). Hyper-use of the ED. World journal of emergency medicine, 27(5), 588-594. https://doi.org/10.1016/j.ajem.2008.05.006
2. Pines, J. M., Prabhu, A., McCusker, C. M., & Hollander, J. E. (2010). The effect of ED crowding on education. Am J Emerg Med, 28(2), 217-220. https://doi.org/10.1016/j.ajem.2008.10.030
3. Dikme, O., Atilla, R., Colak, Oray, N., Saracoglu, G., Girgin, M., & Akyol, P. Y. (2019). Dokuz Eylul Triage System: A five category triage algorithm, reliability and validity study. Anatolian J Emerg Med, 2(3), 1-7.
4. Morey, J. R., Zhang, X., Yaeger, K. A., Fiano, E., Marayati, N. F., & Kellner, C. P., (2021). Real-world experience with artificial intelligence-based triage in transferred large vessel occlusion stroke patients. Cerebrovasc Dis, 50(4), 450-455. https://doi.org/10.1159/000515320
5. McCarthy, J. What is artificial intelligence? Computer Science Department, Stanford University. Available at: https://www-formal.stanford.edu/jmc/whatisai.pdf. Accessed December 31, 2025.
6. Fahim, Y. A., Hasani, I. W., Kabba, S., & Ragab, W. M. (2025). Artificial intelligence in healthcare and medicine: clinical applications, therapeutic advances, and future perspectives. European journal of medical research, 30(1), 848. https://doi.org/10.1186/s40001-025-03196-w
7. Zadeh, L.A. (1969). Biological Applications of the Theory of Fuzzy Set and Systems, In: Proctor, L. D., editors. The Proceedings of an International Symposium on Biocybernetics of the Central Nervous System, Little, Brown and Company, Boston; p. 199-206.
8. Kalmanson, D., & Stegall, H. F. (1975). Cardiovascular investigations and fuzzy sets theory. The American journal of cardiology, 35(1), 80-84. https://doi.org/10.1016/0002-9149(75)90562-7

9. Jain, R., Mazumdar, J., & Moran, W. (1998). Application of fuzzy classifier system to coronary artery disease and breast cancer. Australasian physical & engineering sciences in medicine, 21(3), 141-147.
10. Güler, I., Hardalaç, F., & Barişçi, N. (2002). Application of FFT analyzed cardiac Doppler signals to fuzzy algorithm. Computers in biology and medicine, 32(6), 435-444. https://doi.org/10.1016/s0010-4825(02)00021-5
11. Guler, D., Hardalac, F., Ergu, U., & Barisci, N. (2004). Classification of Aorta Doppler signals using variable coded-hierarchical genetic fuzzy system. Expert systems with applications, 26(3), 321-333. https://doi.org/10.1016/j.eswa.2003.09.004
12. Gulkesen, K. H., Koksal, I. T., Ozdem, S., & Saka, O. (2010). Prediction of prostate cancer using decision tree algorithm. Turkish journal of medical sciences, 40(5), 681-686. https://doi.org/10.3906/sag-0906-47
13. Eken, C., Bilge, U., Kartal, M., & Eray, O. (2009). Artificial neural network, genetic algorithm, and logistic regression applications for predicting renal colic in emergency settings. International journal of emergency medicine, 2(2), 99-105. https://doi.org/10.1007/s12245-009-0103-1
14. Liu, Y., Wei, L. Q., Li, G. Q., Lv, F. Y., Wang, H., & Zhang, Y. H., & Cao, W. L. (2010). A decision-tree model for predicting extubation outcome in elderly patients after a successful spontaneous breathing trial. Anesthesia and analgesia, 111(5), 1211-1218. https://doi.org/10.1213/ANE.0b013e3181f4e82e
15. Genişol, İ. & Bakırarar, B. (2025). Application of machine learning techniques for survival prediction in pediatric malignant non-seminomatous germ cell testicular tumors: a SEER database study. Pamukkale Medical Journal, 18(2), 358-367. https://doi.org/10.31362/patd.1528303
16. Harrison, R. F. & Kennedy, R. L. (2005). Artificial neural network models for prediction of acute coronary syndromes using clinical data from the time of presentation. Annals of emergency medicine, 46(5), 431-439. https://doi.org/10.1016/j.annemergmed.2004.09.012
17. Bektaş, F., Eken, C., Soyuncu, S., Kilicaslan, I., & Cete, Y. (2008). Artificial neural network in predicting craniocervical junction injury: an alternative approach to trauma patients. European journal of emergency medicine : official journal of the European Society for Emergency Medicine, 15(6), 318-323. https://doi.org/10.1097/MEJ.0b013e3282fce7af
18. Azeez, D., Ali, M. A., Gan, K. B., & Saiboon, I. (2023). Comparison of adaptive neuro-fuzzy inference system and artificial neutral networks model to categorize patients in the emergency department. SpringerPlus, 2, 416. https://doi.org/10.1186/2193-1801-2-416
19. Williams, C. Y. K., Zack, T., Miao, B. Y., Sushil, M., Wang, M., & Kornblith, A. E., (2024) Use of a large language model to assess clinical acuity of adults in the emergency department. JAMA network open, 7(5), e248895. https://doi.org/10.1001/jamanetworkopen.2024.8895
20. Arslan, B., Nuhoglu, C., Satici, M. O., & Altinbilek, E. (2025). Evaluating LLM-based generative AI tools in emergency triage: A comparative study of ChatGPT Plus, Copilot Pro, and triage nurses. The American journal of emergency medicine, 89, 174-181. https://doi.org/10.1016/j.ajem.2024.12.024
21. Zaboli, A., Brigo, F., Brigiari, G., Massar, M., Parodi, M., & Pfeifer, N., (2025). Chat-GPT in triage: Still far from surpassing human expertise - An observational study. The American journal of emergency medicine, 92, 165-171. https://doi.org/10.1016/j.ajem.2025.03.028
22. Porto, B. M. (2024). Improving triage performance in emergency departments using machine learning and natural language processing: a systematic review. BMC emergency medicine, 24(1), 219. https://doi.org/10.1186/s12873-024-01135-2