RF-MRPM: A Random Forest–based Mortality Risk Prediction Model Using Clinical Features for Intensive Care Unit Patients with Superior Predictive Accuracy over the APACHE Score

Abstract

Accurate early prediction of mortality in intensive care unit (ICU) patients is crucial for clinical decision-making and resource allocation. Traditional severity scoring systems, such as APACHE, provide valuable prognostic information but may be limited in capturing complex nonlinear relationships among routinely collected clinical variables. Machine learning models offer the potential to enhance predictive performance using real-world clinical data. In this study, a real-world ICU dataset comprising 15,728 adult patients was analyzed. The data were obtained from a clinical information system developed by a software company. Following systematic data preprocessing, a novel Random Forest–Based Mortality Prediction Model (RF-MRPM) was developed to predict in-hospital mortality as a binary outcome. Comparative analyses were conducted against logistic regression, support vector machines, gradient boosting, and XGBoost models. The random forest model demonstrated the highest overall performance, achieving an AUC of 0.917, an accuracy of 0.846, a precision of 0.754, a recall of 0.793, and an F-measure of 0.773. In contrast, an APACHE-only logistic regression model yielded an AUC of 0.754, indicating substantially lower discriminative ability. Feature importance analysis identified APACHE score, serum creatinine, white blood cell count, age, heart rate, and electrolyte levels as the most influential predictors of mortality. Calibration analysis showed good agreement between predicted and observed mortality risks. In conclusion, RF-MRPM provides accurate and well-calibrated mortality predictions using routinely available ICU data and serve as a complementary decision-support tool alongside traditional severity scoring systems.

Keywords

• Machine Learning
• Mortality Prediction
• Clinical Decision Support
• Healthcare Analytics
• Intensive Care Unit
• Critical Care Informatics

RF-MRPM: Yoğun Bakım Ünitesi hastaları için klinik özellikler kullanarak geliştirilen, APACHE skoruna kıyasla üstün tahmin doğruluğu sağlayan Random Forest tabanlı mortalite risk tahmin modeli

Abstract

Yoğun bakım ünitesindeki (YBÜ) hastalarda ölüm oranının doğru ve erken tahmini, klinik karar verme ve kaynak tahsisi için çok önemlidir. APACHE gibi geleneksel şiddet puanlama sistemleri, değerli prognostik bilgiler sağlasa da, rutin olarak toplanan klinik değişkenler arasındaki karmaşık doğrusal olmayan ilişkileri yakalamada sınırlı kalabilir. Makine öğrenmesi modelleri, gerçek dünya klinik verilerini kullanarak tahmin performansını artırma potansiyeli sunmaktadır. Bu çalışmada, 15.728 yetişkin hastadan oluşan gerçek dünya YBÜ veri seti analiz edildi. Veriler, bir yazılım şirketi tarafından geliştirilen bir klinik bilgi sisteminden elde edildi. Sistematik veri ön işleme sonrasında, hastane içi ölüm oranını ikili bir sonuç olarak tahmin etmek için yeni bir Rastgele Orman Tabanlı Ölüm Tahmini Modeli (RF-MPM) geliştirildi. Lojistik regresyon, destek vektör makineleri, gradyan artırma ve XGBoost modellerine karşı karşılaştırmalı analizler yapıldı. Rastgele orman modeli, 0,917'lik bir AUC, 0,846'lık bir doğruluk, 0,754'lük bir hassasiyet, 0,793'lük bir geri çağırma ve 0,773'lük bir F-ölçüsü elde ederek en yüksek genel performansı göstermiştir. Buna karşılık, yalnızca APACHE'yi içeren lojistik regresyon modeli, önemli ölçüde daha düşük ayırt edici yeteneği gösteren 0,754'lük bir AUC vermiştir. Özellik önem analizi, APACHE skoru, serum kreatinin, beyaz kan hücresi sayısı, yaş, kalp hızı ve elektrolit seviyelerini mortalitenin en etkili belirleyicileri olarak tanımlamıştır. Kalibrasyon analizi, tahmin edilen ve gözlemlenen mortalite riskleri arasında iyi bir uyum göstermiştir. Sonuç olarak, RF-MPM, rutin olarak mevcut yoğun bakım verilerini kullanarak doğru ve iyi kalibre edilmiş mortalite tahminleri sağlar ve geleneksel şiddet skorlama sistemlerinin yanında tamamlayıcı bir karar destek aracı olarak hizmet eder.

Keywords

• Makine Öğrenmesi
• Mortalite Tahmini
• Klinik Karar Destek Sistemi
• Sağlık Veri Analitiği
• Yoğun Bakım Ünitesi
• Kritik Bakım Bilişimi

References

1. A. Dhami, K. A. Onyeukwu, S. Sattar, A. Batra, Y. Mostafa, M. Haris, et al., "The prognostic performance of artificial intelligence and machine learning models for mortality prediction in intensive care units: A systematic review", Cureus, vol. 17, no. 8, 2025, Art. no. e90465.
2. H. Sun, M. Kang, H. Zhang, J. Jia, and Q. Wang, "Machine learning for predicting mortality in intensive care unit patients: A prognostic performance systematic review and meta-analysis", Nurs. Crit. Care, vol. 30, no. 6, 2025, Art. no. e70206.
3. P. N. Ferreira, J. A. Duarte, F. João, and A. Marta, "Performance of Acute Physiology and Chronic Health Evaluation (APACHE II), Simplified Acute Physiology Score (SAPS III), and Sequential Organ Failure Assessment (SOFA) scores in a medical intermediate care unit", Cureus, vol. 17, no. 7, 2025, Art. no. e88321.
4. J. I. Salluh and M. Soares, "ICU severity of illness scores: APACHE, SAPS and MPM", Curr. Opin. Crit. Care, vol. 20, no. 5, pp. 557–565, 2014.
5. A. L. E. Falcão, A. G. de Almeida Barros, A. A. M. Bezerra, N. L. Ferreira, C. M. Logato, F. P. Silva, et al., "The prognostic accuracy evaluation of SAPS 3, SOFA and APACHE II scores for mortality prediction in the surgical ICU: an external validation study and decision-making analysis", Ann. Intensive Care, vol. 9, no. 1, 2019, Art. no. 18.
6. X. Liu, M. Shen, M. Lie, Z. Zhang, C. Liu, D. Li, et al., "Evaluating prognostic bias of critical illness severity scores based on age, sex, and primary language in the United States: A retrospective multicenter study", Crit. Care Explor., vol. 6, no. 1, 2024, Art. no. e1033.
7. R. Raj, M. B. Skrifvars, S. Bendel, T. Selander, R. Kivisaari, J. Siironen, et al., "Predicting six-month mortality of patients with traumatic brain injury: usefulness of common intensive care severity scores", Crit. Care, vol. 18, no. 2, 2014, Art. no. R60.
8. D. Shillan, J. A. Sterne, A. Champneys, and B. Gibbison, "Use of machine learning to analyse routinely collected intensive care unit data: a systematic review", Crit. Care, vol. 23, no. 1, 2019, Art. no. 284.

9. M. H. Choi, D. Kim, E. J. Choi, Y. J. Jung, Y. J. Choi, J. H. Cho, et al., "Mortality prediction of patients in intensive care units using machine learning algorithms based on electronic health records", Sci. Rep., vol. 12, no. 1, 2022, Art. no. 7180.
10. B. Nistal-Nuño, "Developing machine learning models for prediction of mortality in the medical intensive care unit", Comput. Methods Programs Biomed., vol. 216, 2022, Art. no. 106663.
11. P. Rockenschaub, E. M. Akay, B. G. Carlisle, A. Hilbert, J. Wendland, F. Meyer-Eschenbach, et al., "External validation of AI-based scoring systems in the ICU: a systematic review and meta-analysis", BMC Med. Inform. Decis. Mak., vol. 25, no. 1, 2025, Art. no. 5.
12. B. Ren, Y. Zhang, S. Chen, J. Dai, J. Chong, Y. Zhong, et al., "Interpretable prediction of hospital mortality in bleeding critically ill patients based on machine learning and SHAP", BMC Med. Inform. Decis. Mak., vol. 25, no. 1, 2025, Art. no. 263.
13. W. Liengswangwong, R. Siriwannabhorn, S. Leela-Amornsin, C. Yuksen, P. Sanguanwit, C. Duangsri, et al., "Comparison of Modified Early Warning Score (MEWS), Simplified Acute Physiology Score II (SAPS II), Sequential Organ Failure Assessment (SOFA), and Acute Physiology and Chronic Health Evaluation II (APACHE II) for early prediction of septic shock in diabetic patients in Emergency Departments", BMC Emerg. Med., vol. 24, no. 1, 2024, Art. no. 161.
14. B. Mamandipoor, J. McAuley, M. Krause, et al., "A deep learning-based early warning system for renal replacement therapy in the intensive care unit", preprint, Jan. 06, 2026. [Online]. Available: https://doi.org/10.21203/rs.3.rs-8419139/v1
15. R. H. Chou, B. W. Y. Hsu, C. L. Yu, T. Y. Chen, S. M. Ou, K. H. Lee, et al., "Machine-learning models are superior to severity scoring systems for the prediction of the mortality of critically ill patients in a tertiary medical center", J. Chin. Med. Assoc., vol. 87, no. 4, pp. 369–376, 2024.
16. Z. Zheng, J. Xia, J. Luo, L. Du, X. Zhou, X. Yang, et al., "Using machine learning for early prediction of in-hospital mortality during ICU admission in liver cancer patients", Sci. Rep., vol. 15, no. 1, 2025, Art. no. 40507.
17. S. An, Z. Ye, W. Che, Y. Gao, J. Li, and J. Zheng, "Development and validation of machine learning models to predict in-hospital mortality in ICU patients with sepsis and chronic kidney disease", BMC Infect. Dis., vol. 25, no. 1, 2025, pp. 1–14.
18. J. Gao, Y. Lu, N. Ashrafi, I. Domingo, K. Alaei, and M. Pishgar, "Prediction of sepsis mortality in ICU patients using machine learning methods", BMC Med. Inform. Decis. Mak., vol. 24, no. 1, 2024, Art. no. 228.
19. L. Liao and E. Aagaard, "Translating machine learning interpretability into clinical insights for ICU mortality prediction", arXiv preprint arXiv:2508.00919, 2025.
20. P. Xie, Y. Hu, J. Li, Y. Ma, and J. Xiao, "Unlocking the potential of real-time ICU mortality prediction: redefining risk assessment with continuous data recovery", NPJ Digit. Med., vol. 8, no. 1, 2025, Art. no. 733.
21. B. Huang, D. Liang, R. Zou, X. Yu, G. Dan, H. Huang, et al., "Mortality prediction for patients with acute respiratory distress syndrome based on machine learning: a population-based study", Ann. Transl. Med., vol. 9, no. 9, 2021, Art. no. 794.
22. N. Nikravangolsefid, S. Reddy, H. H. Troung, M. Charkviani, J. Ninan, K. Kashani, et al., "Machine learning models for predicting mortality in sepsis: a systematic review", Crit. Care Med., vol. 52, no. 1, 2024, Art. no. S693.
23. H. Lee and P. Tsoi, "Feature-enhanced machine learning for all-cause mortality prediction in healthcare data", arXiv preprint arXiv:2503.21241, 2025.
24. M. Ma, P. Sun, Y. Li, and W. Huo, "Predicting the risk of mortality in ICU patients based on dynamic graph attention network of patient similarity", Math. Biosci. Eng., vol. 20, no. 8, pp. 15326–15345, 2023.
25. P. Rockenschaub, A. Hilbert, T. Kossen, P. Elbers, F. von Dincklage, V. I. Madai, et al., "The impact of multi-institution datasets on the generalizability of machine learning prediction models in the ICU", Crit. Care Med., vol. 52, no. 11, pp. 1710–1721, 2024.
26. D. B. Olawade, A. Da’Costa, J. E. Origbo, A. Osonuga, E. Egbon, J. Teke, et al., "Smart emergency care: a narrative review of predictive machine learning models", Ann. Transl. Med., vol. 13, no. 5, 2025, Art. no. 60.
27. K. Żerdziński, J. Janiec, K. Jóźwik, P. Łajczak, and Ł. J. Krzych, "Artificial intelligence in intensive care: An overview of systematic reviews with clinical maturity and readiness mapping", J. Clin. Med., vol. 15, no. 1, 2025, Art. no. 185.
28. P. Hadweh, A. Niset, M. Salvagno, M. Al Barajraji, S. El Hadwe, F. S. Taccone, et al., "Machine learning and artificial intelligence in intensive care medicine: critical recalibrations from rule-based systems to frontier models", J. Clin. Med., vol. 14, no. 12, 2025, Art. no. 4026.
29. A. Maćkiewicz and W. Ratajczak, "Principal components analysis (PCA)", Comput. Geosci., vol. 19, no. 3, pp. 303–342, 1993.
30. E. Bisong, "Building machine learning and deep learning models on Google Cloud Platform", Berkeley, CA, USA: Apress, 2019, pp. 59–64.
31. W. S. Noble, "What is a support vector machine?" Nat. Biotechnol., vol. 24, no. 12, pp. 1565–1567, 2006. S. J. Rigatti, "Random Forest", J. Insur. Med., vol. 47, no. 1, pp. 31–39, 2017.
32. A. Natekin and A. Knoll, "Gradient boosting machines, a tutorial", Front. Neurorobot., vol. 7, 2013, Art. no. 21.
33. T. Chen, T. He, M. Benesty, V. Khotilovich, Y. Tang, H. Cho, et al., "XGBoost: Extreme gradient boosting", R package version 0.4-2, 2015, pp. 1–4.
34. Z. Zhang, C. Zhao, Y. Zhou, L. Yao, P. Liu, Z. Fang, et al., "Random forest-driven mortality prediction in critical IBD care: a dual-database model integrating comorbidity patterns and real-time physiometrics", Front. Med., vol. 12, 2025, Art. no. 1624899.
35. E. Yan, N. Butris, Y. Alhamdah, P. Kapoor, L. E. Lovblom, J. Wong, et al., "Evaluating prevalence and trajectory of functional disability in older surgical patients: An observational cohort study", J. Clin. Anesth., vol. 99, 2024, Art. no. 111681.
36. B. Chen, J. Zhou, S. Chen, F. Wang, P. Liu, Y. Xu, et al., "Interpretable machine learning prediction model for predicting mortality risk of ICU patients with pressure ulcers based on the Braden Scale: A clinical study based on MIMIC-IV", J. Clin. Nurs., 2025.
37. J. Zhang, Y. Fu, Y. Liu, T. Liu, Y. Deng, L. Dai, et al., "Risk prediction models for short-term mortality in ICU stroke patients: a systematic review and meta-analysis", Front. Neurol., vol. 16, 2025, Art. no. 1623645.
38. Y. C. Yeh, Y. T. Kuo, K. C. Kuo, Y. W. Cheng, D. S. Liu, F. Lai, et al., "Early prediction of mortality upon intensive care unit admission", BMC Med. Inform. Decis. Mak., vol. 24, no. 1, 2024, Art. no. 394.
39. Z. Chen, T. Li, S. Guo, D. Zeng, and K. Wang, "Machine learning-based in-hospital mortality risk prediction tool for intensive care unit patients with heart failure", Front. Cardiovasc. Med., vol. 10, 2023, Art. no. 1119699.
40. T. Huang, D. Le, L. Yuan, S. Xu, and X. Peng, "Machine learning for prediction of in-hospital mortality in lung cancer patients admitted to intensive care unit", PLoS One, vol. 18, no. 1, 2023, Art. no. e0280606.
41. M. M. Alam, "An efficient random forest algorithm-based telemonitoring framework to predict mortality and length of stay of patients in ICU", Multimed. Tools Appl., vol. 83, no. 17, pp. 50581–50600, 2024.
42. Y. Zhang, J. Hu, T. Hua, J. Zhang, Z. Zhang, and M. Yang, "Development of a machine learning-based prediction model for sepsis-associated delirium in the intensive care unit", Sci. Rep., vol. 13, no. 1, 2023, Art. no. 12697.
43. Y. Sun, Z. He, J. Ren, and Y. Wu, "Prediction model of in-hospital mortality in intensive care unit patients with cardiac arrest: a retrospective analysis of MIMIC-IV database based on machine learning", BMC Anesthesiol., vol. 23, no. 1, 2023, Art. no. 178.
44. C. Luo, Y. Zhu, Z. Zhu, R. Li, G. Chen, and Z. Wang, "A machine learning-based risk stratification tool for in-hospital mortality of intensive care unit patients with heart failure", J. Transl. Med., vol. 20, no. 1, 2022, Art. no. 136.
45. L. Zhang, T. Huang, F. Xu, S. Li, S. Zheng, J. Lyu, et al., "Prediction of prognosis in elderly patients with sepsis based on machine learning (random survival forest)", BMC Emerg. Med., vol. 22, no. 1, 2022, Art. no. 26.
46. M. H. Choi, D. Kim, E. J. Choi, Y. J. Jung, Y. J. Choi, J. H. Cho, et al., "Mortality prediction of patients in intensive care units using machine learning algorithms based on electronic health records", Sci. Rep., vol. 12, no. 1, 2022, Art. no. 7180.
47. J. Hashemi, B. Patel, Y. S. Chatzizisis, and G. S. Kassab, "Real time reduced order model for angiography fractional flow reserve", Comput. Methods Programs Biomed., vol. 216, 2022, Art. no. 106674.
48. K. Pang, L. Li, W. Ouyang, X. Liu, and Y. Tang, "Establishment of ICU mortality risk prediction models with machine learning algorithm using MIMIC-IV database", Diagnostics, vol. 12, no. 5, 2022, Art. no. 1068.
49. J. Lee, J. Cai, F. Li, and Z. A. Vesoulis, "Predicting mortality risk for preterm infants using random forest", Sci. Rep., vol. 11, no. 1, 2021, Art. no. 7308.
50. K. Alghatani, N. Ammar, A. Rezgui, and A. Shaban-Nejad, "Predicting intensive care unit length of stay and mortality using patient vital signs: machine learning model development and validation", JMIR Med. Inform., vol. 9, no. 5, 2021, Art. no. e21347.
51. F. Li, H. Xin, J. Zhang, M. Fu, J. Zhou, and Z. Lian, "Prediction model of in-hospital mortality in intensive care unit patients with heart failure: machine learning-based, retrospective analysis of the MIMIC-III database", BMJ Open, vol. 11, no. 7, 2021, Art. no. e044779.
52. X. Guan, B. Zhang, M. Fu, M. Li, X. Yuan, Y. Zhu, et al., "Clinical and inflammatory features based machine learning model for fatal risk prediction of hospitalized COVID-19 patients: results from a retrospective cohort study", Ann. Med., vol. 53, no. 1, 2021, pp. 257–266.
53. A. Rodríguez, M. Ruiz-Botella, I. Martín-Loeches, M. Jimenez Herrera, J. Solé-Violan, J. Gómez, et al., "Deploying unsupervised clustering analysis to derive clinical phenotypes and risk factors associated with mortality risk in 2022 critically ill patients with COVID-19 in Spain", Crit. Care, vol. 25, no. 1, 2021, Art. no. 63.