An Evaluation Study on the Application of Heart Failure Prediction with Categorically Different Type Machine Learning Methods

Yıl 2024, Cilt: 14 Sayı: 1, 73 - 85, 30.01.2024

İsmail Atacak

Öz

Heart failure is a serious health problem that negatively impacts the quality of life and can lead to fatal consequences if left untreated. Early diagnosis and proper treatment can minimize these problems. In this study, a model was developed to measure the performances of machine learning (ML) methods in different categories for heart failure prediction. and performance analyses were performed on both categorical and general basis. Tree, meta and function categories, which include methods known to produce successful results in classification problems, were preferred as category and five methods from each category were used. In the experimental studies, the performance of the ML methods was measured using basic metrics and classification error metrics based on the confusion matrix. When the experimental results were evaluated categorically, the best performances were obtained with the Alternating Decision Tree (ADT) method in the tree category for the metrics except for Recall and False Negative Rate, the Logit Boost (LBST) method in the meta category for the metrics except for Area under the ROC curve (AUC), and the Radial Bases Function Classifier (RBFC) method in the function category for the metrics except for the Precision and False Positive Rate (FPR). When considered the results in terms of the performances of all methods, the RBFC method exhibited the best performance with values of 0.8725 for Accuracy, 0.9173 for Recall, 0.8885 for F-score, 0.0827 for FNR, and 0.1275 for Misclassification Rate (MCR). On the other hand, the ADT method showed the best performance in terms of Precision, AUC, and FPR metrics with values of 0.8718, 0.9300 and 0.1610, respectively.

Anahtar Kelimeler

Machine learning, classification, heart failure

Kalp Yetmezliği Tahmininin Kategorik Olarak Farklı Tip Makine Öğrenmesi Yöntemleri ile Uygulanmasına Yönelik Bir Değerlendirme Çalışması

Öz

Kalp yetmezliği yaşam kalitesini olumsuz etkileyen ve tedavi edilmediğinde ölümcül sonuçlar doğurabilen ciddi bir sağlık problemidir. Erken teşhis ve doğru tedavinin uygulanması bu problemleri en aza indirebilir. Bu çalışmada farklı kategorilerde yer alan makine öğrenmesi (MÖ) yöntemlerinin kalp yetmezliği tahminindeki performanslarını ölçmek için bir model geliştirilerek, kategorik ve genel olarak performans analizleri gerçekleştirilmiştir. Kategori temelinde sınıflandırma problemlerinde başarılı sonuçlar ürettiği bilinen yöntemleri içeren ağaç, meta ve fonksiyon kategorileri tercih edilmiş ve her kategoriden beş yöntem kullanılmıştır. Deneysel çalışmalarda MÖ yöntemlerinin performansı Karışıklık matrisine dayanan temel metrikler ile sınıflandırma hata metrikleri üzerinden ölçülmüştür. Deneysel sonuçlar kategorik olarak değerlendirildiğinde en iyi performansların ağaç kategorisinde Duyarlılık ve Yanlış Negatif Oranı (False Negative Rate (FNR) ) dışındaki metriklerde Alternatif Karar Ağacı (Alternating Decision Tree | ADTree (ADT)) yöntemi, meta kategorisinde ROC eğrisi altında kalan alan (Area under the curve (AUC)) dışındaki metriklerde Eklemeli Lojistik Regresyon (Additive Logistic Regression | LogitBoost (LBST)) yöntemi ve fonksiyon kategorisinde Kesinlik ve Yanlış Pozitif Oranı (False Positive Rate (FPR)) dışındaki metriklerde Radyal Temelli Fonksiyon Sınıflandırıcı (Radial Bases Function Classifier (RBFC)) yöntemi ile elde edildiğini göstermektedir. Sonuçlara tüm yöntemlerin performansları açısından bakıldığında Doğruluk, Duyarlılık, F-skor, FNR ve Yanlış Sınıflandırma Oranı (Misclassification Rate (MCR)) metrikleri açısından 0.8725, 0.9173, 0.8885, 0.0827 ve 0.1275 değerleri ile RBFC yönteminin, Kesinlik, AUC ve FPR metrikleri açısından 0.8718, 0.9300 ve 0.1610 değerleri ile ADT yönteminin en iyi performansa sahip olduğu görülmüştür.

Anahtar Kelimeler

Makine öğrenmesi, sınıflandırma, kalp yetmezliği

Kaynakça

• [1] WHO, “Cardiovascular diseases (CVDs).” Accessed: Sep. 19, 2023. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds)
• [2] S. Bozkurt Keser and K. Keskin, “Kalp Yetmezliği Hastalarının Sağ Kalım Tahmini: Sınıflandırmaya Dayalı Makine Öğrenmesi Algoritmalarının Bir Uygulaması,” Afyon Kocatepe University Journal of Sciences and Engineering, vol. 23, no. 2, pp. 362–369, May 2023, doi: 10.35414/AKUFEMUBID.1033377.
• [3] G. Gürgen and S. Serttaş, “Kalp Yetmezliği Hastalığının Erken Teşhisinde Makine Öğrenimi Algoritmalarının Performans Karşılaştırması,” Euroasia Journal of Mathematics, Engineering, Natural & Medical Sciences, vol. 165, no. 10, pp. 165–174, 2023, doi: 10.5281/zenodo.8238065.
• [4] SBSGM, “Sağlık İstatistikleri Yıllığı 2019,” Ankara, 2019. Accessed: Nov. 07, 2023. [Online]. Available: https://sbsgm.saglik.gov.tr/Eklenti/40564/0/saglik-istatistikleri-yilligi-2019pdf.pdf
• [5] O. Gold and A. Iorshase, “Heart failure prediction framework using random forest and J48 with Adaboost algorithms,” Science World Journal, vol. 18, no. 2, pp. 165–175, Oct. 2023, doi: 10.4314/SWJ.V18I2.1.
• [6] S. Kaushik and R. Birok, “Heart Failure prediction using Xgboost algorithm and feature selection using feature permutation,” IEEE International Conference on Electrical, Computer and Communication Technologies, 2021, doi: 10.1109/ICECCT52121.2021.9616626.
• [7] N. S. Mansur Huang, Z. Ibrahim, and N. Mat Diah, “Machine learning techniques for early heart failure prediction / Nur Shahellin Mansur Huang, Zaidah Ibrahim and Norizan Mat Diah,” 2021, Accessed: Nov. 07, 2023. [Online]. Available: https://mjoc.uitm.edu.my/
• [8] Z. Masetic and A. Subasi, “Congestive heart failure detection using random forest classifier,” Comput Methods ds Programs Biomed, vol. 130, pp. 54–64, Jul. 2016, doi: 10.1016/J.CMPB.2016.03.020.
• [9] S. Kermani, M. Ghelich Oghli, A. Mohammadzadeh, and R. Kafieh, “NF-RCNN: Heart localization and right ventricle wall motion abnormality detection in cardiac MRI,” Physica Medica, vol. 70, pp. 65–74, Feb. 2020, doi: 10.1016/J.EJMP.2020.01.011.
• [10] K. A. Lara Hernandez, T. Rienmüller, D. Baumgartner, and C. Baumgartner, “Deep learning in spatiotemporal cardiac imaging: A review of methodologies and clinical usability,” Comput Biol Med, vol. 130, Mar. 2021, doi: 10.1016/J.COMPBIOMED.2020.104200.
• [11] N. Lessmann et al., “Deep convolutional neural networks for automatic coronary calcium scoring in a screening study with low-dose chest CT,” https://doi.org/10.1117/12.2216978, vol. 9785, pp. 255–260, Mar. 2016, doi: 10.1117/12.2216978.
• [12] N. Zhang et al., “Deep Learning for Diagnosis of Chronic Myocardial Infarction on Nonenhanced Cardiac Cine MRI,” Radiology, vol. 291, no. 3, pp. 606–607, 2019, doi: 10.1148/RADIOL.2019182304.
• [13] M. Porumb, E. Iadanza, S. Massaro, and L. Pecchia, “A convolutional neural network approach to detect congestive heart failure,” Biomed Signal Process Control, vol. 55, p. 101597, Jan. 2020, doi:10.1016/J.BSPC.2019.101597.
• [14] A. N. Shihab, M. J. Mokarrama, R. Karim, S. Khatun, and M. S. Arefin, “An iot-based heart disease detection system using rnn,” Advances in Intelligent Systems and Computing, vol. 1200 AISC, pp. 535–545, 2021, doi: 10.1007/978-3-030-51859-2_49/COVER.
• [15] G. Maragatham and S. Devi, “LSTM Model for Prediction of Heart Failure in Big Data,” J Med Syst, vol. 43, no. 5, May 2019, doi: 10.1007/S10916-019-1243-3.
• [16] S. Gao, Y. Zheng, and X. Guo, “Gated recurrent unit-based heart sound analysis for heart failure screening,” Biomed Eng Online, vol. 19, no. 1, Jan. 2020, doi: 10.1186/S12938-020-0747-X.
• [17] A. Çınar and S. A. Tuncer, “Classification of normal sinus rhythm, abnormal arrhythmia and congestive heart failure ECG signals using LSTM and hybrid CNN-SVM deep neural networks,” Comput Methods Biomech Biomed Engin, vol. 24, no. 2, pp. 203–214, 2021, doi: 10.1080/10255842.2020.1821192.
• [18] D. Li, X. Li, J. Zhao, and X. Bai, “Automatic staging model of heart failure based on deep learning,” Biomed Signal Process Control, vol. 52, pp. 77–83, Jul. 2019, doi: 10.1016/J.BSPC.2019.03.009.
• [19] V. K. Sudha and D. Kumar, “Hybrid CNN and LSTM Network For Heart Disease Prediction,” SN Comput Sci, vol. 4, no. 2, Mar. 2023, doi: 10.1007/S42979-022-01598-9.
• [20] L. Ali et al., “An Optimized Stacked Support Vector Machines Based Expert System for the Effective Prediction of Heart Failure,” IEEE Access, vol. 7, pp. 54007–54014, 2019, doi: 10.1109/ACCESS.2019.2909969.
• [21] Y. Chen, X. Qin, L. Zhang, and B. Yi, “A Novel Method of Heart Failure Prediction Based on DPCNN-XGBOOST Model,” Computers Materials & Continua, vol. 65, no. 1, pp. 495–510, Jul. 2020, doi: 10.32604/CMC.2020.011278.
• [22] V. Grgić, D. Mušić, and E. Babović, “Model for predicting heart failure using Random Forest and Logistic Regression algorithms,” IOP Conf Ser Mater Sci Eng, vol. 1208, no. 1, p. 012039, Nov. 2021, doi: 10.1088/1757-899X/1208/1/012039.
• [23] N. Foziljonova and I. Wasito, “Prediction of survival rate of heart failure patients using machine learning techniques,” J Theor Appl Inf Technol, vol. 15, no. 9, 2022, Accessed: Nov. 07, 2023. [Online]. Available: www.jatit.org
• [24] M. Mudassar, M. Afzal, and T. Muhammad, “A Machine Learning Based Predictive Model to Diagnose Heart Failure Patients using Imbalanced Classification Problem,” 2023 4th International Conference on Advancements in Computational Sciences, ICACS 2023 - Proceedings, 2023, doi:10.1109/ICACS55311.2023.10089759.
• [25] M. Zeng, “The Prediction of Heart Failure based on Four Machine Learning Algorithms,” Highlights in Science Engineering and Technology, vol. 39, pp. 1377–1382, Apr. 2023, doi: 10.54097/HSET.V39I.6771.
• [26] C. Coşkun and F. Kuncan, “Evaluation of Performance of Classification Algorithms in Prediction of Heart Failure Disease,” KSÜ Mühendislik Bilimleri Dergisi, vol. 25, no. 4, pp. 622–632, Dec. 2022, doi: 10.17780/KSUJES.1144570.
• [27] fedesoriano., “Heart Failure Prediction Dataset.” Accessed: Nov. 08, 2023. [Online]. Available: https://www.kaggle.com/datasets/fedesoriano/heart-failure-prediction
• [28] C. N. Villavicencio, J. J. E. Macrohon, X. A. Inbaraj, J. H. Jeng, and J. G. Hsieh, “Covid-19 prediction applying supervised machine learning algorithms with comparative analysis using weka,” Algorithms, vol. 14, no. 7, Jul. 2021, doi: 10.3390/A14070201.
• [29] Ç. Cem Berke, “Rastgele Orman Algoritması.” Accessed: Sep. 08, 2023. [Online]. Available: https://medium.com/@cemthecebi/rastgele-orman-algoritmas%C4%B1-1600ca4f4784
• [30] S. Gayathri, A. K. Krishna, V. P. Gopi, and P. Palanisamy, “Automated Binary and Multiclass Classification of Diabetic Retinopathy Using Haralick and Multiresolution Features,” IEEE Access, vol. 8, pp. 57497–57504, 2020, doi: 10.1109/ACCESS.2020.2979753.
• [31] J. Li, H. Peng, H. Hu, Z. Luo, and C. Tang, “Multimodal Information Fusion for Automatic Aesthetics Evaluation of Robotic Dance Poses,” Int J Soc Robot, vol. 12, no. 1, pp. 5–20, Jan. 2020, doi: 10.1007/S12369-019-00535-W/METRICS.
• [32] Y. Zhao and Y. Zhang, “Comparison of decision tree methods for finding active objects,” Advances in Space Research, vol. 41, no. 12, pp. 1955–1959, 2008, doi: 10.1016/J.ASR.2007.07.020.
• [33] E. Doğru, “Makine Öğrenmesi Teknikleri Kullanarak Hizmet Aksatma Saldırıları Tespiti,” Yüksek Lisans Tezi, Gazi Üniversitesi, Fen Bilimleri Enstitüsü, Ankara, 2020.
• [34] H. Yan and W. Chen, “Landslide susceptibility modeling based on GIS and ensemble techniques,” Arabian Journal of Geosciences 2022 15:8, vol. 15, no. 8, pp. 1–22, Apr. 2022, doi: 10.1007/S12517-022-09974-8.
• [35] Y.-S. Chen et al., “Comparable Studies of Financial Bankruptcy Prediction Using Advanced Hybrid Intelligent Classification Models to Provide Early Warning in the Electronics Industry,” Mathematics 2021, Vol. 9, Page 2622, vol. 9, no. 20, p. 2622, Oct. 2021, doi: 10.3390/MATH9202622.
• [36] K. Khosravi, Z. Sheikh Khozani, and J. Hatamiafkoueieh, “Prediction of embankments dam break peak outflow: a comparison between empirical equations and ensemble-based machine learning algorithms,” Natural Hazards, vol. 118, no. 3, pp. 1989–2018, Sep. 2023, doi: 10.1007/S11069-023-06060-4.
• [37] B. T. Pham et al., “GIS-based ensemble soft computing models for landslide susceptibility mapping,” Advances in Space Research, vol. 66, no. 6, pp. 1303–1320, Sep. 2020, doi: 10.1016/J.ASR.2020.05.016.
• [38] F. Bulut, “Sınıflandırıcı Topluluklarının Dengesiz Veri Kümeleri Üzerindeki Performans Analizleri,” Bilişim Teknolojileri Dergisi, vol. 9, no. 2, pp. 153–159, 2016, Accessed: Nov. 08, 2023. [Online]. Available: http://search/yayin/detay/220680
• [39] P. Ghosh et al., “Efficient prediction of cardiovascular disease using machine learning algorithms with relief and lasso feature selection techniques,” IEEE Access, vol. 9, pp. 19304–19326, 2021, doi: 10.1109/ACCESS.2021.3053759.
• [40] B. T. Pham and I. Prakash, “Evaluation and comparison of LogitBoost Ensemble, Fisher’s Linear Discriminant Analysis, logistic regression and support vector machines methods for landslide susceptibility mapping,” Geocarto Int, vol. 34, no. 3, pp. 316–333, Feb. 2019, doi: 10.1080/10106049.2017.1404141.
• [41] M. H. Kamarudin, C. Maple, T. Watson, and N. S. Safa, “A LogitBoost-Based Algorithm for Detecting Known and Unknown Web Attacks,” IEEE Access, vol. 5, pp. 26190–26200, Nov. 2017, doi: 10.1109/ACCESS.2017.2766844.
• [42] M. Kaya Keleş, “Breast cancer prediction and detection using data mining classification algorithms: A comparative study,” Tehnicki Vjesnik, vol. 26, no. 1, pp. 149–155, Feb. 2019, doi: 10.17559/TV-20180417102943.
• [43] Akca MF, “Nedir Bu Destek Vektör Makineleri? (Makine Öğrenmesi Serisi-2).” Accessed: Sep. 10, 2023. [Online]. Available: https://medium.com/deep-learning-turkiye/nedir-bu-destek-vekt%C3%B6r-makineleri-makine-%C3%B6%C4%9Frenmesi-serisi-2-94e576e4223e
• [44] M. P. Akhter, Z. Jiangbin, I. R. Naqvi, M. Abdelmajeed, A. Mehmood, and M. T. Sadiq, “Document-Level Text Classification Using Single-Layer Multisize Filters Convolutional Neural Network,” IEEE Access, vol. 8, pp. 42689–42707, 2020, doi: 10.1109/ACCESS.2020.2976744.
• [45] M. A. Hussain and L. Gogoi, “Performance analyses of five neural network classifiers on nodule classification in lung CT images using WEKA: a comparative study.,” Phys Eng Sci Med, vol. 45, no. 4, pp. 1193–1204, Dec. 2022, doi: 10.1007/S13246-022-01187-3.
• [46] W. Chen et al., “Landslide susceptibility modelling using GIS-based machine learning techniques for Chongren County, Jiangxi Province, China,” Science of the Total Environment, vol. 626, pp. 1121–1135, Jun. 2018, doi: 10.1016/J.SCITOTENV.2018.01.124.
• [47] C. Eyüpoğlu, “Büyük Veride Etkin Gizlilik Koruması İçin Yazılım Tasarımı ,” Doktora Tezi, İstanbul Üniversitesi, Fen Bilimleri Enstitüsü, İstanbul, 2018.
• [48] A. Firdaus, N. B. Anuar, M. F. A. Razak, and A. K. Sangaiah, “Bio-inspired computational paradigm for feature investigation and malware detection: interactive analytics,” Multimed Tools Appl, vol. 77, no. 14, pp. 17519–17555, Jul. 2018, doi: 10.1007/S11042-017-4586-0/METRICS.
• [49] J. Huang, S. Ling, X. Wu, and R. Deng, “GIS-Based Comparative Study of the Bayesian Network, Decision Table, Radial Basis Function Network and Stochastic Gradient Descent for the Spatial Prediction of Landslide Susceptibility,” Land (Basel), vol. 11, no. 3, Mar. 2022, doi: 10.3390/LAND11030436.
• [50] T. Srinivas et al., “Novel Based Ensemble Machine Learning Classifiers for Detecting Breast Cancer,” Math Probl Eng, vol. 2022, 2022, doi: 10.1155/2022/9619102.
• [51] H. Torun, “Karışıklık Matrisi (Confusion Matrix).” Accessed: Sep. 15, 2023. [Online]. Available: https://hakan.io/karisiklik-matrisi-confusion-matrix/
• [52] Y. Tang, “Python Algorithms: Accuracy, Precision, Recall, and F Score.” Accessed: Sep. 17, 2023. [Online]. Available: https://pythonalgos.com/2022/05/16/accuracy-precision-recall-and-f-score/
• [53] Ş. Ay, “Model Performansını Değerlendirmek — Metrikler.” Accessed: Sep. 17, 2023. [Online]. Available: https://medium.com/deep-learning-turkiye/model-performans%C4%B1n%C4%B1-de%C4%9Ferlendirmek-metrikler-cb6568705b1