Analysis of Artificial Intelligence Methods in Classifying Heart Attack Risk: Black-Box Models vs. Glass-Box Models

Abstract

Artificial Intelligence (AI) is becoming more and more involved in human life day by day. Healthcare is one of the areas where AI is widely used, such as in the diagnosis prediction, and/or classification of diseases. Techniques such as machine learning provide high-accuracy results, but many algorithms have black-box structures, where the reasoning behind the predictions is not known. Explainable AI emerges to address this by providing explanations for complex models. While interpretable ("glass-box") models are desirable, they may have lower accuracy than complex ("black-box") models. Finding the right balance is crucial, especially in critical areas such as healthcare. It is also important to provide individual explanations for the predictions. This study uses patient data to explore a model to predict heart attack risk. Therefore, we compare glass-box models (logistic regression, naive Bayes, decision tree, and explainable boosting) with black-box models (random forest, support vector machine, multi-layer perceptron, gradient boosting, and stochastic gradient boosting). The results show that explainable boosting achieves the highest accuracy. To delve into individual explanations on a patient basis, the explainable boosting algorithm is compared with the random forest algorithm, which gives the best results among the black-box models. Here, LIME and SHAP are used to provide interpretability of random forests. As a result, it is concluded that the random forest algorithm has differences in the importance weights of the variables compared to the explainable boosting algorithm. Both results provide valuable tools for healthcare stakeholders to choose the most appropriate model.

Keywords

• Artificial Learning
• Explainable Artificial Intelligence
• Classification
• Healthcare Industry
• Heart Attack

Kalp Krizi Riskinin Sınıflandırılmasında Yapay Zekâ Yöntemlerinin Analizi: Kara Kutu Modelleri ve Cam Kutu Modelleri

Abstract

Yapay Zekâ (YZ) her geçen gün insan hayatına giderek daha fazla dâhil olmaktadır. Sağlık sektörü, hastalıkların teşhisi, tahmini ve/veya sınıflandırılması gibi yapay zekânın yaygın olarak kullanıldığı alanlardan biridir. Makine öğrenimi gibi teknikler yüksek doğrulukta sonuçlar sağlar ancak çoğu algoritma, tahminlerin ardındaki mantığın bilinmediği kara kutulardır. Açıklanabilir Yapay Zekâ, karmaşık modeller için açıklamalar sağlayarak bu sorunu çözmek üzere ortaya çıkmaktadır. Yorumlanabilir ("cam kutu") modeller tercih edilmekle birlikte, karmaşık ("kara kutu") modellerden daha düşük doğruluğa sahip olabilirler. Özellikle sağlık gibi kritik alanlarda doğru dengeyi bulmak çok önemlidir. Ayrıca tahminlerin bireysel olarak açıklanması da büyük önem taşımaktadır. Bu çalışmada hasta verilerine dayanarak kalp krizi riskini tahmin etmeye yönelik bir model araştırıyoruz. Bu amaçla, cam kutu modellerini (lojistik regresyon, naif Bayes, karar ağacı ve açıklanabilir artıma) kara kutu modelleriyle (rastgele orman, destek vektör makinesi, çok katmanlı algılayıcılar ve gradyan artırma) karşılaştırıyoruz. Sonuçlar açıklanabilir güçlendirmenin en yüksek doğruluğu sağladığını göstermektedir. Hasta bazında bireysel açıklamalara girebilmek için açıklanabilir artırma algoritması, kara kutu modelleri arasında en iyi sonuçları veren rastgele orman algoritması ile karşılaştırılmıştır. Burada rastgele ormanların yorumlanabilirliğini sağlamak için LIME ve SHAP teknikleri kullanılmıştır. Sonuç olarak rastgele orman algoritmasının açıklanabilir artırma algoritmasına göre değişkenlerin önem ağırlıklarında farklılıklar olduğu sonucuna varılmıştır. Her iki sonuç da sağlık hizmeti paydaşlarının en uygun modeli seçmeleri için değerli araçlar sunmaktadır.

Keywords

• Yapay Öğrenme
• Açıklanabilir Yapay Zekâ
• Sınıflandırma
• Sağlık Sektörü
• Kalp Krizi

References

1. Arrieta, A.B. et al. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. fusion, vol. 58, pp. 82–115.
2. Longo, L. et al. (2024). Explainable artificial intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions. Inf. Fusion, p. 102301.
3. Langer, M. et al. (2021). What do we want from Explainable Artificial Intelligence (XAI)?--A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. Artif. Intell., vol. 296, p. 103473.
4. Retzlaff, C.O. et al. (2024). Post-hoc vs ante-hoc explanations: xAI design guidelines for data scientists. Cogn. Syst. Res., vol. 86, p. 101243.
5. Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell., vol. 1, no. 5, pp. 206–215.
6. Cinà, G., Röber, T., Goedhart, R., and Birbil, I. (2022). Why we do need explainable ai for healthcare, arXiv Prepr. arXiv2206.15363.
7. Wysocki, O. et al. (2023). Assessing the communication gap between AI models and healthcare professionals: Explainability, utility and trust in AI-driven clinical decision-making. Artif. Intell., vol. 316, p. 103839.
8. Nasarian, E., Alizadehsani, R., Acharya, U.R., and Tsui, K.-L. (2024). Designing interpretable ML system to enhance trust in healthcare: A systematic review to proposed responsible clinician-AI-collaboration framework. Inf. Fusion, p. 102412.

9. Riyaz, L., Butt, M.A., Zaman, M., and Ayob, O. (2022). Heart disease prediction using machine learning techniques: a quantitative review, in International Conference on Innovative Computing and Communications: Proceedings of ICICC 2021, Volume 3, pp. 81–94.
10. Habehh, H. and Gohel, S. (2021). Machine learning in healthcare. Curr. Genomics, vol. 22, no. 4, p. 291.
11. Liang, Z., Zhang, G., Huang, J.X., and Hu, Q. V. (2014). Deep learning for healthcare decision making with EMRs, in 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 556–559.
12. Patel, M.J., Andreescu, C., Price, J.C., Edelman, K.L., Reynolds III, C.F. and Aizenstein, H.J. (2015). Machine learning approaches for integrating clinical and imaging features in late-life depression classification and response prediction. Int. J. Geriatr. Psychiatry, vol. 30, no. 10, pp. 1056–1067.
13. Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, vol. 542, no. 7639, pp. 115–118, doi: 10.1038/nature21056.
14. o’Brien, A. R., Wilson, L.O.W., Burgio, G. and Bauer, D.C. (2019). Unlocking HDR-mediated nucleotide editing by identifying high-efficiency target sites using machine learning. Sci. Rep., vol. 9, no. 1, p. 2788.
15. Pan, X., et al. (2020). ToxDL: deep learning using primary structure and domain embeddings for assessing protein toxicity. Bioinformatics, vol. 36, no. 21, pp. 5159–5168.
16. Ahsan, M.M. and Siddique, Z. (2022). Machine learning-based heart disease diagnosis: A systematic literature review. Artif. Intell. Med., vol. 128, p. 102289.
17. Sahu, R., Mohanty, K., Dash, S.R., Brahnam, S., and Barra, P. (2023). Prediction of Heart Attack and Death: Comparison Between 1 DCNN and Conventional ML Approaches, in 2023 1st International Conference on Circuits, Power and Intelligent Systems (CCPIS), pp. 1–6.
18. Rao, K.D., Kumar, M.S.D., Akshitha, D. and Rao, K.N. (2022). Machine Learning Based Cardiovascular Disease Prediction, in 2022 International Conference on Computer, Power and Communications (ICCPC), pp. 118–122.
19. Mahmud, I., Kabir, M.M., Mridha, M.F., Alfarhood, S., Safran, M. and Che, D. (2023). Cardiac Failure Forecasting Based on Clinical Data Using a Lightweight Machine Learning Metamodel. Diagnostics, vol. 13, no. 15, p. 2540.
20. Khan Mamun, M.M.R. and Elfouly, T. (2023). Detection of Cardiovascular Disease from Clinical Parameters Using a One-Dimensional Convolutional Neural Network. Bioengineering, vol. 10, no. 7, p. 796.
21. Ozcan, M. and Peker, S. (2023). A classification and regression tree algorithm for heart disease modeling and prediction. Healthc. Anal., vol. 3, p. 100130.
22. Yu, H. (2023). Analysis and Prediction of Heart Disease Based on Machine Learning Algorithms, in In 2023 8th International Conference on Intelligent Computing and Signal Processing (ICSP), pp. 1418–1423.
23. Saeed, W. and Omlin, C. (2023). Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities. Knowledge-Based Syst., vol. 263, p. 110273.
24. Lundberg, S.M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. Adv. Neural Inf. Process. Syst., vol. 30.
25. Ribeiro, M.T., Singh, S. and Guestrin, C. (2016). ‘Why should i trust you?’ Explaining the predictions of any classifier, in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135–1144.
26. Schwalbe, G. and Finzel, B. (2023). A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts. Data Min. Knowl. Discov., pp. 1–59.
27. James, G., Witten, D., Hastie, T., Tibshirani, R. and others (2013). An introduction to statistical learning, vol. 112. Springer.
28. Shah, K., Patel, H., Sanghvi, D., and Shah, M. (2020). A comparative analysis of logistic regression, random forest and KNN models for the text classification. Augment. Hum. Res., vol. 5, no. 1, p. 12.
29. Aborisade, O. and Anwar, M. (2018). Classification for authorship of tweets by comparing logistic regression and naive bayes classifiers, in 2018 IEEE International Conference on Information Reuse and Integration (IRI), pp. 269–276.
30. Stephens, C.R., Huerta, H.F. and Linares, A.R. (2018). When is the Naive Bayes approximation not so naive?. Mach. Learn., vol. 107, pp. 397–441.
31. Jadhav, S.D. and Channe, H.P. (2016). Comparative study of K-NN, naive Bayes and decision tree classification techniques. Int. J. Sci. Res., vol. 5, no. 1, pp. 1842–1845.
32. Dong, S. (2022). Virtual currency price prediction based on segmented integrated learning, in 2022 IEEE 2nd International Conference on Power, Electronics and Computer Applications (ICPECA), pp. 549–552.
33. Pattanayak, S., Loha, C., Hauchhum, L., and Sailo, L. (2021). Application of MLP-ANN models for estimating the higher heating value of bamboo biomass. Biomass Convers. Biorefinery, vol. 11, pp. 2499–2508.
34. Visani, G., Bagli, E., Chesani, F., Poluzzi, A. and Capuzzo, D. (2022). Statistical stability indices for LIME: Obtaining reliable explanations for machine learning models. J. Oper. Res. Soc., vol. 73, no. 1, pp. 91–101.
35. Wang, D., Thunéll, S., Lindberg, U., Jiang, L., Trygg, J. and Tysklind, M. (2022). Towards better process management in wastewater treatment plants: Process analytics based on SHAP values for tree-based machine learning methods. J. Environ. Manage., vol. 301, p. 113941.
36. Heart Disease Prediction, dataset by informatics-edu, 2020. [Online]. Available: https://data.world/informatics-edu/heart-disease-prediction. [Accessed: 11-May-2024].