Hard-Aware Heart Disease Prediction: SHAP-Based Interaction Engineering and Hybrid Ensemble Approach

Abstract

This study presents a stepwise model development process (T0–T3) aimed at improving reliability and accuracy in addressing classification problems encountered in the diagnosis of heart disease. The process begins with baseline models (T0) and includes SHapley Additive exPlanations (SHAP) based feature engineering and hyperparameter optimization (T1), probability calibration (T2), and a specialized hybrid approach for hard examples (T3). Data was split into a 70% training and 30% test set, and the best model was identified using a multi-criteria selection approach (single metrics, composite scoring). Our findings show that calibration improves the reliability of probability scores, which in turn facilitates the determination of triage thresholds. The hybrid architecture, while boosting sensitivity for challenging cases, remains competitive with the overall performance of a single best model. This work offers a transparent and accountable framework for clinical decision support by focusing not only on accuracy metrics but also on reliability measures.

Keywords

• Machine learning
• Tiered model development
• SHAP
• Clinical decision support

Ethical Statement

During the writing process of our study, the information of which is given above, international scientific, ethical and citation rules have been followed, no falsification has been made on the data collected, and Journal of Materials and Mechatronics: A (JournalMM) and its editorial board have no responsibility for any ethical violations that may be encountered. I undertake that I have full responsibility and that this study has not been evaluated in any academic environment other than Journal of Materials and Mechatronics: A (JournalMM).

Zor Örnek Farkındalıklı Kalp Hastalığı Tahmini: SHAP Tabanlı Etkileşim Mühendisliği ve Hibrit Topluluk Öğrenme Yaklaşımı

Abstract

Bu çalışma, kalp hastalığı teşhisinde karşılaşılan sınıflandırma problemlerine yönelik, güvenilirliği ve doğruluğu artırmayı hedefleyen aşamalı bir model geliştirme süreci (T0–T3) sunmaktadır. Bu süreç, varsayılan modellerden (T0) başlayarak, SHapley Additive exPlanations (SHAP) tabanlı özellik mühendisliği ve hiperparametre optimizasyonu (T1), olasılık kalibrasyonu (T2) ve zor örnekler için özel bir hibrit yaklaşımı (T3) içerir. Veriler %70 eğitim/%30 test şeklinde ayrılmış ve en iyi model çok ölçütlü seçim yöntemleri (tekil metrikler, bileşik puanlama) kullanılarak belirlenmiştir. Bulgular, kalibrasyonun olasılık güvenilirliğini artırarak triyaj eşiği belirlemede kolaylık sağladığını; hibrit mimarinin ise zor örneklerde duyarlılığı artırırken genel performansta rekabetçi kaldığını göstermektedir. Bu çalışma, sadece doğruluk metriklerine değil, aynı zamanda güvenilirlik ölçütlerine de odaklanarak klinik karar desteği için şeffaf ve hesap verebilir bir çerçeve sunmaktadır.

Keywords

• Makine öğrenmesi
• Aşamalı model geliştirme
• SHAP
• Klinik karar desteği

Ethical Statement

Yukarıda bilgisi verilen çalışmamızın yazım sürecinde uluslararası bilimsel, etik ve alıntı kurallarına uyulmuş, toplanan veriler üzerinde herhangi bir tahrifat yapılmamıştır, karşılaşılacak tüm etik ihlallerde Journal of Materials and Mechatronics: A (JournalMM) Dergisinin ve editör kurulunun hiçbir sorumluluğu yoktur. Tüm sorumluluğun bana ait olduğunu ve bu çalışmanın Journal of Materials and Mechatronics: A (JournalMM) Dergisinden başka hiçbir akademik yayın ortamında değerlendirilmemiş olduğunu taahhüt ederim.

References

1. Alférez, G. H., Esteban, O. A., Clausen, B. L., & Ardila, A. M. M. (2022). Automated machine learning pipeline for geochemical analysis. Earth Science Informatics, 15(3), 1683-1698.
2. Alvarez, S. A. (2002). An exact analytical relation among recall, precision, and classification accuracy in information retrieval. Boston College, Boston, Technical Report BCCS-02, 1, 1-22.
3. Aragonés Lozano, M., Pérez Llopis, I., & Esteve Domingo, M. (2023). Threat hunting system for protecting critical ınfrastructures using a machine learning approach. Mathematics, 11(16), 3448.
4. Aydemi̇r, Ö. (2022). Dengesiz veri kümelerinin sınıflandırılmasında poligon alan metriğinin sınıflandırıcı performans değerlendirilmesi için kullanılması. Yüzüncü Yıl Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 27(2), 194-205.
5. Behera, D. K., Dash, S., Behera, A. K., & Dash, C. S. K. (2021, December). Extreme gradient boosting and soft voting ensemble classifier for diabetes prediction. In 2021 19th OITS International Conference on Information Technology (OCIT) (pp. 191-195). IEEE.
6. Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, (pp. 785-794).
7. Demir, E., Bozkurt, F., & Ayık, Y. Z. (2023). Early-stage heart failure disease prediction with deep learning approach. Journal of Scientific Reports-A, 055, 34-49.
8. Detrano, R., Janosi, A., Steinbrunn, W., Pfisterer, M., Schmid, J.-J., Sandhu, S., Guppy, K. H., Lee, S., & Froelicher, V. (1989). International application of a new probability algorithm for the diagnosis of coronary artery disease. The American Journal of Cardiology, 64(5), 304-310.

9. Dogan, A., & Birant, D. (2019). A Weighted Majority Voting Ensemble Approach for Classification. 2019 4th International Conference on Computer Science and Engineering (UBMK), 1-6. 2019 4th International Conference on Computer Science and Engineering (UBMK).
10. Erdem, K., Yasin, E., Yıldız, M. B., & Koklu, M. (2024). Classification of heart diseases with ensemble learning algorithms. Sinop Üniversitesi Fen Bilimleri Dergisi, 9(2), 369-387.
11. Fedesoriano. (2021). Heart Failure Prediction Dataset. Kaggle. https://www.kaggle.com/datasets/fedesoriano/heart-failure-prediction
12. Feng, C., Sutherland, A., King, S., Muggleton, S., & Henery. (1992). Statlog (Heart) [Dataset]. UCI Machine Learning Repository.
13. Filippou, K., Aifantis, G., Papakostas, G. A., & Tsekouras, G. E. (2023). Structure learning and hyperparameter optimization using an automated machine learning (AutoML) pipeline. Information, 14(4), 232.
14. Fränti, P., & Mariescu-Istodor, R. (2023). Soft precision and recall. Pattern Recognition Letters, 167, 115-121.
15. Ganie, S. M., Pramanik, P. K. D., & Zhao, Z. (2025). Ensemble learning with explainable AI for improved heart disease prediction based on multiple datasets. Scientific Reports, 15(1), 13912.
16. García, V., Mollineda, R. A., & Sánchez, J. S. (2009, June). Index of balanced accuracy: A performance measure for skewed class distributions. In Iberian conference on pattern recognition and image analysis (pp. 441-448). Berlin, Heidelberg: Springer Berlin Heidelberg.
17. Haihong, E., Zhou, K., & Song, M. (2019, December). Spark-based machine learning pipeline construction method. In 2019 International Conference on Machine Learning and Data Engineering (iCMLDE) (pp. 1-6). IEEE.
18. Hastie, T., Tibshirani, R., & Friedman, J. (2008). Model assessment and selection. In The elements of statistical learning: data mining, inference, and prediction (pp. 219-259). New York, NY: Springer New York.
19. Kaya, E. N., & Görmez, Y. (2025). Proje efor tahmini için makine öğrenmesi modellerinin geliştirilmesi ve shap yöntemi kullanılarak açıklanması. Mühendislik Bilimleri ve Tasarım Dergisi, 13(2), 528-544.
20. Kull, M., & Flach, P. A. (2014, September). Reliability maps: a tool to enhance probability estimates and improve classification accuracy. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 18-33). Berlin, Heidelberg: Springer Berlin Heidelberg.
21. Kumar, A., Liang, P. S., & Ma, T. (2019). Verified uncertainty calibration. Advances in Neural Information Processing Systems, 32.
22. Kusuma, C. C., Chandra, N., Utomo, G. N., Parimartha, P. A., Hidayat, J., Anggreainy, M. S., & Parmonangan, I. H. (2023, September). Diagnosing Diabetes Using Soft Voting Ensemble Machine Learning Model. In 2023 10th International Conference on ICT for Smart Society (ICISS) (pp. 1-6). IEEE.
23. Küçükmanisa, A., & Kilimci, Z. H. (2024). Heart Disease Prediction with Machine Learning-Based Approaches. Sakarya University Journal of Science, 28(1), 101-107. Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.
24. Niculescu-Mizil, A., & Caruana, R. (2005, August). Predicting good probabilities with supervised learning. In Proceedings of the 22nd international conference on Machine learning (pp. 625-632).
25. Öznacar, T., & Sertkaya, Z. T. (2024). Heart Failure Prediction: A Comparative Study of SHAP, LIME, and ICE in Machine Learning Models. International Journal of Computational and Experimental Science and Engineering, 10(4).
26. Pearly, A. A., & Karthik, B. (2024, October). A Hybrid Deep Learning Framework for Lung Disorder Detection Using Multi-Scale Feature Extraction and Ensemble Classification. In International Conference on Advancements in Smart Computing and Information Security (pp. 118-132). Cham: Springer Nature Switzerland.
27. Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifier, 10(3), 61-74.
28. Saihood, Q., & Sonuç, E. (2023). A practical framework for early detection of diabetes using ensemble machine learning models. Turkish Journal of Electrical Engineering and Computer Sciences, 31(4), 722-738.
29. Sanders, F. (1967). The Verification of Probability Forecasts. Journal of Applied Meteorology, 6(5), 756-761.
30. Shah, M., Gandhi, K., Patel, K. A., Kantawala, H., Patel, R., & Kothari, A. (2023). Theoretical Evaluation of Ensemble Machine Learning Techniques. 2023 5th International Conference on Smart Systems and Inventive Technology (ICSSIT), 829-837.
31. Shah, P., Shukla, M., Dholakia, N. H., & Gupta, H. (2025). Predicting cardiovascular risk with hybrid ensemble learning and explainable AI. Scientific Reports, 15(1), 17927.
32. Solanki, S., Verma, S., & Chahar, K. (2020). A comparative study of information retrieval using machine learning. Advances in Computing and Intelligent Systems: Proceedings of ICACM 2019, 35-42.
33. Stehouwer, N., Rowland-Seymour, A., Gruppen, L., Albert, J. M., & Qua, K. (2025). Validity and reliability of Brier scoring for assessment of probabilistic diagnostic reasoning. Diagnosis, 12(1), 53-60.
34. Sungur, F., & Bakır, H. (2024). Hiperparametre ayarlama ve veri dengelemenin kalp hastalığı tahmini için kullanılan makine öğrenimi algoritmaları üzerindeki etkilerinin incelenmesi. Bilişim Teknolojileri Dergisi, 17(1), 45-58.
35. Takcı, H. (2022). Optimum parametreler yardımıyla performansı artırılmış KNN algoritması tabanlı kalp hastalığı tahmini. Gazi Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi, 38(1), 451-460.
36. Tomak, L., & Bek, Y. (2011). İşlem Karakteristik Eğrisi Analizi ve Eğri Altında Kalan Alanların Karşılaştırılması. Deneysel ve Klinik Tıp Dergisi, 27(2).
37. Yapıcı, İ. Ş., Arslan, R. U., & Erkaymaz, O. (2024). Kalp Yetmezliği Tanılı Hastaların Hayatta Kalma Tahmininde Topluluk Makine Öğrenme Yöntemlerinin Performans Analizi. Karaelmas Fen ve Mühendislik Dergisi, 14(1), 59-69.