Comparison of Machine Learning Regression Methods to Predict Health Expenditures

Abstract

As a result of experimental studies on different datasets, it is recommended to use machine learning regression methods as an alternative to classical regression methods in the existence of variables which are difficult to model. Health expenditure is an indicator which is difficult to model and there is no study in the literature about modelling health expenditure comparing machine learning regression methods. In this study a multiple regression model was conducted to predict health expenditure per capita. Performance results of Lasso Regression, Random Forest Regression and Support Vector Machine Regression compared when different hyperparameter values were determined. Lambda (λ) value for Lasso Regression, number of trees for Random Forest Regression, epsilon () value for Support Vector Regression was determined as hyperparameter values. Study results performed by using “k” fold cross validation changed from 5 to 50, indicate the difference between machine learning results in terms of R², RMSE and MAE values that are statistically significant (p<0.001). Surface and bar plots and statistical test results of prediction performances show that Random Forest Regression (R² ˃ 0.7500, RMSE ≤ 0.6000 ve MAE ≤ 0.4000) has better prediction performance according to different hyperparameter values. It is hoped that study results make contribution to studies about determining optimal hyperparameter values for machine learning regression methods for studies about modelling health expenditures. 

Keywords

• Machine Learning,Lasso Regression,Random Forest Regression,Support Vector Regression,Health Expenditure

SAĞLIK HARCAMASININ TAHMİNİNDE MAKİNE ÖĞRENMESİ REGRESYON YÖNTEMLERİNİN KARŞILAŞTIRILMASI

Abstract

Farklı veri setleri üzerinde yapılan uygulamalar sonucunda modellenmesi zor olan değişkenlerin varlığında klasik regresyon yöntemlerine alternatif olarak makine öğrenmesi regresyon yöntemlerinin kullanımı tavsiye edilmektedir. Sağlık harcaması modellenmesi zor olan bir değişken olup, literatürde makine öğrenmesi regresyon yöntemleri karşılaştırılarak bu değişkenin modellendiği bir çalışmaya rastlanmamıştır. Bu çalışmada kişi başı sağlık harcamasının tahmini amacıyla bir çoklu regresyon modeli oluşturulmuştur. Farklı hiperparametre değerleri belirlendiğinde elde edilen Lasso Regresyon, Rastgele Ağaç Regresyonu ile Destek Vektör Makinesi Regresyon performans sonuçları karşılaştırılmıştır. Çalışmada hiperparametre değeri olarak Lasso Regresyon için lamda (λ) değeri, Rastgele Ağaç Regresyonu için ağaç sayısı, Destek Vektör Regresyonu için epsilon () değeri esas alınmıştır. Sonuçlar 5 ile 50 arasında değişen “k” parça çapraz geçerlilik uygulanarak performe edildiğinde makine öğrenmesi regresyon yöntemlerine ait performans sonuçlarının R², RMSE ve MAE değerleri bakımından istatistiksel olarak anlamlı farklılıklar gösterdiği (p<0.001) tespit edilmiştir. Tahmin performanslarına ait yüzey ve çubuk grafikleri ile istatistiksel test sonuçları incelendiğinde farklı hiperparametre değerlerine göre Rastgele Ağaç Regresyonun (R² ˃ 0.7500, RMSE ≤ 0.6000 ve MAE ≤ 0.4000) daha iyi tahmin sonuçlarına sahip olduğu belirlenmiştir. Çalışma sonuçlarının, sağlık harcamasının modellendiği araştırmalar için makine öğrenmesi regresyon yöntemleri kullanıldığında en uygun hiperparametre değerlerinin belirlenmesi konusunda katkı sağlaması beklenmektedir. 

Keywords

• Makine Öğrenmesi,Lasso Regresyon,Rastgele Ağaç Regresyonu,Destek Vektör Regresyonu,Sağlık Harcaması

References

1. Alpar R. (2011) Uygulamalı çok değişkenli istatistiksel yöntemler, Detay Yayıncılık, Ankara, 415-620.
2. Basu, A., Manning, W.G. ve Mullahy, J. (2004). Comparing alternative model: log and cox proportional hazard? Health Economics, 13(8), 749-765. doi: 10.1002/hec.852.
3. Belloni, A., Chernozhukov, V., Hansen, C. (2012) Inference for high-dimensional sparse econometric models. https://arxiv.org/abs/1201.0220. doi: 10.1017/CBO9781139060035.008. Erişim Tarihi: 01.01.2016.
4. Bergstra, J. ve Bengio, Y. (2012) Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13, 281-305. http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf. Erişim Tarihi: 01.02.2016.
5. Box, G.E.P. ve Cox, D.R. (1964) An analysis of transformations, Journal of the Royal Statistical Society, 26(2), 211-252. doi: 10.1.1.321.3819.
6. Brieman, L. (2001) Random forests, Machine Learning, 45, 5-32. doi: 10.1023%2FA%3A1010933404324.
7. Cherkassky, V. ve Ma, Y. (2004) Practical selection of SVM parameters and noise estimation for SVM regression, Neural Networks, 17(1), 113-126. doi:10.1016/S0893-6080(03)00169-2.
8. Cosgun E., Karaağaoğlu E. (2011). Veri madenciliği yöntemleriyle mikrodizilim gen ifade analizi, Hacettepe Tıp Dergisi, 42, 180-189. http://docplayer.biz.tr/3432783-Veri-madencili-i-yontemleriyle-mikrodizilim-gen-ifade-analizi.html. Erişim Tarihi: 01.02.2016.

9. Collins, B. (2016) Big data and health economics: strengths, weaknesses, opportunities and threats, Pharmacoeconomics, 34(2), 101-106. doi: 10.1007/s40273-015-0306-7.
10. Cristianini, N. ve Shawe-Taylor, J. (2000). An introduction to support vector machines and other Kernel based learning methods, Cambridge University Press, UK, 93-122.
11. Crown, W.H. (2015) Potential application of machine learning in health outcomes research and statistical cautions, Value in Health, 18(2), 137-140. doi: 10.1016/j.jval.2014.12.005.
12. Duan, K., Keerthi, S.S., Poo, A.N. (2003) Evaluation of simple performance measures for tuning SVM hyperparameters, Neurocomputing, 51, 41-59. doi.org/10.1016/S0925-2312(02)00601-X.
13. Einav, L., Levin, J.D. (2014) The data revolution and economic analysis. NBER/Innovation Policy and the Economy, 14(1): 1-24. doi: 10.3386/w19035.
14. Elasan, S., Keskin, S., Arı E. (2016) İlişkili bileşen regresyonu: DNA hasarını belirleme modeli üzerinde uygulanması, Türkiye Klinikleri Biyoistatistik Dergisi, 8(1): 45-52. doi: 10.5336/biostatic.2015-48311.
15. Frank, I.E., Friedman J.H. (1993) A statistical view of some chemometrics regression tools. Technometrics, 35, 109-148. doi: 10.2307/1269656.
16. Frenk, J. (2010) The global health system: strengthening national health systems as the next step for global progress, PLOS Medicine, 7(1), 1-3. doi: 10.1371/journal.pmed.1000089.
17. Gislason, P.O., Benediktsson, J.A., Sveinsson, J.R. (2006) Random Forest for land cover classification. Pattern Recognition Letters, 27(4), 294-300. doi: 10.1016/j.patrec.2005.08.011.
18. Gupta, I., Mitra, A. (2004) Economic growth, health and poverty: an exploratory study for India, Development Policy Review, 22(2), 193-206. doi: 10.1111/j.1467-7679.2004.00245.x.
19. Hassan, S.S., Farhan, M., Mangayil, R., Huttunen, H., Aho, T. (2013) Bioprocess data mining using regularized regression and random forests, BMC System Biology, 7(1):1-7. doi: 10.1186/1752-0509-7-S1-S5.
20. Hastie, T., Tibshirani, R. ve Friedman, J. (2009) Random Forest. The elements of statistical learning data mining, Inference and Prediction. Springer Series in Statistics, 587-613.
21. Hawkins, D.M. (2004) The problem of overfitting, Journal of Chemical Information and Modeling, 44(1), 1-12. doi: 10.1021/ci0342472.
22. Jaggi, M. (2014) An equivalence between the lasso and support vector machines, https://arxiv.org/pdf/1303.1152.pdf, Erişim Tarihi: 16.5.2017. arXiv:1303.1152v2.
23. Jones, A.M., Rice, N., d’Uva, T.B. ve Balai, S. (2007) Applied health economics, Routledge, Taylor & Francis, London and New York, 280-319.
24. Kavaklıoğlu, K. (2011) Modeling and prediction of Turkey’s electricity consumption using support vector regression, Applied Energy, 88(1), 368-375. doi: 10.1016/j.apenergy.2010.07.021.
25. Kazem, A., Sharifi, E., Hussain, F.K., Saberi, M. ve Hussain, O.K. (2013) Support vector regression with chaos-based firefly algorithm for stock market price forecasting. Applied Soft Computing, 13(2), 947-958. doi: 10.1016/j.asoc.2012.09.024.
26. Kohavi, R. (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection, International Joint Conference on Artificial Intelligence (IJCAI’95), vol.2, 1137-1143.
27. Liaw, A., Wiener, M. (2002) Classification and regression by random forest, R News, vol.2/3, 18-22. http://www.bios.unc.edu/~dzeng/BIOS740/randomforest.pdf. Erişim Tarihi: 01.01.2016.
28. Manning, W. (2006) Dealing with skewed data on costs and expenditures, Jones A.M. (2006) The Elgar Companion to Health Economics, Second Edition, Edward Elgar Publishing, Inc. Massachusetts, USA, p.439-446.
29. Manning, W.G. (1998) The logged dependent variable, heteroscedasticity, and the retransformation problem, Journal of Health Economics, 17(3), 283-295. doi: 10.1016/S0167-6296(98)00025-3.
30. Martin, M.J.J., Gonzalez, M.P.L.A. ve Garcia, M.D.C. (2011) Review of the literature on the determinants of healthcare expenditure, Applied Economics, 43(1), 19-46. doi: 10.1080/00036841003689754.
31. Mattera, D. ve Haykin, S. (1999) Support vector machines for dynamic reconstruction of a chaotic system, Ed. Schöl B. Burges C.J.C. Smola A.J. (1999) Advances in Kernel Methods, Massachusetts Institute of Technology (MIT), 211-239.
32. Mihaylova, B., Briggs, A., O’Hagan, A. ve Thompson, S.G. (2011) Review of statistical methods for analysing healthcare resources and costs, Health Economics, 20(8), 897-916. doi: 10.1002/hec.1653.
33. Rodriguez, J.J., Diez-Pastor, J.F., Gonzalez A.A. ve Garcia-Osorio, C. (2015) An experimental study on combining binarization techniques and ensemble methods of decision trees, Multiple Classifier Systems 12th International Workshop, MCS 2015, Günzburg, Germany, June 29-July 1 2015 Proceedings, Springer.
34. Schölkopf, B., Smola, A.J. (2002) Learning with kernels. Support vector machines, regularization, optimization, and beyond, The MIT Press, Cambridge, Massachusetts, London, England.
35. Sinha, R.K., Chatterjee, K., Nair, N. ve Tripathy, P.K. (2016) Determinants of out-of-pocket and catastrophic health expenditure: a cross sectional study, British Journal of Medicine & Medical Research, 11(8), 1-11. doi : 10.9734/BJMMR/2016/21470.
36. Suthaharan, S. (2016) Support vector machine. Machine learning models and algorithms for big data classification, Integrated Series in Information Systems, vol.36, 207-235.
37. Tibshirani, R. (1996) Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, 58(1): 267-288. doi: 10.1111/j.1467-9868.2011.00771.x.
38. Tsamardinos, I., Rakhshani, A. ve Lagani, V. (2015). Performance-estimation properties of cross-validation-based protocols with simultaneous hyper-parameter optimization, International Journal of Artificial Intelligence Tools, 24(5), 1-
39. http://www.mensxmachina.org/wp-content/uploads/2014/03/SETN-2014-Model Selection.pdf. Erişim Tarihi: 01.02.2016.
40. Vapnik, V., Golowich, S.E. ve Smola, A. (1997). Support vector method for function approximation, regression estimation and signal processing, In M. Mozer, M. Jordan and T. Petshe, editors, Advances in Neural Information Processing Systems, 9. Cambridge MA. 1997. MIT Press. 281-287.
41. Wang, W. ve Xu, Z. (2004). A heuristic training for support vector regression, Neurocomputing, 61, 259-275. doi: 10.1016/j.neucom.2003.11.012.
42. WHO (World Health Organization) The World Health Report 2000: Improving health systems: improving performance, The World Health Organization.
43. Witten, I.H. ve Frank, E. (2005) Data mining practical machine learning tools and techniques, Second Edition, Morgan Kaufmann Publications, Elsevier, San Francisco, USA.
44. Yılmaz, E. (2016). Kardiotokogram verisinden fetal iyilik halinin belirlenmesi için bir karar destek sistemi, Uludag University Journal of The Faculty of Engineering, 21(2):331-340. doi: 10.17482/uumfd.278033.
45. Zheng, A. (2015) Evaluating machine learning models a beginner’s guide to key concepts and pitfalls, O’Reilly, USA.