Predicting credit card customer churn using support vector machine based on Bayesian optimization

Abstract

In this study, we have employed a hybrid machine learning algorithm to predict customer credit card churn. The proposed model is Support Vector Machine (SVM) with Bayesian Optimization (BO). BO is used to optimize the hyper-parameters of the SVM. Four different kernels are utilized. The hyper-parameters of the utilized kernels are calculated by the BO. The prediction power of the proposed models are compared by four different evaluation metrics. Used metrics are accuracy, precision, recall and F₁-score. According to each metrics linear kernel has the highest performance. It has accuracy of %91. The worst performance achieved by sigmoid kernel which has accuracy of %84.

Keywords

• Churn analysis
• support vector machine
• machine learning
• hyper-parameter optimization

References

1. Başoğlu Kabran, F., Ünlü K. D., A two-step machine learning approach to predict S&P 500 bubbles, Journal of Applied Statistics, (2020), 1-19. https://doi.org/10.1080/02664763.2020.1823947
2. Huang, W., Nakamori, Y., Wang, S. Y., Forecasting stock market movement direction with support vector machine, Computers & Operations Research, 32(10) (2005), 2513-2522. https://doi.org/10.1016/j.cor.2004.03.016
3. Tay, F. E., Cao, L., Application of support vector machines in financial time series forecasting, Omega, 29(4) (2001), 309-317. https://doi.org/10.1016/S0305-0483(01)00026-3
4. Xie, W., Yu, L., Xu, S., Wang, S., A new method for crude oil price forecasting based on support vector machines, International conference on computational science, (2006) 44-451. Springer, Berlin, Heidelberg.
5. Lu, W. Z., Wang, W. J., Potential assessment of the "support vector machine" method in forecasting ambient air pollutant trends, Chemosphere, 59(5) (2005), 693-701. https://doi.org/10.1016/j.chemosphere.2004.10.032
6. De Caigny, A., Coussement, K., De Bock, K. W., Lessmann, S., Incorporating textual information in customer churn prediction models based on a convolutional neural network, International Journal of Forecasting, 36(4) (2020), 1563-1578. https://doi.org/10.1016/j.ijforecast.2019.03.029
7. Qu, Z., Wang, W., Qu, N., Liu, Y., Lv, H., Hu, K., Song, J. A , Forecasting method of electricity sales considering the user churn rate in a power market environment, Journal of Electrical Engineering & Technology, 14(4) (2019), 1585-1596. https://doi.org/10.1007/s42835-019-00215-9
8. Nie, G., Rowe, W., Zhang, L., Tian, Y.,Shi, Y., Credit card churn forecasting by logistic regression and decision tree. Expert Systems with Applications, 38(12) (2011), 15273-15285. https://doi.org/10.1016/j.eswa.2011.06.028

9. Yu, X., Guo, S., Guo, J., Huang, X., An extended support vector machine forecasting framework for customer churn in e-commerce, Expert Systems with Applications, 38(3) (2011), 1425-1430. https://doi.org/10.1016/j.eswa.2010.07.049
10. Andrews, R., Zacharias, R., Antony, S.,James, M. M., Churn prediction in telecom sector using machine learning, International Journal of Information, 8(2) (2019), https://doi.org/10.30534/ijiscs/2019/31822019
11. Sabbeh, S. F., Machine-learning techniques for customer retention: A comparative study, International Journal of Advanced Computer Science and Applications, 9(2) (2018).
12. Pamina, J., Raja, B., SathyaBama, S., Sruthi, M. S., VJ, A., An effective classifier for predicting churn in telecommunication, Journal of Adv. Research in Dynamical & Control Systems, 11 (2019).
13. Vo, N. N., Liu, S., Li, X., Xu, G. . Leveraging unstructured call log data for customer churn prediction, Knowledge-Based Systems, 212 (2021), 106586, https://doi.org/10.1016/j.knosys.2020.106586
14. Lalwani, P., Mishra, M. K., Chadha, J. S., Sethi, P., Customer churn prediction system: a machine learning approach, Computing, (2021), 1-24, https://doi.org/10.1007/s00607-021- 00908-y
15. Zhuang, Y., Research on E-commerce customer churn prediction based on improved value model and XG-boost algorithm, Management Science and Engineering, 12(3) (2018), 51-56.
16. Cortes, C., Vapnik, V., Support-vector networks. Machine learning, 20(3) (1995), 273-297.
17. Snoek, J., Larochelle, H., Adams, R. P., Practical Bayesian optimization of machine learning algorithms. arXiv preprint, (2012), arXiv:1206.2944.
18. Injadat, M., Salo, F., Nassif, A. B., Essex, A., Shami, A., Bayesian optimization with machine learning algorithms towards anomaly detection. IEEE global communications conference, (2018), 1-6.
19. Yang, L., Shami, A. On hyperparameter optimization of machine learning algorithms: Theory and practice, Neurocomputing, 415 (2020), 295-316. https://doi.org/10.1016/j.neucom.2020.07.061
20. Kaggle https://www.kaggle.com/sakshigoyal7/credit-card-customers, Acceded:January 3, 2021.