The Effectiveness of Homogeneous Classifier Ensembles on Customer Churn Prediction in Banking, Insurance, and Telecommunication Sectors

Abstract

The prediction of customer churn is a big challenging problem for companies in different sectors such as banking, telecommunication, and insurance. It is a crucial estimation for many businesses since obtaining new customers frequently costs more than holding present ones. For this reason, analysts and researchers are focus on to investigate reasons behind of customer churn analyzing behaviors of them. In this paper, an ensemble-based framework is proposed to predict the customer churn in various sectors, namely banking, insurance, and telecommunication. To demonstrate the effectiveness of proposed ensemble framework, k-NN, logistic regression, naïve Bayes, support vector machine, decision tree, random forest, multilayer perceptron algorithms are employed. Moreover, the effects of the inclusion of feature extraction process are investigated. Experiment results indicate that that random forest algorithm is capable to predict churn customers with 89.93% of accuracy in banking, 95.90% of accuracy in telecommunication, and 77.53% of accuracy in insurance sectors when feature extraction procedure is carried out.

Keywords

• Churn Prediction
• ensemble learning
• machine learning
• random forest
• customer loyalty

References

1. Babu, S., Ananthanarayanan, D. N., Ramesh, V. (2014). A Survey on Factors Impacting Churn in Telecommunication using Data Mining Techniques. Int. J. Eng. Res. Technol., 3, 1745-1748.
2. Ullah, I., Raza, B., Malik, A. K., Imran, M., Islam, S. U., Kim, S. W. (2019). A Churn Prediction Model using Random Forest: Analysis of Machine Learning Techniques for Churn Prediction and Factor Identification in Telecom Sector. IEEE Access, 7, 60134-60149.
3. Idris, A., Khan, A. (2012, December). Customer churn prediction for telecommunication: Employing various features selection techniques and tree based ensemble classifiers. In International Multitopic Conference (pp. 23-27).
4. Kaur, M., Singh, K., Sharma, N. (2013). Data mining as a tool to predict the churn behaviour among Indian bank customers. Int. J. Recent Innov. Trends Comput. Commun., 1(9), 720-725.
5. Verbeke, W., Martens, D., Mues, C., Baesens, B. (2011). Building comprehensible customer churn prediction models with advanced rule induction techniques. Expert Syst. Appl., 38(3), 2354-2364.
6. Rokach, L. (2010). Ensemble-based classifiers. Artificial Intelligence Review, 33(1-2), 1–39.
7. Polikar, R. (2006). Ensemble based systems in decision making. IEEE Circuits and Systems Magazine, 6(3), 21-45.
8. Gopika, D., Azhagusundari, B. (2014). An analysis on ensemble methods in classification tasks. International Journal of Advanced Research in Computer and Communication Engineering, 3(7), 7423–7427.

9. Ren, Y., Zhang, L., Suganthan, P., N. (2016. Ensemble Classification and Regression-Recent Developments, Applications and Future Directions. IEEE Computational Intelligence Magazine, 11(1), 41-53.
10. Kilimci, Z. H., Akyokus, S., Omurca, S. I. (2016, August). The effectiveness of homogenous ensemble classifiers for Turkish and English texts. In IEEE International Symposium on INnovations in Intelligent SysTems and Applications (pp. 1-7).
11. Ahmad, A., K., Jafar, A., Aljoumaa, K. (2019). Customer churn prediction in telecom using machine learning in big data platform. J Big Data, 6(28), 1-24.
12. Ullah, I., Raza, B., Malik, A., K., Imran, M, Islam, S., U., Kim, S., W. (2019). A churn prediction model using random forest: Analysis of machine learning techniques for churn prediction and factor identification in telecom sector. IEEE Access, 7, 60134-60149.
13. Lalwani, P., Mishra, M., K., Chadha, J., S., Sethi, P. (2022). Customer churn prediction system: a machine learning approach. Computing, 104, 271–294.
14. Pamina, J., Beschi Raja, J., Sam Peter, S., Soundarya, S., Sathya Bama, S., Sruthi, M.S. (2020). Inferring Machine Learning Based Parameter Estimation for Telecom Churn Prediction. In: Smys, S., Tavares, J., Balas, V., Iliyasu, A. (eds) Computational Vision and Bio-Inspired Computing. ICCVBIC 2019. Advances in Intelligent Systems and Computing, vol 1108. Springer.
15. Rahman, M., Kumar, V. (2020, November). Machine learning based customer churn prediction in banking. In IEEE International Conference on Electronics, Communication and Aerospace Technology (pp. 1196-1201).
16. Jain, H., Yadav, G., Manoov, R. (2021). Churn Prediction and Retention in Banking, Telecom, and IT Sectors Using Machine Learning Techniques. In: Patnaik, S., Yang, XS., Sethi, I. (eds) Advances in Machine Learning and Computational Intelligence. Algorithms for Intelligent Systems. Springer, Singapore.
17. Dias, J., Godinho, P., Torres, P. (2020). Machine Learning for Customer Churn Prediction in Retail Banking. In:, et al. Computational Science and Its Applications – ICCSA 2020. ICCSA 2020. Lecture Notes in Computer Science, vol 12251. Springer, Cham.
18. Xanthopoulos, P., Pardalos, P.M., Trafalis, T.B. (2013). Linear discriminant analysis. In: Robust Data Mining. SpringerBriefs in Optimization. Springer, New York, NY.
19. Tolles, J., Meurer, W., J. (2016). Logistic regression relating patient characteristics to outcomes. JAMA. 316 (5), 533–534.
20. Tsigkritis, T., Groumas, G., Schneider, M. (2018). On the use of k-NN in anomaly detection. Journal of Information Security, 9, 70-84.
21. Martín-Valdivia, M., T., Rushdi, Saleh, M, Ureña-López, L., A., MontejoRáez, A. (2011). Experiments with SVM to classify opinions in different domains. Expert Systems with Applications, 38(12), 14799-14804.
22. Ren, J., Lee, S., D., Chen, X., Kao, B., Cheng, R., Cheung, D. (2009, December). Naive Bayes classification of uncertain data. In: IEEE International Conference on Data Mining (pp. 944 –949).
23. Horn, C. (2010). Analysis and classification of Twitter messages (Master's thesis). Graz University of Technology, Graz, Austria.
24. Rish, I. (2001, August). An empirical study of the naive Bayes classifier. In: IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence (pp. 41-46).
25. Jiang, L., Zhang, H., Cai, Z. (2008). A novel Bayes model: Hidden naive Bayes. IEEE Transactions on knowledge and data engineering, 21(10), 1361-1371.
26. Frank, E., Trigg, L., Holmes, G., Witten, I. H. (2000). Naive Bayes for regression. Machine Learning, 41(1), 5-25.
27. Lewis, D. D. (1998, April). Naive (Bayes) at forty: The independence assumption in information retrieval. In European conference on machine learning (pp. 4-15).
28. Kilimci, Z. H., Ganiz, M. C. (2015, September). Evaluation of classification models for language processing. In: 2015 IEEE International Symposium on Innovations in Intelligent SysTems and Applications (pp. 1-8).
29. Kilimci, Z. H., Akyokuş, S. (2016, May). N-gram pattern recognition using multivariate-Bernoulli model with smoothing methods for text classification. In: 2016 24th IEEE Signal Processing and Communication Application Conference (pp. 597-600).
30. Cortes, C., Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297.
31. Quinlan, J. R. (1986). Induction of decision trees. Machine learning, 1(1), 81-106.
32. Ho, T. K. (1995, August). Random decision forests. In Proceedings of 3rd international conference on document analysis and recognition (Vol. 1, pp. 278-282).
33. Murtagh, F. (1991). Multilayer perceptrons for classification and regression. Neurocomputing, 2(5-6), 183-197.