Classification of Customer Complaints using Convolutional Neural Network Based Deep Learning Method

Abstract

Nowadays, the number of companies is increasing, and companies are growing in line with the increasing population and changing needs. In this context, many companies operating in the same field emerge, thus companies need to enhance their competitive abilities. For a company, focusing on retaining existing customers is more cost-effective than acquiring new customers. One of the most critical elements in not losing a customer is the effective management of customer complaints, which is a sub-branch of customer relationship management. With the advancements in technology, automated systems are frequently used in customer complaint management, as in many areas, but it has not yet reached the desired levels. In this study, novel models using deep learning were developed to contribute to the field of customer complaint management. In this context, a model was created to predict which complaint type customer comments concern, using a convolutional neural network layer. The models were analyzed using a dataset in the field of finance, and the hyper-parameters of the models were optimized using the Bayesian optimization method. Accuracy of up to 85.83% were achieved with models developed at different depths. When compared to studies with similar datasets in the literature, it was observed that the proposed model outperformed other studies.

Keywords

• Customer Complaint Management
• Deep Learning
• Convolutional Neural Network
• Consumer Behavior

Evrişimsel Sinir Ağları Tabanlı Derin Öğrenme Yöntemiyle Müşteri Şikayetlerinin Sınıflandırılması

Öz

Günümüzde, artan nüfus ve değişen ihtiyaçlar doğrultusunda firma sayıları giderek artmakta ve firmalar büyümektedir. Bu bağlamda, aynı alanda faaliyet gösteren birçok firma ortaya çıkmakta, bu nedenle firmaların rekabet kabiliyetini artırması gerekmektedir. Bir firma için mevcut müşterinin elde tutulmasına odaklanmak, yeni müşteri kazanmaktan daha maliyetli olmaktadır. Bir müşterinin kaybedilmemesi için en önemli unsurlardan birisi müşteri ilişkileri yönetiminin bir alt dalı olan müşteri şikâyetlerinin iyi bir şekilde yönetilmesinden geçmektedir. Teknolojide meydana gelen gelişmeler doğrultusunda, birçok alanda olduğu gibi müşteri şikâyeti yönetiminde de teknolojiden sıklıkla faydalanılmaktadır ancak bu durum henüz istenilen seviyelere ulaşmamıştır. Bu çalışmada müşteri şikâyeti yönetimi alanına katkı sağlamak için derin öğrenmeden faydalanan özgün modeller geliştirilmiştir. Bu kapsamda, evrişimsel sinir ağı katmanı kullanılarak müşteri yorumlarının hangi şikâyet türünü ilgilendirdiğini tahmin eden bir model geliştirilmiştir. Finans alanındaki bir veri seti kullanılarak analiz edilen modelin hiper-parametreleri Bayesian optimizasyon yöntemi kullanılarak optimize edilmiştir. Farklı derinliklerde geliştirilen modellerle %85.83’lere ulaşan doğruluk oranı elde edilmiştir. Literatürde benzer veri seti ile yapılan çalışmalar incelendiğinde önerilen modelin, diğer çalışmalara göre üstün olduğu gözlemlenmiştir.

Anahtar Kelimeler

• Müşteri Şikâyet Yönetimi
• Derin Öğrenme
• Evrişimsel Sinir Ağları
• Tüketici Davranışları

Kaynakça

1. Alamsyah, D. P., Arifin, T., Ramdhani, Y., Hidayat, F. A., & Susanti, L. (2022). Classification of customer complaints: TF-IDF approaches. 2022 2nd International Conference on Intelligent Technologies (CONIT), 1–5. https://doi.org/10.1109/CONIT55038.2022.9848056
2. Aldunate, Á., Maldonado, S., Vairetti, C., & Armelini, G. (2022). Understanding customer satisfaction via deep learning and natural language processing. Expert Systems with Applications, 209, 118309. https://doi.org/10.1016/j.eswa.2022.118309
3. Ali Hakami, N., & Hosni Mahmoud, H. A. (2022). Deep learning analysis for reviews in Arabic e-commerce sites to detect consumer behavior towards sustainability. Sustainability, 14(19), 12860.
4. Anagun, Y., Bolel, N. S., Isik, S., & Ozkan, S. E. (2022). Deep learning-based customer complaint management. Journal of Organizational Computing and Electronic Commerce, 32(3–4), 217–231. https://doi.org/10.1080/10919392.2023.2210049
5. Cho, Y., Im, I., Hiltz, R., & Fjermestad, J. (2002). An analysis of online customer complaints: Implications for web complaint management. Proceedings of the 35th Annual Hawaii International Conference on System Sciences, 2308–2317. https://doi.org/10.1109/HICSS.2002.994162
6. Correa, N., & Correa, A. (2022). Neural text classification for digital transformation in the financial regulatory domain. 2022 IEEE ANDESCON, 1–6. https://doi.org/10.1109/ANDESCON56260.2022.9989638
7. Demirel, G. K., & Şen, A. (2023). Makine öğrenmesi tekniklerinin bütçe verimliliğine uygulanması üzerine bir çalışma. İşletme Araştırmaları Dergisi, 15(2), 953-969.
8. DiCarlo, M., Berglund, E. Z., Kaza, N., Grieshop, A., Shealy, L., & Behr, A. (2023). Customer complaint management and smart technology adoption by community water systems. Utilities Policy, 80, 101465. https://doi.org/10.1016/j.jup.2022.101465

9. Erkayman, B., Erdem, E., Aydin, T., & Mahmat, Z. (2023). New artificial intelligence approaches for brand switching decisions. Alexandria Engineering Journal, 63, 625-643. https://doi.org/10.1016/j.aej.2022.11.043
10. Ferri, E. (2018). The evolving practice of complaint management. Bloomberg Law, 1-8.
11. Financial Ombudsman Service (2023). Annual complaints data and insight 2022/23. https://www.financial-ombudsman.org.uk/data-insight/annual-complaints-data/annual-complaints-data-insight-202223 Erişim Tarihi: 17.08.2023.
12. Ghosal, I., & Prasad, B. (2023). Transforming consumer behavior to new paradigms through deep learning applications. International Journal of Advances in Business and Management Research (IJABMR), 1(1), 26-29.
13. Gormez, Y., Aydin, Z., Karademir, R., & Gungor, V. C. (2020). A deep learning approach with Bayesian optimization and ensemble classifiers for detecting denial of service attacks. International Journal of Communication Systems, 33(11), e4401. https://doi.org/10.1002/dac.4401
14. Harrison, R., Walton, M., Healy, J., Smith-Merry, J., & Hobbs, C. (2016). Patient complaints about hospital services: Applying a complaint taxonomy to analyse and respond to complaints. International Journal for Quality in Health Care, 28(2), 240–245. https://doi.org/10.1093/intqhc/mzw003
15. Hayuningrum, V. (2021). Customer complaints auto-categorization: performance comparison of recurrent and convolutional neural networks, Master’s Thesis in Data Science & Society, Tilburg University.
16. İlkuçar, M., & Artun, C. (2023). Misafir yorumlarının makine öğrenmesi yardımıyla duygu analizi: Fethiye beş yıldızlı oteller örneği. Journal of Business in the Digital Age, 6(1), 33-41.
17. Jain, P. K., Saravanan, V., & Pamula, R. (2021). A hybrid cnn-lstm: a deep learning approach for consumer sentiment analysis using qualitative user-generated contents. ACM Transactions on Asian and Low-Resource Language Information Processing, 20(5), 84:1-84:15. https://doi.org/10.1145/3457206
18. Janiesch, C., Zschech, P., & Heinrich, K. (2021). Machine learning and deep learning. Electronic Markets, 31(3), 685–695. https://doi.org/10.1007/s12525-021-00475-2
19. Karataş, A. F., Mercan, Ö. B., Özdil, U., & Ozan, Ş. (2023). Çağrı merkezlerinde olumsuzluk içeren çağrıların evrişimsel sinir ağları ile tespiti. Bilişim Teknolojileri Dergisi, 16(1), 13-19. https://doi.org/10.17671/gazibtd.1156330
20. Kaynar, O., Tuna, M. F., Görmez, Y., & Deveci̇, M. A. (2017). Makine öğrenmesi yöntemleriyle müşteri kaybı analizi. Cumhuriyet Üniversitesi İktisadi ve İdari Bilimler Dergisi, 18(1), 1-14.
21. Keramati, A., Ghaneei, H., & Mirmohammadi, S. M. (2016). Developing a prediction model for customer churn from electronic banking services using data mining. Financial Innovation, 2(1), 10. https://doi.org/10.1186/s40854-016-0029-6
22. Keras. (2023). Keras: Deep learning for humans. https://keras.io/ Erişim Tarihi: 25.07.2023.
23. Khedkar, S., & Shinde, S. (2020a). Deep learning and ensemble approach for praise or complaint classification. Procedia Computer Science, 167, 449–458. https://doi.org/10.1016/j.procs.2020.03.254
24. Khedkar, S., & Shinde, S. (2020b). Deep learning-based approach to classify praises or complaints from customer reviews. In S. Bhalla, P. Kwan, M. Bedekar, R. Phalnikar, & S. Sirsikar (Eds.), Proceeding of International Conference on Computational Science and Applications (pp. 391–402). Springer. https://doi.org/10.1007/978-981-15-0790-8_38
25. Kohler, M., Sondermann, L., Forero, L., & Pacheco, M. A. (2020). Classifying and grouping narratives with convolutional neural networks, PCA and t-SNE. In A. M. Madureira, A. Abraham, N. Gandhi, & M. L. Varela (Eds.), Hybrid Intelligent Systems (pp. 22–30). Springer International Publishing. https://doi.org/10.1007/978-3-030-14347-3_3
26. Lang, T., & Rettenmeier, M. (2017, April). Understanding consumer behavior with recurrent neural networks. In Workshop on Machine Learning Methods for Recommender Systems.
27. Meyer-Waarden, L., & Sabadie, W. (2023). Relationship quality matters: How restaurant businesses can optimize complaint management. Tourism Management, 96, 104709. https://doi.org/10.1016/j.tourman.2022.104709
28. Oyewola, D. O., Omotehinwa, T. O., & Dada, E. G. (2023). Consumer complaints of consumer financial protection bureau via two-stage residual one-dimensional convolutional neural network (TSR1DCNN). Data and Information Management, 100046. https://doi.org/10.1016/j.dim.2023.100046
29. Peker, S. (2022). Predicting firms’ performances in customer complaint management using machine learning techniques. In C. Kahraman, A. C. Tolga, S. Cevik Onar, S. Cebi, B. Oztaysi, & I. U. Sari (Eds.), Intelligent and Fuzzy Systems (pp. 280–287). Springer International Publishing. https://doi.org/10.1007/978-3-031-09176-6_33
30. Qianyu, Z., Dongping, L., & Xiaozhou, Z. (2021, June). Research on financial consumer behavior based on deep Learning. In 2021 International Conference on Big Data Analysis and Computer Science (BDACS) (pp. 179-182).
31. Salama, A., Hassanien, A. E., & Fahmy, A. (2019). Sheep ıdentification using a hybrid deep learning and bayesian optimization approach. IEEE Access, 7, 31681–31687. https://doi.org/10.1109/ACCESS.2019.2902724
32. Salminen, J., Mustak, M., Corporan, J., Jung, S., & Jansen, B. J. (2022). Detecting pain points from user-generated social media posts using machine learning. Journal of Interactive Marketing, 57(3), 517–539. https://doi.org/10.1177/10949968221095556
33. Sci-Kit Optimize (2023). Scikit-optimize: Sequential model-based optimization toolbox. https://scikit-optimize.github.io/ Erişim Tarihi: 27.07.2023.
34. Sezgin, M., & Duman, A. (2023). Elektronik ağızdan ağıza pazarlama kapsamında konaklama işletmelerine yönelik çevrimiçi yorumların duygu analizi yöntemiyle incelenmesi: Alanya örneği. Türk Turizm Araştırmaları Dergisi, 7(2), 244-265. https://doi.org/10.26677/TR1010.2023.1240
35. Seymen, O. F., Ölmez, E., Doğan, O., Er, O., & Hiziroğlu, K. (2023). Customer churn prediction using ordinary artificial neural network and convolutional neural network algorithms: A comparative performance assessment. Gazi University Journal of Science, 36(2), Article 2. https://doi.org/10.35378/gujs.992738
36. Shin, J., Son, S., & Cha, Y. (2022). Spatial distribution modeling of customer complaints using machine learning for indoor water leakage management. Sustainable Cities and Society, 87, 104255. https://doi.org/10.1016/j.scs.2022.104255
37. Shivaprasad, V. (2020). Analysis of customer complaint data of consumer financial protection bureau using different text mining techniques (Doctoral dissertation, Dublin Business School).
38. Shobana, G., Sanjay, S. S., Saran, V., & Vardan, G. K. (2022). Consumer grievance handler. 2022 IEEE 3rd Global Conference for Advancement in Technology (GCAT), 1–5. https://doi.org/10.1109/GCAT55367.2022.9971905
39. Singh, A., Saha, S., Hasanuzzaman, Md., & Dey, K. (2022). Multitask learning for complaint ıdentification and sentiment analysis. Cognitive Computation, 14(1), 212–227. https://doi.org/10.1007/s12559-021-09844-7
40. Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 25. https://proceedings.neurips.cc/paper/2012/hash/05311655a15b75fab86956663e1819cd-Abstract.html Statista Research Department (2023). Company responses to consumer complaints to the Consumer Financial Protection Bureau (CFPB) in the United States in 2019, by response type https://www.statista.com/statistics/1105735/company-response-consumer-complaint-cfpb-usa/ Erişim Tarihi: 17.08.2023.
41. Strasser, T. (2023). Don’t trust the machine? Der fremdsprachliche unterricht englisch, 2023(184), 20–27.
42. Sun, L., Yan, H., Xin, K., & Tao, T. (2019). Contamination source identification in water distribution networks using convolutional neural network. Environmental Science and Pollution Research, 26(36), 36786–36797. https://doi.org/10.1007/s11356-019-06755-x
43. ŞikayetVar (2023). Tüm şikâyetler https://www.sikayetvar.com/sikayetler Erişim Tarihi: 17.08.2023.
44. Tahsin, M. U., Shanto, M. S. H., & Rahman, R. M. (2023). Combining natural language processing and federated learning for consumer complaint analysis: A case study on laptops. SN Computer Science, 4(5), 537. https://doi.org/10.1007/s42979-023-01989-6
45. Tiwari, S. (2021). Consumer complaints dataset for NLP. Consumer complaints dataset for NLP-Kaggle. https://www.kaggle.com/datasets/shashwatwork/consume-complaints-dataset-fo-nlp, Erişim Tarihi: 25.07.2023.
46. Tuna, M. F., Akdoğan, Ş., & Kaynar, O. (2021). Otellere ilişkin yorum dışı müşteri geri bildirimlerinin analizi. Cumhuriyet Üniversitesi İktisadi ve İdari Bilimler Dergisi, 22(2), 50-81. https://doi.org/10.37880/cumuiibf.869489
47. Vairetti, C., Aránguiz, I., Maldonado, S., Karmy, J. P., & Leal, A. (2024). Analytics-driven complaint prioritisation via deep learning and multicriteria decision-making. European Journal of Operational Research, 312(3), 1108-1118.
48. Yang, W., Tan, L., Lu, C., Cui, A., Li, H., Chen, X., Xiong, K., Wang, M., Li, M., Pei, J., & Lin, J. (2019). Detecting customer complaint escalation with recurrent neural networks and manually-engineered features. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers), 56–63. https://doi.org/10.18653/v1/N19-2008