Classification of Customer Complaints Using BERTopic Topic Modelling Technique

Abstract

The analysis of customer complaints enables companies to amend mistakes, protect brand value and attract new customers. Utilizing machine learning techniques for data classification and prediction provide decision makers with time and cost benefits, particularly with increased complaint data size. Therefore, this study employed BERTopic topic modelling technique, a contemporary approach, to examine customer complaint distribution with respect to distinct topics, by product and in general. In the study, the complaints submitted to a consumer electronics retailer in 2020 were adopted and classified. The monthly variation of the complaints was also investigated with dynamic topic modelling. The results showed that the complaints concentrated more heavily on shipping, television, mobile phone, laptop, earphones, tablets, store clerks and order cancellation topics.

Keywords

• Customer Complaints
• Topic Modelling
• BERTopic
• Dynamic Topic Modelling

BERTopic Konu Modelleme Tekniği Kullanılarak Müşteri Şikayetlerinin Sınıflandırılması

Abstract

Müşteri şikâyetlerinin analizi işletmeler açısından geçmişte yaptıkları hataları düzeltme, marka değerini koruma ve yeni müşteriler edinmeleri açısından önemli bir kavramdır. Özellikle şikâyet verisinin büyüklüğü arttıkça verinin sınıflandırılması ve tahminlenmesi için makine öğrenmesi tekniklerinden yararlanmak zaman ve maliyet açısından karar vericilere avantaj sağlamaktadır. Bu yüzden çalışmada, müşteri şikayetlerinin ürün bazında ve genel anlamda hangi farklı konularda dağılım gösterdiğinin bulunması amacıyla güncel bir yaklaşım olan BERTopic konu modelleme tekniğinden yararlanılmıştır. Buna yönelik olarak da veri seti olarak 2020 yılına ait bir tüketici elektroniği perakende şirketine yapılan şikayetler kullanılmış ve sınıflandırılmıştır. Bunun yanında, şikayetlerin aylık olarak zaman içindeki değişimi de dinamik konu modelleme kullanılarak incelenmiştir. Sonuçlara göre en fazla şikâyet kargolama, televizyon, cep telefonu, dizüstü bilgisayar, kulaklık, tablet, mağaza çalışanları, sipariş iptali konularında yoğunlaşmıştır.

Keywords

• Müşteri Şikayetleri
• Konu Modelleme
• BERTopic
• Dinamik Konu Modelleme

References

1. Abuzayed, A., & Al-Khalifa, H. (2021). BERT for Arabic topic modeling: An experimental study on BERTopic technique. Procedia Computer Science, 189, 191–194. https://doi.org/10.1016/j.procs.2021.05.096
2. Akbıyık, A., & Arı, O. (2022). Lojistik regresyon ile faydalı müşteri yorumlarını tahminlime. Journal of Research in Business, 7, IMISC 2021 Special Issue, 15–32. https://doi.org/10.54452/jrb.1024602
3. Alabay, M. N. (2012). Müşteri şikâyetleri yönetimi. Uluslararası Yönetim İktisat ve İşletme Dergisi, 8(16), 137–157.
4. Alhaj, F., Al-Haj, A., Sharieh, A., & Jabri, R. (2022). Improving Arabic cognitive distortion classification in Twitter using BERTopic. International Journal of Advanced Computer Science and Applications, 13(1), 854–860. https://doi.org/10.14569/ijacsa.2022.0130199
5. Altıntaş, V., Albayrak, M., & Topal, K. (2021). Kanser hastalığı ile ilgili paylaşımlar için Dirichlet ayrımı ile gizli konu modelleme. Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi, 36(4), 2183–2196 https://doi.org/10.17341/gazimmfd.734730
6. Angelov, D. (2020). Top2Vec: Distributed representations of topics. arXiv. https://doi.org/10.48550/ARXIV.2008.09470
7. Aşkun, O. (2015). Şikâyet iletilerinin örgütsel öğrenme üzerine etkisi. Marmara Üniversitesi İktisadi ve İdari Bilimler Dergisi, 24(1), 221–243.
8. Baird, A., Xia, Y., & Cheng, Y. (2022). Consumer perceptions of telehealth for mental health or substance abuse: A Twitter-based topic modeling analysis. JAMIA Open, 5(2), 1–8. https://doi.org/10.1093/jamiaopen/ooac028

9. Bal, V. (2014). Online satış girişimcilerinin karşılaştıkları müşteri şikâyetlerinin analizi. AİBÜ Sosyal Bilimler Enstitüsü Dergisi, 14(1), 59–74.
10. Bayrak, A. T., Türker, B. B., Yıldız, E., & Özbek, E. E. (2021). Complaint detection and classification of customer reviews. SIU, 29th Signal Processing and Communications Applications Conference, 9-11 June 2021, İstanbul, Türkiye, pp. 1–4. https://doi.org/10.1109/siu53274.2021.9478016
11. Bayram, U. (2022). Revealing the reflections of the pandemic by investigating COVID-19 related news articles using machine learning and network analysis. Bilişim Teknolojileri Dergisi, 15(2), 209–220. https://doi.org/10.17671/gazibtd.949599
12. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.
13. Bozyiğit, F., Doğan, O., & Kılınç, D. (2022). Categorization of customer complaints in food industry using machine learning approaches. Journal of Intelligent Systems: Theory and Applications, 5(1), 85–91. https://doi.org/10.38016/jista.954098
14. Campello, R. J., Moulavi, D. & Sander, J. (2013). Density-based clustering based on hierarchical density estimates. PAKDD 2013, 17th Pacific-Asia Conference, April 2013, Gold Coast, Australia, pp. 160–172.
15. Chong, M., & Chen, H. (2021). Racist framing through stigmatized naming: A topical and geo‐locational analysis of #chinavirus and #chinesevirus on Twitter. 84th Annual Meeting of the Association for Information Science & Technology, 29 October – 3 November 2021, Salt Lake City, USA, pp. 70–79. https://doi.org/10.1002/pra2.437
16. Çağlar Çetinkaya, N. (2020). Hizmet kalitelerine ilişkin müşteri şikayetleri: Bir içerik analizi. Selçuk 2. Uluslararası Sosyal Bilimler Kongresi, 7 Haziran 2020, Konya, pp. 39–54.
17. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391–407. https://doi.org/10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9
18. Demirel, Y. (2017). Müşteri ilişkileri yönetimi: Teori, uygulama, ölçüm, 3. Baskı. Ankara: Seçkin Yayıncılık.
19. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding (Version 2). arXiv. https://doi.org/10.48550/ARXIV.1810.04805
20. Du, T., Umar, P., Rajtmajer, S., & Squicciarini, A. (2022). The contribution of verified accounts to self-disclosure in COVID-related Twitter conversations. Sixteenth International AAAI Conference on Web and Social Media, 6-9 June 2022, Atlanta, Georgia, USA, pp. 1393-1397.
21. Dwivedi, M., Shibu, T. P., & Venkatesh, U. (2007). Social software practices on the internet. International Journal of Contemporary Hospitality Management, 19(5), 415–426. https://doi.org/10.1108/09596110710757570
22. Ebeling, R., Córdova Sáenz, C. A., Nobre, J., & Becker, K. (2021). The effect of political polarization on social distance stances in the Brazilian COVID-19 scenario. Journal of Information and Data Management, 12(1). https://doi.org/10.5753/jidm.2021.1889
23. Egger, R., & Yu, J. (2022). A topic modeling comparison between LDA, NMF, Top2Vec, and BERTopic to demystify Twitter posts. Frontiers in Sociology, 7, 886498. https://doi.org/10.3389/fsoc.2022.886498
24. Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Schütze, H., & Goldberg, Y. (2021). Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9, 1012–1031. https://doi.org/10.1162/tacl_a_00410
25. Faed, A. (2010). Handling e-complaints in customer complaint management system using FMEA as a qualitative system, IMS 2010, 6th International Conference on Advanced Information Management and Service, 30 November - 02 December 2010, Seoul, Korea, pp. 205–209.
26. Filieri, R., Lin, Z., Li, Y., Lu, X., & Yang, X. (2022). Customer emotions in service robot encounters: A hybrid machine-human intelligence approach. Journal of Service Research, 109467052211039. https://doi.org/10.1177/10946705221103937
27. Galitsky, B. (2020). Artificial intelligence for customer relationship management, Springer International Publishing, Cham. https://doi.org/10.1007/978-3-030-52167-7
28. Garding, S., & Bruns, A. (2015). Complaint management and channel choice, Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-18179-0
29. Ghazzawi, A., & Alharbi, B. (2019). Analysis of customer complaints data using data mining techniques. Procedia Computer Science, 163, 62–69. https://doi.org/10.1016/j.procs.2019.12.087
30. Goodman, J. (1992). Leveraging the customer database to your competitive advantage, Journal of Direct Marketing, 55, 26–27.
31. Greedharry, M., Seewoogobin, V., & Gooda Sahib-Kaudeer, N. (2019). A smart mobile application for complaints in Mauritius. Advances in Intelligent Systems and Computing, Springer Singapore, Singapore. https://doi.org/10.1007/978-981-13-3338-5_32
32. Grootendorst, M. (2021). BERTopic. https://maartengr.github.io/BERTopic/api/bertopic.html
33. Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv. https://doi.org/10.48550/ARXIV.2203.05794
34. Gropp, C., Herzog, A., Safro, I., Wilson, P. W., & Apon, A. W. (2016). Scalable dynamic topic modeling with clustered latent dirichlet allocation (CLDA) (Version 3). arXiv. https://doi.org/10.48550/ARXIV.1610.07703
35. Gupta, M., Singh, A., Jain, R., Saxena, A., & Ahmed, S. (2021). Multi-class railway complaints categorization using Neural Networks: RailNeural. Journal of Rail Transport Planning and Management, 20, 100265. https://doi.org/10.1016/j.jrtpm.2021.100265
36. HaCohen-Kerner, Y., Dilmon, R., Hone, M., & Ben-Basan, M. A. (2019). Automatic classification of complaint letters according to service provider categories. Information Processing and Management, 56(6), 102102. https://doi.org/10.1016/j.ipm.2019.102102
37. Halstead, D., & Dröge, C. (1991). Consumer attitudes toward complaining and the prediction of multiple complaint responses. Advances in Consumer Research, 18(1), 210–216.
38. Hendry, D., Darari, F., Nurfadillah, R., Khanna, G., Sun, M., Condylis, P. C., & Taufik, N. (2021). Topic modeling for customer service chats. ICACSIS, International Conference on Advanced Computer Science and Information Systems, 23-25 October 2021, Depok, Indonesia, pp. 1–6. https://doi.org/10.1109/icacsis53237.2021.9631322
39. Hofmann, T. (1999). Probabilistic latent semantic indexing. SIGIR ’99, 22nd annual international ACM SIGIR conference on Research and development in information retrieval, August 1999, New York, USA, pp. 50–57. https://doi.org/10.1145/312624.312649
40. Homburg, C., & Fürst, A., 2007. See no evil, hear no evil, speak no evil: a study of defensive organizational behavior towards customer complaints. Journal of the Academy of Marketing Science. 35(4), 523–536. https://doi.org/10.1007/s11747-006-0009-x
41. İlhan Omurca, S., Ekinci, E., Yakupoğlu, E., Arslan, E., & Çapar, B. (2021). Automatic detection of the topics in customer complaints with artificial intelligence. Balkan Journal of Electrical and Computer Engineering, 9(3), 268–277. https://doi.org/10.17694/bajece.832274
42. Johnston, R. (2001). Linking complaint management to profit. International Journal of Service Industry Management, 12(1), 60-69. https://doi.org/10.1108/09564230110382772
43. Karami, A., & Pendergraft, N. M. (2018). Computational analysis of insurance complaints: GEICO case study. arXiv. https://doi.org/10.48550/ARXIV.1806.09736
44. Kirilenko, A. P., Stepchenkova, S. O., & Dai, X. (2021). Automated topic modeling of tourist reviews: Does the Anna Karenina principle apply?. Tourism Management, 83, 104241. https://doi.org/10.1016/j.tourman.2020.104241
45. Lee, D. D., & Seung, H. S. (2000). Algorithms for non-negative matrix factorization. NIPS 2001, Advances in neural information processing systems, 3-8 December 2001, Vancouver, Canada.
46. McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform manifold approximation and projection for dimension reduction (Version 3). arXiv. https://doi.org/10.48550/ARXIV.1802.03426
47. Nikolenko, S. I., Koltcov, S., & Koltsova, O. (2017). Topic modelling for qualitative studies. Journal of Information Science, 43(1), 88–102. https://doi.org/10.1177/0165551515617393
48. Oğuzlar, A. (2007). Analitik hiyerarşi süreci ile müşteri şikayetlerinin analizi. Akdeniz İİBF Dergisi, 7(14), 122–134.
49. Oly Ndubisi, N., & Yin Ling, T. (2006). Complaint behaviour of Malaysian consumers. Management Research News, 29(1/2), 65–76. https://doi.org/10.1108/01409170610645457
50. Özçınar, H., & Öztürk, T. (2022). Eğitim bilimleri çalışmalarında kullanılan ağ yaklaşımının kavramsal haritalanması. Pamukkale University Journal of Education. https://doi.org/10.9779/pauefd.1087757
51. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using siamese BERT-Networks (Version 1). arXiv. https://doi.org/10.48550/ARXIV.1908.10084
52. Richins, M. L. (1983). Negative word-of-mouth by dissatisfied consumers: a pilot study. Journal of Marketing, 47(1), 68–78. https://doi.org/10.2307/3203428
53. Salmi, S., Mérelle, S., Gilissen, R., van der Mei, R., & Bhulai, S. (2022). Detecting changes in help seeker conversations on a suicide prevention helpline during the COVID− 19 pandemic: in-depth analysis using encoder representations from transformers. BMC Public Health, 22(530). https://doi.org/10.1186/s12889-022-12926-2
54. Sánchez-Franco, M. J., & Rey‐Moreno, M. (2021). Do travelers’ reviews depend on the destination? An analysis in coastal and urban peer‐to‐peer lodgings. Psychology and Marketing, 39(2), 441–459. https://doi.org/10.1002/mar.21608
55. Sann, R., Lai, P-C., Liaw, S-Y., & Chen, C-T. (2021). Modelling online complaining behaviour in the hospitality industry: An application of data mining algorithms. APacCHRIE 2021 Conference, 2-4 June 2021, Singapore, pp. 699–702.
56. Sarı, F. Ö., Alikılıç, Ö., & Onat, F. (2013). E-Complaining: Analysis of lodging customers’ e-complaints from a Turkish internet website. ICIBET-2013, International Conference on Information, Business and Education Technology, 14-15 March 2013, Beijing, China, pp. 561–565. https://doi.org/10.2991/icibet.2013.183
57. Tanrısever, C. (2018). Paket tur satın alan müşterilerin şikâyet analizi. Turizm Akademik Dergisi, 5(1), 114–123.
58. Taşar, D. E., Ozan, Ş., Özdil, U., Akça, M. F., Ölmez, O., Gülüm, S., Kutal, S., & Belhan, C. (2021). Kısa konuşma cümlelerinin dönüştürücü yöntemleriyle otomatik etiketlenmesi. ASYU, 2021 Akıllı Sistemlerde Yenilikler ve Uygulamaları Konferansı, 6-8 Ekim 2021, Elazığ, Türkiye. https://doi.org/10.1109/asyu52992.2021.9598957
59. Yakut Aymankuy, Ş. (2011). Yerli turistlerin internet ortamındaki şikayetlerinin satınalma kararlarına etkileri. Balıkesir Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 14(25), 218–238.
60. Yang, Y., Xu, D.-L., Yang, J.-B., & Chen, Y.-W. (2018). An evidential reasoning-based decision support system for handling customer complaints in mobile telecommunications. Knowledge-Based Systems, 162, 202–210. https://doi.org/10.1016/j.knosys.2018.09.029
61. Zhunis, A., Lima, G., Song, H., Han, J., & Cha, M. (2022). Emotion bubbles: Emotional composition of online discourse before and after the COVID-19 outbreak. WWW '22, ACM Web Conference, 25-29 April 2022, Virtual Event, Lyon, France, pp. 2603–2613. https://doi.org/10.1145/3485447.3512132