Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis

Abstract

This research examines the customers’ comments about the chatbots published in a customer complaint website while shopping from an e-commerce site or app. First, 89 customers’ complaints were imported from a customer complaint platform to a single document. Then, the document was subjected to content analysis using a qualitative research tool, Maxqda Plus 2022, and each comment was categorized under related complaint categories. Second, the frequency of customer complaints categories was calculated using the same tool. Additionally, visual maps for each category were created to make the complaints more understandable. While these categorical variables have been addressed in previous studies, variables based on consumer feedback have only been included in this study. According to the research findings, the most frequent customer complaint category is about meaningfulness (with a share of 47.6% in the general total). The least frequent ones are the inability to find a real contact person and the absence of chatbot service (with a share of 7.9%).

Keywords

• E-commerce
• Chatbot
• Customer experience
• Customer satisfaction
• Dissatisfaction

References

1. Acumen Research and Consulting. (2022). Access: 09.01.2022, https://www.globenewswire.com/news-release/2022/07/25/2485463/0/en/Chatbot-Market-Size-to-Grow-to-USD-3-411-Million-by-2030-Propelled-By-the-Growing-Use-of-Bots-for-Marketing-and-Promotion-Activities.html google scholar
2. Almansor, E. H., Hussan, F. K. & Hussain, O. K. (2021). Supervised ensemble sentiment-based framework to measure chatbot quality of services. Computing, 103(3), 491-507. google scholar
3. Anantaram, C. & Sangroya, A. (2017). Identifying latent beliefs in customer complaints to trigger epistemic rules for relevant human-bot dialog. 3rd International Conference on Control, Automation and Robotics (ICCAR), 731-734. google scholar
4. Arya, M. (2019). Access: 09.01.2022, https://chatbotslife.com/a-brief-history-of-chatbots-d5a8689cf52f google scholar
5. Behere, T., Vaidya, A., Birhade, A., Shinde, K., Deshpande, P. & Jahirabadkar, S. (2020). Text summarization and classification of conversation data between service chatbot and customer. 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), 833-838. google scholar
6. Brandtzaeg, P. B. & F0lstad, A. (2017). Why people use chatbots. International Conference on Internet Science, 377-392. google scholar
7. Bührke, J., Brendel, A. B., Lichtenberg, S., Greve, M. & Mirbabaie, M. (2021). Is making mistakes human? On the perception of typing errors in chatbot communication. In Proceedings of the 54th Hawaii International Conference on System Sciences. google scholar
8. Chaves, A. P & Gerosa, M. A. (2020). How should my chatbot interact? A survey on social characteristics in human-chatbot interaction design. International Journal of Human-Computer interaction. google scholar

9. Cheng, H.-T. & Pan, Y. (2021). "I’m not a Chatbot": An empirical investigation of humanized profiles of social media customer service representatives. 54th Hawaii International Conference on System Sciences, 4167-4176. google scholar
10. Chung, M., Ko, E., Joung, H. & Kim, S. J. (2020). Chatbot e-service and customer satisfaction regarding luxury brands. Journal of Business Research, 117, 587-595. google scholar
11. Collins, C., Arbour, S., B. N., Yama, S., Laffier, J. & Zhao, Z. (2021). Covid connect: Chat-driven anonymous story-sharing for peer support. Designing Interactive Systems Conference, 301-318. google scholar
12. Crolic, C., Thomaz, F., Hadi, R. & Stephen, A. T. (2022). Blame the bot: Anthropomorphism and anger in customer-chatbot interactions. Journal of Marketing, 132-148. google scholar
13. Cui, L., Huang, S., Wei, F., Tan, C., Duan, C. & Zhou, M. (2017). Superagent: A customer service chatbot for e-commerce websites. Proceedings of ACL 2017, 97-102. google scholar
14. Davenport, T. H. & Klahr, P. (1998). Managing customer support knowledge. California Management Review, 40(3), 195-208. google scholar
15. Davidow, M. (2003). Organizational responses to customer complaints: What works and what doesn’t. Journal of Service Research, 5(3), 225-250. google scholar
16. Dosovitsky, G. & Bunge, E. L. (2021). Bonding with bot: User feedback on a chatbot for social isolation. Frontiers in Digital Health, 3, 1-11. google scholar
17. Duan, Y., Yoon, M., Liang, Z. & Hoorn, J. F. (2021). Self-disclosure to a robot: only for those who suffer the most. Robotics, 10(3). google scholar
18. Federici, S., Filippis, M. L., Mele, M. L., Borsci, S., Bracalenti, M., Gaudino, G. & Simonetti, E. (2020). Inside pandora’s box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs. Disability and Rehabilitation: Assistive Technology, 15(7), 832-837. google scholar
19. F0lstad, A., Nordheim, C. B., & Bj0rkli, C. A. (2018). What makes users trust a chatbot for customer service? An exploratory interview study. International Conference on Internet Science, 194-208. google scholar
20. Gashi, F., Regli, S. F., May, R., Tschopp, P. & Denecke, K. (2021). Developing intelligent interviewers to collect the medical history: Lessons learned and guidelines. dHealth, 18-25. google scholar
21. Harrison-Walker, L. J. (2001). E-complaining: A content analysis of an internet complaint forum. Journal of Services Marketing, 15(5), s. 397-412. google scholar
22. Hill, J., Ford, W. R. & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between human-human online conversations and human-chatbot conversations. Computers in Human Behavior, 49, 245-250. google scholar
23. Huang, J., Zhou, M. & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. IJCAI, 423-428. google scholar
24. Jayachandran, S., Hewett, K. & Kaufman. (2004). Customer response capability in a sense-and-respond era: the role of customer knowledge process. Journal of the Academy of Marketing Science, 32(3), 219-233. google scholar
25. Jenkins, M. C., Churchill, R., Cox, S. & Smith, D. (2007). Analysis of user interaction with service oriented chatbot systems. International Conference On Human-Computer Interaction, 76-83. google scholar
26. Khan, S. & Rabbani, M. R. (2020). Chatbot as Islamic finance expert (CaIFE): When finance meets artificial intellige. Proceedings of the 2020 4th International Symposium on Computer Science and Intelligent Control, 1-5. google scholar
27. Kuberkar, S. & Singhal, T. K. (2020). Factors influencing adoption intention of A.I. powered chatbot for public transport services within a smart city. International Journal of Emerging Technologies in Learning, 11(3), 948-958. google scholar
28. Liu, C., Zhou, S., Zhang, Y., Liu, D., Peng, Z. & Ma, X. (2022). Exploring the effects of self-mockery to improve task-oriented chatbot’s social intelligence. Designing Interactive Systems Conference, 1315-1329. google scholar
29. Narasiman, S. K., Srinivassababu, T. H., Suhit Raja, S. & Babu, R. (2019). IndQuery-An online portal for registering e-complaints integrated with smart chatbot. International Conference on Emerging Current Trends in Computing and Expert Technology, 1286-1294. google scholar
30. Narendra, L. W. & Setyaningsih, E. R. (2021). Designing a transactional smart assistant in indonesian using rasa framework. 7th International Conference on Electrical, Electronics and Information Engineering, 1-6. google scholar
31. Narynov, S., Zhumanov, Z., Gumar, A., K.M. & Omarov, B. (2021). Development of chatbot psychologist applying natural language understanding techniques. 21st International Conference on Control, Automation and Systems (ICCAS), 636-641. google scholar
32. Ni, L., Lu, C., Liu, N. & Liu, J. (2017). Mandy: Towards a smart primary care chatbot application. International symposium on knowledge and systems sciences, 38-52. google scholar
33. Orellana, C., Tobar, M. Y., J., P.-O. D. & Guachi-Guachi, L. (2021). A chatterbot based on genetic algorithm: Preliminary results. International Conference on Applied Informatics , 3-12. google scholar
34. Othlinghaus-Wulhorst, J. & Hoppe, H. U. (2020). A technical and conceptual framework for serious role-playing games in the area of social skill training. Frontiers in Computer Science, 2(28), 1-20. google scholar
35. Raundale, P. & Sawale, A. (2021). Dialog prediction in institute admission: A deep learning way. 2nd International Conference for Emerging Technology (INCET), 1-5. google scholar
36. Rigamonti, L., Estel, K., Gehlen, T., Wolfarth, B., Lawrence, J. B. & Back, D. A. (2021). Use of artificial intelligence in sports medicine: a report of5 fictional cases. BMC Sports Science, Medicine and Rehabilitation, 13(1), 1-7. google scholar
37. Sangroya, A., Anantaram, C., Saini, P. & Rawat, M. (2018). Extracting latent beliefs and using epistemic reasoning to tailor a chatbot. JCAI, 5853-5855. google scholar
38. Shawar, B. A. & Atwell, E. (2007). Different measurements metrics to evaluate a chatbot system. Proceedings of the Workshop on Bridging the Gap: Academic And Industrial Research in Dialog Technologies, 89-96. google scholar
39. Suhaili, S. M., Salim, N. & Jambli, M. N. (2021). Service chatbots: A systematic review. Expert Systems with Applications, 184. google scholar
40. Suresh, N., Mukabe, N., Hashiyana, V., Limbo, A. & Hauwanga, A. (2021). Career counseling chatbot on facebook messenger using A.I. ACM International Conference Proceeding Series, 65-73. google scholar
41. Temple, J. G. & Elie, C. J. (2019). Beyond the chatbot: enhancing search with cognitive capabilities. International Conference on Applied Human Factors and Ergonomics, 283-290. google scholar
42. Van Den Broeck, E., Zarouali, B. & Poels, K. (2019). Chatbot advertising effectiveness: When does the message get through? Computers in Human Behavior, 98, 150-157. google scholar
43. Waghmare, C. (2019). Business Benefits of Using Chatbots. Apress. google scholar
44. Xu, A., Liu, Z., Guo, Y., Sinha, V. & Akkiraju, R. (2017). A new chatbot for customer service on social media. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 3506-3510. google scholar
45. Zamora, J. (2017). I’m sorry, dave, i’m afraid i can’t do that: Chatbot perception and expectations. Proceedings of the 5th International Conference on Human Agent Interaction, 253-260. google scholar