Customer Dissatisfaction Towards Chatbot Services of e-Commerce Shopping Sites: A Qualitative Analysis

Year 2024, , 112 - 120, 17.05.2024

Burak Can Altay , Naim Çetintürk

https://doi.org/10.26650/JTL.2024.1355850

Abstract

This research examines the customers’ comments about the chatbots published in a customer complaint website while shopping from an e-commerce site or app. First, 89 customers’ complaints were imported from a customer complaint platform to a single document. Then, the document was subjected to content analysis using a qualitative research tool, Maxqda Plus 2022, and each comment was categorized under related complaint categories. Second, the frequency of customer complaints categories was calculated using the same tool. Additionally, visual maps for each category were created to make the complaints more understandable. While these categorical variables have been addressed in previous studies, variables based on consumer feedback have only been included in this study. According to the research findings, the most frequent customer complaint category is about meaningfulness (with a share of 47.6% in the general total). The least frequent ones are the inability to find a real contact person and the absence of chatbot service (with a share of 7.9%).

Keywords

E-commerce, Chatbot, Customer experience, Customer satisfaction, Dissatisfaction

References

• Acumen Research and Consulting. (2022). Access: 09.01.2022, https://www.globenewswire.com/news-release/2022/07/25/2485463/0/en/Chatbot-Market-Size-to-Grow-to-USD-3-411-Million-by-2030-Propelled-By-the-Growing-Use-of-Bots-for-Marketing-and-Promotion-Activities.html google scholar
• Almansor, E. H., Hussan, F. K. & Hussain, O. K. (2021). Supervised ensemble sentiment-based framework to measure chatbot quality of services. Computing, 103(3), 491-507. google scholar
• Anantaram, C. & Sangroya, A. (2017). Identifying latent beliefs in customer complaints to trigger epistemic rules for relevant human-bot dialog. 3rd International Conference on Control, Automation and Robotics (ICCAR), 731-734. google scholar
• Arya, M. (2019). Access: 09.01.2022, https://chatbotslife.com/a-brief-history-of-chatbots-d5a8689cf52f google scholar
• Behere, T., Vaidya, A., Birhade, A., Shinde, K., Deshpande, P. & Jahirabadkar, S. (2020). Text summarization and classification of conversation data between service chatbot and customer. 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), 833-838. google scholar
• Brandtzaeg, P. B. & F0lstad, A. (2017). Why people use chatbots. International Conference on Internet Science, 377-392. google scholar
• Bührke, J., Brendel, A. B., Lichtenberg, S., Greve, M. & Mirbabaie, M. (2021). Is making mistakes human? On the perception of typing errors in chatbot communication. In Proceedings of the 54th Hawaii International Conference on System Sciences. google scholar
• Chaves, A. P & Gerosa, M. A. (2020). How should my chatbot interact? A survey on social characteristics in human-chatbot interaction design. International Journal of Human-Computer interaction. google scholar
• Cheng, H.-T. & Pan, Y. (2021). "I’m not a Chatbot": An empirical investigation of humanized profiles of social media customer service representatives. 54th Hawaii International Conference on System Sciences, 4167-4176. google scholar
• Chung, M., Ko, E., Joung, H. & Kim, S. J. (2020). Chatbot e-service and customer satisfaction regarding luxury brands. Journal of Business Research, 117, 587-595. google scholar
• Collins, C., Arbour, S., B. N., Yama, S., Laffier, J. & Zhao, Z. (2021). Covid connect: Chat-driven anonymous story-sharing for peer support. Designing Interactive Systems Conference, 301-318. google scholar
• Crolic, C., Thomaz, F., Hadi, R. & Stephen, A. T. (2022). Blame the bot: Anthropomorphism and anger in customer-chatbot interactions. Journal of Marketing, 132-148. google scholar
• Cui, L., Huang, S., Wei, F., Tan, C., Duan, C. & Zhou, M. (2017). Superagent: A customer service chatbot for e-commerce websites. Proceedings of ACL 2017, 97-102. google scholar
• Davenport, T. H. & Klahr, P. (1998). Managing customer support knowledge. California Management Review, 40(3), 195-208. google scholar
• Davidow, M. (2003). Organizational responses to customer complaints: What works and what doesn’t. Journal of Service Research, 5(3), 225-250. google scholar
• Dosovitsky, G. & Bunge, E. L. (2021). Bonding with bot: User feedback on a chatbot for social isolation. Frontiers in Digital Health, 3, 1-11. google scholar
• Duan, Y., Yoon, M., Liang, Z. & Hoorn, J. F. (2021). Self-disclosure to a robot: only for those who suffer the most. Robotics, 10(3). google scholar
• Federici, S., Filippis, M. L., Mele, M. L., Borsci, S., Bracalenti, M., Gaudino, G. & Simonetti, E. (2020). Inside pandora’s box: a systematic review of the assessment of the perceived quality of chatbots for people with disabilities or special needs. Disability and Rehabilitation: Assistive Technology, 15(7), 832-837. google scholar
• F0lstad, A., Nordheim, C. B., & Bj0rkli, C. A. (2018). What makes users trust a chatbot for customer service? An exploratory interview study. International Conference on Internet Science, 194-208. google scholar
• Gashi, F., Regli, S. F., May, R., Tschopp, P. & Denecke, K. (2021). Developing intelligent interviewers to collect the medical history: Lessons learned and guidelines. dHealth, 18-25. google scholar
• Harrison-Walker, L. J. (2001). E-complaining: A content analysis of an internet complaint forum. Journal of Services Marketing, 15(5), s. 397-412. google scholar
• Hill, J., Ford, W. R. & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between human-human online conversations and human-chatbot conversations. Computers in Human Behavior, 49, 245-250. google scholar
• Huang, J., Zhou, M. & Yang, D. (2007). Extracting chatbot knowledge from online discussion forums. IJCAI, 423-428. google scholar
• Jayachandran, S., Hewett, K. & Kaufman. (2004). Customer response capability in a sense-and-respond era: the role of customer knowledge process. Journal of the Academy of Marketing Science, 32(3), 219-233. google scholar
• Jenkins, M. C., Churchill, R., Cox, S. & Smith, D. (2007). Analysis of user interaction with service oriented chatbot systems. International Conference On Human-Computer Interaction, 76-83. google scholar
• Khan, S. & Rabbani, M. R. (2020). Chatbot as Islamic finance expert (CaIFE): When finance meets artificial intellige. Proceedings of the 2020 4th International Symposium on Computer Science and Intelligent Control, 1-5. google scholar
• Kuberkar, S. & Singhal, T. K. (2020). Factors influencing adoption intention of A.I. powered chatbot for public transport services within a smart city. International Journal of Emerging Technologies in Learning, 11(3), 948-958. google scholar
• Liu, C., Zhou, S., Zhang, Y., Liu, D., Peng, Z. & Ma, X. (2022). Exploring the effects of self-mockery to improve task-oriented chatbot’s social intelligence. Designing Interactive Systems Conference, 1315-1329. google scholar
• Narasiman, S. K., Srinivassababu, T. H., Suhit Raja, S. & Babu, R. (2019). IndQuery-An online portal for registering e-complaints integrated with smart chatbot. International Conference on Emerging Current Trends in Computing and Expert Technology, 1286-1294. google scholar
• Narendra, L. W. & Setyaningsih, E. R. (2021). Designing a transactional smart assistant in indonesian using rasa framework. 7th International Conference on Electrical, Electronics and Information Engineering, 1-6. google scholar
• Narynov, S., Zhumanov, Z., Gumar, A., K.M. & Omarov, B. (2021). Development of chatbot psychologist applying natural language understanding techniques. 21st International Conference on Control, Automation and Systems (ICCAS), 636-641. google scholar
• Ni, L., Lu, C., Liu, N. & Liu, J. (2017). Mandy: Towards a smart primary care chatbot application. International symposium on knowledge and systems sciences, 38-52. google scholar
• Orellana, C., Tobar, M. Y., J., P.-O. D. & Guachi-Guachi, L. (2021). A chatterbot based on genetic algorithm: Preliminary results. International Conference on Applied Informatics , 3-12. google scholar
• Othlinghaus-Wulhorst, J. & Hoppe, H. U. (2020). A technical and conceptual framework for serious role-playing games in the area of social skill training. Frontiers in Computer Science, 2(28), 1-20. google scholar
• Raundale, P. & Sawale, A. (2021). Dialog prediction in institute admission: A deep learning way. 2nd International Conference for Emerging Technology (INCET), 1-5. google scholar
• Rigamonti, L., Estel, K., Gehlen, T., Wolfarth, B., Lawrence, J. B. & Back, D. A. (2021). Use of artificial intelligence in sports medicine: a report of5 fictional cases. BMC Sports Science, Medicine and Rehabilitation, 13(1), 1-7. google scholar
• Sangroya, A., Anantaram, C., Saini, P. & Rawat, M. (2018). Extracting latent beliefs and using epistemic reasoning to tailor a chatbot. JCAI, 5853-5855. google scholar
• Shawar, B. A. & Atwell, E. (2007). Different measurements metrics to evaluate a chatbot system. Proceedings of the Workshop on Bridging the Gap: Academic And Industrial Research in Dialog Technologies, 89-96. google scholar
• Suhaili, S. M., Salim, N. & Jambli, M. N. (2021). Service chatbots: A systematic review. Expert Systems with Applications, 184. google scholar
• Suresh, N., Mukabe, N., Hashiyana, V., Limbo, A. & Hauwanga, A. (2021). Career counseling chatbot on facebook messenger using A.I. ACM International Conference Proceeding Series, 65-73. google scholar
• Temple, J. G. & Elie, C. J. (2019). Beyond the chatbot: enhancing search with cognitive capabilities. International Conference on Applied Human Factors and Ergonomics, 283-290. google scholar
• Van Den Broeck, E., Zarouali, B. & Poels, K. (2019). Chatbot advertising effectiveness: When does the message get through? Computers in Human Behavior, 98, 150-157. google scholar
• Waghmare, C. (2019). Business Benefits of Using Chatbots. Apress. google scholar
• Xu, A., Liu, Z., Guo, Y., Sinha, V. & Akkiraju, R. (2017). A new chatbot for customer service on social media. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 3506-3510. google scholar
• Zamora, J. (2017). I’m sorry, dave, i’m afraid i can’t do that: Chatbot perception and expectations. Proceedings of the 5th International Conference on Human Agent Interaction, 253-260. google scholar