Ensemble Learning Approach to Chatbot Design Based on Paraphrase Detection

Year 2023, Volume: 6 Issue: 2, 129 - 137, 30.11.2023

Himmet Toprak Kesgin , Onur Öztunç , Banu Diri

https://doi.org/10.34088/kojose.1198245

Abstract

In this paper, we present a design for an ensemble chatbot based on paraphrase detection. Our proposed chatbot is intended to assist companies in reducing the need for costly call center operations by providing a 24-hour service to customers seeking information about products or services. Our algorithm is designed to work effectively on small data sets, such as an existing FAQ, and does not require a large number of instances. We evaluated the performance of our chatbot using publicly available data from the websites of major telecommunication companies and found that the ensemble model improved success rates by 6% compared to the single best model, with a top 3 accuracy of 84.54% and a top 1 accuracy of 70.10%.

Keywords

Chatbot, Question Answering, Ensemble Learning, Paraphrase Detection, Chatbot Design

References

• [1] AbuShawar B., Atwell E., 2015. ALICE chatbot: Trials and outputs. Computación y Sistemas, 19(4), pp. 625‒632. Instituto Politécnico Nacional, Centro de Investigación en Computación.
• [2] Luo B., Lau R.Y.K., Li C., Si Y.W., 2022. A critical review of state-of-the-art chatbot designs and applications. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(1), pp. e1434. Wiley Online Library.
• [3] Devlin J., Chang M.W., Lee K., Toutanova K., 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Paper presented at the North American Chapter of the Association for Computational Linguistics.
• [4] Amer E., Hazem A., Farouk O., Louca A., Mohamed Y., Ashraf M., 2021. A proposed chatbot framework for COVID-19. Paper presented at the 2021 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), IEEE, pp. 263-268.
• [5] Rajpurkar P., Zhang J., Lopyrev K., Liang P., 2016. SQuAD: 100,000+ Questions for Machine Comprehension of Text. Paper presented at the Conference on Empirical Methods in Natural Language Processing.
• [6] Taşar D.E., Şükrü O., Kutal S., Ölmez O., Gülüm S., Akca F., Belhan C., 2021. Performance Trade-Off for Bert Based Multi-Domain Multilingual Chatbot Architectures. Journal of Artificial Intelligence and Data Science, 1(2), pp. 144‒149. Izmir Katip Celebi University.
• [7] Sak H., Senior A.W., Beaufays F., 2014. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. Paper presented at Interspeech.
• [8] Yin Z., Chang K.H., Zhang R., 2017. DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2131‒2139. ACM.
• [9] Serban I., Sankar C., Germain M., Zhang S., Lin Z., Subramanian S., Kim T., Pieper M., Chandar A.P.S., Ke N.R., Mudumba S., de Brébisson A., Sotelo J.M.R., Suhubdy D., Michalski V., Nguyen A., Pineau J., Bengio Y., 2017. A Deep Reinforcement Learning Chatbot. ArXiv, vol. abs/1709.02349.
• [10] Mikolov T., Sutskever I., Chen K., Corrado G.S., Dean J., 2013. Distributed Representations of Words and Phrases and their Compositionality. Paper presented at NIPS.
• [11] Bojanowski P., Grave E., Joulin A., Mikolov T., 2016. Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5, pp. 135‒146.
• [12] Sutskever I., Vinyals O., Le Q.V., 2014. Sequence to Sequence Learning with Neural Networks. ArXiv, vol. abs/1409.3215.
• [13] Boyanov M., Nakov P., Moschitti A., Da San Martino G., Koychev I., 2017. Building Chatbots from Forum Data: Model Selection Using Question Answering Metrics. ArXiv, vol. abs/1710.00689.
• [14] Bilgin T.T., Yavuz E., 2021. Conceptual design of python ide with embedded turkish spoken chatbot that analyzes and corrects the syntax errors. Avrupa Bilim ve Teknoloji Dergisi, (29), pp. 415‒424.
• [15] İçseri İ., Aydın Ö., Tutuk K., 2021. Müşteri Hizmetleri Yönetiminde Yapay Zeka Temelli Chatbot Geliştirilmesi. Avrupa Bilim ve Teknoloji Dergisi, (29), pp. 358‒365.
• [16] Toprak G., Rasheed J., 2022. Machine Learning based Natural Language Processing for Turkish Venue Recommendation Chatbot Application. Avrupa Bilim ve Teknoloji Dergisi, (38), pp. 501‒506.
• [17] Barış A., 2020. A new business marketing tool: chatbot. GSI Journals Serie B: Advancements in Business and Economics, 3(1), pp. 31‒46.
• [18] Eroglu-Hall E., Sevim N., Bulut A., 2022. Çevrimiçi tüketici tutumları chatbotlara yönelik. EKEV Akademi Dergisi, (91), pp. 33‒53.
• [19] Luhn H.P., 1957. A statistical approach to mechanized encoding and searching of literary information. IBM Journal of Research and Development, 1(4), pp. 309‒317.
• [20] Sparck Jones K., 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28(1), pp. 11‒21.
• [21] Kusner M., Sun Y., Kolkin N., Weinberger K., 2015. From word embeddings to document distances. Paper presented at International Conference on Machine Learning, pp. 957‒966
• [22] Guo S., Wang Q., 2022. Application of knowledge distillation based on transfer learning of ERNIE model in intelligent dialogue intention recognition. Sensors, 22(3), pp. 1270.