Intuitionistic Fuzzy Topsis Approach for Chatbot Selection: AI-Assisted Performance Evaluation

Abstract

This study presents a novel multi-criteria decision-making (MSDM) framework to evaluate generative AI chatbots using the Intuitionistic Fuzzy TOPSIS (IF-TOPSIS) method. Five prominent chatbots-ChatGPT, Claude, Copilot, DeepSeek, and Gemini-are assessed based on expert opinions from academia, industry, and an AI agent, creating a balanced and hybrid human-AI evaluation approach. The inclusion of an AI-based expert introduces methodological innovation by reflecting the evolving role of AI in decision-making. The IF-TOPSIS method effectively manages uncertainty, hesitation, and subjectivity in expert judgments, providing a more robust and nuanced ranking of alternatives. Results show that ChatGPT and Gemini achieve the highest scores, while DeepSeek ranks lowest, highlighting differences in performance across various evaluation criteria. The framework specifically addresses the needs of Turkish academic and industrial research environments, where context-sensitive and transparent evaluations are critical. This study not only advances the methodological integration of AI in expert-based assessments but also offers a practical decision-support model that helps both individuals and institutions select the most appropriate generative AI chatbot for their specific needs.

Keywords

• Intuitionistic Fuzzy TOPSIS (IF-TOPSIS)
• Multi-Criteria Decision-Making (MCDM)
• Generative AI Chatbots
• Decision Support Systems
• Fuzzy Logic Applications

Sezgisel Fuzzy Topsis Tabanlı Yaklaşım ile Chatbot Seçimi: YZ-Destekli Performans Değerlendirmesi

Öz

Bu çalışma, Intuitionistic Fuzzy TOPSIS (IF-TOPSIS) yöntemi kullanılarak üretici yapay zeka sohbet robotlarının değerlendirilmesine yenilikçi bir çok kriterli karar verme (ÇKKV) çerçevesi sunmaktadır. ChatGPT, Claude, Copilot, DeepSeek ve Gemini; akademi, endüstri ve bir yapay zeka temsilcisinden elde edilen uzman görüşlerine dayanarak değerlendirilmiş ve dengeli, hibrit bir insan–YZ değerlendirme yaklaşımı oluşturulmuştur. Yapay zeka temelli bir uzmanın sürece dahil edilmesi, karar verme süreçlerinde YZ’nin gelişen rolünü yansıtarak metodolojik bir yenilik sağlamaktadır. IF-TOPSIS yöntemi, uzman yargılarındaki belirsizlik, tereddüt ve öznelliği yöneterek daha güvenilir ve ayrıntılı bir sıralama sunmaktadır. Bulgular, ChatGPT ve Gemini’nin en yüksek, DeepSeek’in ise en düşük puanları aldığını ve kriterlere göre performans farklılıklarının belirgin olduğunu göstermektedir. Çerçeve, bağlama duyarlı ve şeffaf değerlendirmelerin önemli olduğu Türkiye’deki akademik ve endüstriyel araştırma ortamlarının gereksinimlerini karşılamaya yöneliktir. Bu çalışma, YZ’nin uzman temelli değerlendirmelere entegrasyonunu ilerletmekte ve kullanıcıların ihtiyaçlarına en uygun üretici YZ sohbet robotunu seçmelerine yardımcı olacak uygulanabilir bir karar destek modeli sunmaktadır.

Anahtar Kelimeler

• Sezgisel Bulanık TOPSIS
• Çok Kriterli Karar Verme (ÇKKV)
• Üretken Yapay Zeka Sohbet Robotları
• Karar Destek Sistemleri
• Bulanık Mantık Uygulamaları

References

1. Adamopoulou, E., and Moussiades, L. (2020a). Chatbots: History, technology, and applications. Machine Learning with Applications, 2. https://doi.org/10.1016/j.mlwa.2020.100006
2. Adamopoulou, E., and Moussiades, L. (2020b). An overview of chatbot technology. In I. Maglogiannis, L. Iliadis, & E. Pimenidis (Eds.), Artificial Intelligence Applications and Innovations. AIAI 2020 (IFIP Advances in Information and Communication Technology, Vol. 584, pp. xxx–xxx). Cham: Springer. https://doi.org/10.1007/978-3-030-49186-4_31
3. Alkan, N., & Kahraman, C. (2023). Continuous intuitionistic fuzzy sets (CINFUS) and their AHP & TOPSIS extension: Research proposals evaluation for grant funding. Applied Soft Computing, 145.
4. Brandtzaeg, P. B., & Følstad, A. (2017). Why people use chatbots. In I. Kompatsiaris et al. (Eds.), Internet Science. INSCI 2017 (Lecture Notes in Computer Science, Vol. 10673, pp. 377-392). Cham: Springer. https://doi.org/10.1007/978-3-319-70284-1_30
5. Bukar, U. A., Sayeed, M. S., Fatimah Abdul Razak, S., Yogarayan, S., and Sneesl, R. (2024). Decision-making framework for the utilization of generative artificial intelligence in education: A case study of ChatGPT. IEEE Access, 12, 95368–95389. https://doi.org/10.1109/ACCESS.2024.3425172
6. Chakrabortty, R. K., Abdel-Basset, M., and Ali, A. M. (2023). A multi-criteria decision analysis model for selecting an optimum customer service chatbot under uncertainty. Decision Analytics Journal, 6. https://doi.org/10.1016/j.dajour.2023.100168
7. Çınaroğlu, E., and Zaralı, F. (2022). Evaluation of the airline website quality with the intuitionistic fuzzy TOPSIS method. Journal of Turkish Operations Management, 6(2), 1277–1296. https://doi.org/10.56554/jtom.1090290
8. Damgacı, E., Boran, K., and Boran, F. E. (2017). Sezgisel Bulanık TOPSIS yöntemi kullanarak Türkiye’nin yenilenebilir enerji kaynaklarının değerlendirilmesi. Politeknik Dergisi, 20(3), 629–637.

9. Deschenes, A., and McMahon, M. (2024). A survey on student use of generative AI chatbots for academic research. Evidence Based Library and Information Practice, 19(2), 2–22. https://doi.org/10.18438/eblip30512
10. Dong, C., Li, Y., Gong, H., Chen, M., Li, J., Shen, Y., and Yang, M. (2021). A survey of natural language generation. arXiv: 2112.11739.
11. Ecer, F. (2022). Sezgisel Bulanık TOPSIS ve üçüncü parti tersine lojistik tedarikçisi seçimine uygulanması. In C. Karamaşa, F. Ecer, and A. Aytekin (Eds.), Çok Kriterli Karar Verme Yöntemlerinin Bulanık Küme Uzantıları ve Uygulamaları (pp. 35-48). Ankara: Nobel Akademik Yayıncılık.
12. Efe, B., Boran, F., and Kurt, M. (2015). Sezgisel bulanık TOPSIS yöntemi kullanılarak ergonomik ürün konsept seçimi. Mühendislik Bilimleri ve Tasarım Dergisi, 3(3), 433–440.
13. Fontoura, L., Nascimento, D. L. M., Julio Vieira Neto, J. V., and Caiado, R. G. G. C. (2025). Energy Gen-AI technology framework: A perspective of energy efficiency and business ethics in operation management. Technology in Society, 81. https://doi.org/10.1016/j.techsoc.2025.102847
14. Grand View Research. (2023). Chatbot market size, share & trends analysis report by product, application, region, and segment forecasts, 2023–2030. Retrieved from https://www.grandviewresearch.com/industry-analysis/chatbot-market
15. Gupta, S., Kaur, S., Gupta, M., and Singh, T. (2024). AI empowered academia: A fuzzy prioritization framework for academic challenges. Journal of International Education in Business. https://doi.org/10.1108/JIEB-06-2024-0071
16. Ilieva, G. (2025). Extension of interval-valued hesitant Fermatean fuzzy TOPSIS for evaluating and benchmarking generative AI chatbots. Electronics, 14(3). https://doi.org/10.3390/electronics14030555
17. Ilieva, G., Yankova, T., Ruseva, M., and Kabaivanov, S. (2025). A framework for generative AI-driven assessment in higher education. Information, 16(6). https://doi.org/10.3390/info16060472
18. Karcıoğlu, B., and Yalçın, E. (2022). Sezgisel Bulanık TOPSİS yöntemiyle portföy seçimi: Borsa İstanbul’da bir uygulama. Muhasebe ve Finansman Dergisi, 94, 151–184. https://doi.org/10.25095/mufad.1037322
19. Kerimbayev, N., Menlibay, Z., Garvanova, M., Djaparova, S., and Jotsov, V. (2024). A comparative analysis of generative AI models for improving learning process in higher education. In Proceedings of the 2024 International Conference Automatics and Informatics (ICAI) (pp. 271–276). IEEE. https://doi.org/10.1109/ICAI63388.2024.10851491
20. Khanna, A., Pandey, B., Vashishta, K., Kalia, K., Bhale, P., & Das, T. (2015). A study of today’s A.I. through chatbots and rediscovery of machine intelligence. International Journal of U- and e-Service, Science and Technology, 8, 277–284. http://dx.doi.org/10.14257/ijunesst.2015.8.7.28
21. Madenoğlu, F. S. (2023). A group decision-making method for ranking problems under sustainability, risks, and intuitionistic fuzzy environment. Pamukkale Social Sciences Institute Journal, 56, 123–137. https://doi.org/10.30794/pausbed.1129214
22. MarketsandMarkets. (2023). Chatbot market by component, application, deployment mode, organization size, industry vertical, and region: Global forecast to 2030. Retrieved from https://www.marketsandmarkets.com/Market-Reports/chatbot-market-118246967.html
23. Mashat, N. M., & Kabir, G. (2024). Selection of wind turbine blade material using intuitionistic fuzzy TOPSIS method. Engineering Proceedings, 76(1). https://doi.org/10.3390/engproc2024076014
24. Mavrych, V., Yaqinuddin, A., & Bolgova, O. (2025). Claude, ChatGPT, Copilot, and Gemini performance versus students in different topics of neuroscience. Advances in Physiology Education, 49(2), 430–437. http://doi.org/10.1152/advan.00093.2024
25. Radulescu, C. Z., & Radulescu, M. (2025). Criteria analysis for the selection of a generative artificial intelligence tool for academic research: An improved group DEMATEL method. Applied Sciences, 15(10). https://doi.org/10.3390/app15105416
26. Sallam, M., Snygg, J., Hamdan, A., Allam, D., Kassem, R., & Damani, M. (2025). Evaluating the performance of seven large language models in answering healthcare quality management inquiries. Research and Advances in Education, 4(4), 39–50. https://doi.org/10.63593/RAE.2788-7057.2025.05.005
27. Suhaili, S. M., Salim, N., & Jambli, M. N. (2021). Service chatbots: A systematic review. Expert Systems with Applications, 184. https://doi.org/10.1016/j.eswa.2021.115461
28. Suta, P., Lan, X., Wu, B., Mongkolman, P., & Chan, J. H. (2020). An overview of machine learning in chatbots. International Journal of Mechanical Engineering and Robotics Research, 9(4). https://doi.org/10.18178/ijmerr.9.4.502-510
29. Tuğrul, F. (2022). An approach utilizing the intuitionistic fuzzy TOPSIS method to unmanned air vehicle selection. Ikonion Journal of Mathematics, 4, 31–41. https://doi.org/10.54286/ikjm.1174877
30. Tunçer, M., & Zeybek, G. (2024). Accept or reject? What do academics think about utilizing ChatGPT publications? Nevşehir Hacı Bektaş Veli Üniversitesi SBE Dergisi, 14(1), 203–218. https://doi.org/10.30783/nevsosbilen.1413539
31. Yang, L., & Wang, J. (2024). Factors influencing initial public acceptance of integrating the ChatGPT-type model with government services. Kybernetes, 53(11), 4948–4975. http://doi.org/10.1108/K-06-2023-1011
32. Yıldırım, B. F. (2019). Kredi kartı platformlarının sezgisel Bulanık TOPSIS yöntemi kullanılarak değerlendirilmesi. BDDK Bankacılık ve Finansal Piyasalar Dergisi, 13(1), 37–58.