A Hybrid Decision Support System for AI Chatbot Selection in Enhancing Academical Skills of University Students

Abstract

The remarkable advancement of artificial intelligence (AI) has paved the way for its widespread use across various fields, industries, and application domains. As AI usage becomes more prevalent, the need for proper AI selection and utilization has become increasingly evident. AI tools have emerged as one of the most favored approaches for educational development in recent years. However, the selection of AI chatbots for the academic development of university students often involves a random approach rather than an analytical one. The primary motivation of this research is to develop a decision support system for selecting AI chatbots to enhance university students' four skills (writing, speaking, reading, listening). In this study, the AI chatbot selection problem is examined using a Multi-Criteria Decision-Making (MCDM) approach. The IF (intuitionistic fuzzy sets)-RANCOM (Ranking Comparison Method)-AROMAN (Alternative Ranking Order Method Accounting for Two-Step Normalization Method) hybrid method is proposed as a decision support system. In this hybrid method, fuzzy logic concepts, such as IF sets and IF numbers, are employed to conduct precise calculations. The IF-RANCOM method is used for weighing the AI chatbot selection criteria because it allows for better calculation of the criteria weights, relying on criteria comparisons and expert evaluations. For ranking AI chatbots, the IF-AROMAN method is utilized, as it provides suitable results for determining the most appropriate AI chatbot based on expert evaluations of the alternatives. The IF-RANCOM-AROMAN hybrid method is demonstrated in this paper through four applications. These applications focus on selecting the most suitable AI chatbot for enhancing university students' four skills: writing, speaking, reading, and listening. The research findings indicate that for writing skills development, the best AI chatbot is Write & Improve; for speaking skills, it is ELSA Speak; for reading skills, it is Newsela; and for listening skills, it is EnglishCentral. In addition to providing a decision support tool for the AI chatbot selection process, the research also ranks the selection criteria according to their importance levels. Furthermore, implications for educational development are presented based on the findings.

Keywords

• Artificial Intelligence Selection Problem
• Decision Support System
• Intuitionistic Fuzzy Sets
• Ranking Comparison Method
• Alternative Ranking Order Method Accounting for Two-Step Normalization

Üniversite Öğrencilerinin Akademik Becerilerini Geliştirmede Yapay Zekâ Sohbet Botu Seçimi için Hibrit Bir Karar Destek Sistemi

Abstract

Yapay zekâdaki (YZ) dikkat çekici gelişmeler, çeşitli alanlar, sektörler ve uygulama ortamlarında geniş çapta kullanılmasının önünü açmıştır. YZ kullanımının giderek yaygınlaşmasıyla birlikte, uygun YZ araçlarının doğru seçimi ve etkin kullanımı daha da önemli hâle gelmiştir. Son yıllarda YZ araçları, eğitimsel gelişim için en çok tercih edilen yaklaşımlardan biri olarak öne çıkmaktadır. Ancak üniversite öğrencilerinin akademik gelişimi için YZ sohbet botlarının seçimi çoğu zaman analitik bir yöntem yerine rastgele bir yaklaşımla yapılmaktadır. Bu araştırmanın temel motivasyonu, üniversite öğrencilerinin dört becerisini (yazma, konuşma, okuma, dinleme) geliştirmeye yönelik en uygun YZ sohbet botlarını seçmek için bir karar destek sistemi geliştirmektir. Bu çalışmada, YZ sohbet botu seçimi problemi Çok Kriterli Karar Verme (ÇKKV) yaklaşımı kullanılarak incelenmiştir. Karar destek sistemi olarak IF ( sezgisel bulanık kümeler )–RANCOM (Sıralama Karşılaştırma Yöntemi)–AROMAN (İki Aşamalı Normalizasyonu Dikkate Alan Alternatif Sıralama Yöntemi) hibrit yöntemi önerilmiştir. Bu hibrit yöntemde, IF kümeleri ve IF sayıları gibi bulanık mantık kavramları hassas hesaplamalar yapmak için kullanılmıştır. AI sohbet botu seçim kriterlerinin ağırlıklandırılmasında IF-RANCOM yöntemi tercih edilmiştir; çünkü bu yöntem, kriter ağırlıklarının kriter karşılaştırmaları ve uzman değerlendirmelerine dayalı olarak daha doğru hesaplanmasına imkân tanımaktadır. Sohbet botlarının sıralanmasında ise IF-AROMAN yöntemi kullanılmıştır; zira bu yöntem, uzman değerlendirmelerine dayalı olarak en uygun YZ sohbet botunun belirlenmesi için uygun sonuçlar sunmaktadır. IF-RANCOM-AROMAN hibrit yöntemi, bu makalede dört uygulama üzerinden gösterilmiştir. Bu uygulamalar, üniversite öğrencilerinin dört becerisini geliştirmede en uygun YZ sohbet botunun seçilmesine odaklanmaktadır: yazma, konuşma, okuma ve dinleme. Araştırma bulguları, yazma becerisinin geliştirilmesinde en iyi YZ sohbet botunun Write & Improve; konuşma becerisi için ELSA Speak; okuma becerisi için Newsela; dinleme becerisi için ise EnglishCentral olduğunu göstermektedir. Araştırma, YZ sohbet botu seçim süreci için bir karar destek aracı sunmanın yanında, seçim kriterlerini önem düzeylerine göre de sıralamaktadır. Ayrıca, elde edilen bulgulara dayalı olarak eğitimsel gelişim için çeşitli çıkarımlar da sunulmaktadır.

Keywords

• Yapay Zekâ Seçim Problemi
• Karar Destek Sistemi
• Sezgisel Bulanık Kümeler
• Sıralama Karşılaştırma Yöntemi
• İki Aşamalı Normalizasyonu Dikkate Alan Alternatif Sıralama Yöntemi

References

1. Akanfe, O., Bhatt, P., & Lawong, D. A. (2025). Technology Advancements Shaping the Financial Inclusion Landscape: Present Interventions, Emergence of Artificial Intelligence and Future Directions. Information Systems Frontiers, 1-24. https://doi.org/10.1007/s10796-025-10597-z.
2. Aldoseri, A., Al-Khalifa, K. N., & Hamouda, A. M. (2024). AI-powered innovation in digital transformation: Key pillars and industry impact. Sustainability, 16(5), 1790. https://doi.org/10.3390/su16051790.
3. Alrasheedi, A. F., Mishra, A. R., Pamucar, D., Devi, S., & Cavallaro, F. (2024). Interval-valued intuitionistic fuzzy AROMAN method and its application in sustainable wastewater treatment technology selection. Journal of intelligent & fuzzy systems, 46(3), 7199-7222. https://doi.org/10.3233/JIFS-236697.
4. Atanassov, K. T., & Stoeva, S. (1986). Intuitionistic fuzzy sets. Fuzzy sets and Systems, 20(1), 87-96. https://doi.org/10.1016/S0165-0114(86)80034-3.
5. Aure, P. A., & Cuenca, O. (2024). Fostering social-emotional learning through human-centered use of generative AI in business research education: an insider case study. Journal of Research in Innovative Teaching & Learning, 17(2), 168-181. https://doi.org/10.1108/JRIT-03-2024-0076.
6. Aytekin, A., & Korucuk, S. (2025). Evaluating Barriers to Implementation of Open Innovation in SMEs: A Case Study in Ordu Province. Computer and Decision Making: An International Journal, 2, 440-450. https://doi.org/10.59543/comdem.v2i.11151.
7. Baron, P. (2024). Are AI detection and plagiarism similarity scores worthwhile in the age of ChatGPT and other Generative AI?. Scholarship of Teaching and Learning in the South (SOTL) in the South, 8(2), 151-179. https://doi.org/10.36615/sotls.v8i2.411.
8. Bilquise, G., Ibrahim, S., & Salhieh, S. E. M. (2024). Investigating student acceptance of an academic advising chatbot in higher education institutions. Education and Information Technologies, 29(5), 6357-6382. https://doi.org/10.1007/s10639-023-12076-x.

9. Bošković, S., Švadlenka, L., Jovčić, S., Dobrodolac, M., Simić, V., & Bacanin, N. (2023). An alternative ranking order method accounting for two-step normalization (AROMAN)—A case study of the electric vehicle selection problem. IEEE Access, 11, 39496-39507. https://doi.org/10.1109/ACCESS.2023.3265818.
10. Bouraima, M. B., Jovčić, S., Dobrodolac, M., Pamucar, D., Badi, I., & Maraka, N. D. (2024). Sustainable healthcare system devolution strategy selection using the AROMAN MCDM approach. Spectrum of Decision Making and Applications, 1(1), 46-63. https://doi.org/10.31181/sdmap1120243.
11. Chang, D. H., Lin, M. P. C., Hajian, S., & Wang, Q. Q. (2023). Educational design principles of using AI chatbot that supports self-regulated learning in education: Goal setting, feedback, and personalization. Sustainability, 15(17), 12921. https://doi.org/10.3390/su151712921.
12. Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. Ieee Access, 8, 75264-75278. https://doi.org/10.1109/ACCESS.2020.2988510.
13. Čubranić-Dobrodolac, M., Jovčić, S., Bošković, S., & Babić, D. (2023). A decision-making model for professional drivers selection: A hybridized fuzzy–AROMAN–Fuller approach. Mathematics, 11(13), 2831. https://doi.org/10.3390/math11132831.
14. Ecer, F., Böyükaslan, A., & Hashemkhani Zolfani, S. (2022). Evaluation of cryptocurrencies for investment decisions in the era of Industry 4.0: a borda count-based intuitionistic fuzzy set extensions EDAS-MAIRCA-MARCOS multi-criteria methodology. Axioms, 11(8), 404. https://doi.org/10.3390/axioms11080404.
15. Gual-Montolio, P., Jaén, I., Martínez-Borba, V., Castilla, D., & Suso-Ribera, C. (2022). Using artificial intelligence to enhance ongoing psychological interventions for emotional problems in real-or close to real-time: a systematic review. International Journal of Environmental Research and Public Health, 19(13), 7737. https://doi.org/10.3390/ijerph19137737.
16. Hang, Y., Khan, S., Alharbi, A., & Nazir, S. (2024). Assessing English teaching linguistic and artificial intelligence for efficient learning using analytical hierarchy process and Technique for Order of Preference by Similarity to Ideal Solution. Journal of Software: Evolution and Process, 36(2), e2462. https://doi.org/10.1002/smr.2462.
17. Kara, K., Yalçın, G. C., Simic, V., Yıldırım, A. T., Pamucar, D., & Siarry, P. (2024). A spherical fuzzy-based DIBR II-AROMAN model for sustainability performance benchmarking of wind energy power plants. Expert systems with applications, 253, 124300. https://doi.org/10.1016/j.eswa.2024.124300.
18. Kim, Y., Le, T. L. V., Kim, D., Lee, M., & Lee, S. J. (2024). How Non-native English Speakers Use, Assess, and Select AI-Generated Paraphrases with Information Aids. arXiv preprint arXiv:2405.07475. https://doi.org/10.48550/arXiv.2405.07475. Koç, F. Ş. (2024). The development of listening and speaking skills in efl via an artificially intelligent chatbot application: A quasi-experimental design study (Doctoral dissertation, Middle East Technical University (Turkey)).
19. Komasi, H., Nemati, A., Zolfani, S. H., Williams, N. L., & Bazrafshan, R. (2024). Investigating the effects of COVID-19 on tourism in the G7 countries. Technological and Economic Development of Economy, 30(4), 1064-1086. https://doi.org/10.3846/tede.2024.20821.
20. Labadze, L., Grigolia, M., & Machaidze, L. (2023). Role of AI chatbots in education: systematic literature review. International Journal of Educational Technology in Higher Education, 20(1), 56. https://doi.org/10.1186/s41239-023-00426-1.
21. Lee, D., & Yoon, S. N. (2021). Application of artificial intelligence-based technologies in the healthcare industry: Opportunities and challenges. International journal of environmental research and public health, 18(1), 271. https://doi.org/10.3390/ijerph18010271.
22. Luckin, R; Holmes, W; (2016) Intelligence Unleashed: An argument for AI in Education. UCL Knowledge Lab: London, UK. https://www.pearson.com/content/dam/corporate/global/pearson-dot-com/files/innovation/Intelligence-Unleashed-Publication.pdf
23. Mishra, A. R., Rani, P., Pamucar, D., & Simic, V. (2024). Evaluation and prioritization of sustainable enterprise resource planning in SMEs using q-rung orthopair fuzzy rough set-based decision support model. IEEE Transactions on Fuzzy Systems. https://doi.org/10.1109/TFUZZ.2024.3374799.
24. Mishra, A. R., Pamucar, D., Rani, P., & Hezam, I. M. (2025). Single-Valued Neutrosophic Distance Measure-Based MEREC-RANCOM-WISP for Solving Sustainable Energy Storage Technology Problem. Cognitive Computation, 17(2), 87. https://doi.org/10.1007/s12559-025-10437-x.
25. Mondal, H., Mondal, S., & Juhi, A. (2025). Using artificial intelligence-based chatbots for manuscript preparation: summarising world association of medical editors guideline. Indian Journal of Rheumatology, 20(1), 89-91. https://doi.org/10.1177/09733698241278978.
26. Nikolić, I., Milutinović, J., Božanić, D., & Dobrodolac, M. (2023). Using an interval type-2 fuzzy AROMAN decision-making method to improve the sustainability of the postal network in rural areas. Mathematics, 11(14), 3105. https://doi.org/10.3390/math11143105.
27. Önden, A., Kara, K., Önden, İ., Yalçın, G. C., Simic, V., & Pamucar, D. (2024). Exploring the adoption of the metaverse and chat generative pre-trained transformer: A single-valued neutrosophic Dombi Bonferroni-based method for the selection of software development strategies. Engineering Applications of Artificial Intelligence, 133, 108378. https://doi.org/10.1016/j.engappai.2024.108378.
28. Özekenci, E. K. (2025). A Hybrid Mpsi-Extended Aroman Decision-Making Model for Assessing Green Logistics Performance: The Case of Asia-Pacific Countries. LogForum, 21(1), 87-105. https://doi.org/10.17270/J.LOG.001154.
29. Pishahang, M., Jovcic, S., Hashemkhani Zolfani, S., Simic, V., & Görçün, Ö. F. (2023). MCDM-based wildfire risk assessment: a case study on the state of Arizona. Fire, 6(12), 449. https://doi.org/10.3390/fire6120449.
30. Rani, P., Mishra, A. R., Cavallaro, F., & Alrasheedi, A. F. (2024). Location selection for offshore wind power station using interval-valued intuitionistic fuzzy distance measure-RANCOM-WISP method. Scientific Reports, 14(1), 4706. https://doi.org/10.1038/s41598-024-54929-6.
31. Rani, P., Mishra, A. R., Alshamrani, A. M., Alrasheedi, A. F., & Pamucar, D. (2025a). Adoption of Smart Manufacturing Technologies in Small and Medium Enterprises Using Picture Fuzzy Combinative Distance-Based Assessment Model. Journal of the Knowledge Economy, 1-41. https://doi.org/10.1007/s13132-025-02693-x.
32. Rani, P., Mishra, A. R., Alrasheedi, A. F., Pamucar, D., & Marinkovic, D. (2025b). Digital transformation project risks assessment using hybrid picture fuzzy distance measure-based additive ratio assessment method. Scientific Reports, 15(1), 3286. https://doi.org/10.1038/s41598-025-86598-4.
33. Reyes-Norambuena, P., Martinez-Torres, J., Nemati, A., Hashemkhani Zolfani, S., & Antucheviciene, J. (2024). Towards sustainable urban futures: Integrating a novel grey multi-criteria decision making model for optimal pedestrian walkway site selection. Sustainability, 16(11), 4437. https://doi.org/10.3390/su16114437.
34. Selvakumar, P., & Manjunath, T. C. (2025). AI in Text Paraphrasing. In Using AI Tools in Text Analysis, Simplification, Classification, and Synthesis (pp. 351-376). IGI Global Scientific Publishing. https://www.igi-global.com/chapter/ai-in-text-paraphrasing/369042.
35. Shekhovtsov, A., Kizielewicz, B., & Sałabun, W. (2025). Version [1.3]-[pymcdm–The universal library for solving multi-criteria decision-making problems]. SoftwareX, 102051. https://doi.org/10.1016/j.softx.2025.102051.
36. Shumanov, M., & Johnson, L. (2021). Making conversations with chatbots more personalized. Computers in Human Behavior, 117, 106627. https://doi.org/10.1016/j.chb.2020.106627.
37. Stone, P., Brooks, R., Brynjolfsson, E., Calo, R., Etzioni, O., Hager, G., ... & Teller, A. (2022). Artificial intelligence and life in 2030: the one hundred year study on artificial intelligence. arXiv preprint arXiv:2211.06318. https://doi.org/10.48550/arXiv.2211.06318.
38. Tripathi, D. K., Nigam, S. K., Rani, P., & Shah, A. R. (2023). New intuitionistic fuzzy parametric divergence measures and score function-based CoCoSo method for decision-making problems. Decision Making: Applications in Management and Engineering, 6(1), 535-563. https://doi.org/10.31181/dmame0318102022t.
39. Trung, D. D., Ašonja, A., Van Duc, D., Bao, N. C., & Son, N. H. (2024). Comparison of RAWEC and AROMAN Methods in Material Selection for Manufacturing or Maintenance. In International Conference on Organization and Technology of Maintenance (pp. 190-200). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-80597-4_15.
40. Wangsa, K., Karim, S., Gide, E., & Elkhodr, M. (2024). A systematic review and comprehensive analysis of pioneering AI chatbot models from education to healthcare: ChatGPT, Bard, Llama, Ernie and Grok. Future Internet, 16(7), 219. https://doi.org/10.3390/fi16070219.
41. Więckowski, J., Kizielewicz, B., Shekhovtsov, A., & Sałabun, W. (2023). RANCOM: A novel approach to identifying criteria relevance based on inaccuracy expert judgments. Engineering Applications of Artificial Intelligence, 122, 106114. https://doi.org/10.1016/j.engappai.2023.106114.
42. Więckowski, J., Wątróbski, J., & Sałabun, W. (2024). Inaccuracies in expert judgment: Comparative analysis of RANCOM and AHP methods in housing location selection problem. IEEE Access. https://doi.org/10.1109/ACCESS.2024.3443451.
43. Wu, T. T., Hapsari, I. P., & Huang, Y. M. (2025). Effects of incorporating AI chatbots into think–pair–share activities on EFL speaking anxiety, language enjoyment, and speaking performance. Computer Assisted Language Learning, 1-39. https://doi.org/10.1080/09588221.2025.2478271.
44. Xu, Z. (2007). Intuitionistic fuzzy aggregation operators. IEEE Transactions on fuzzy systems, 15(6), 1179-1187. https://doi.org/10.1109/TFUZZ.2006.890678.
45. Van Dua, T., Trung, D. D., Giang, N. T. P., & Van Duc, D. (2024). Comparison of Two Methods: RAM and AROMAN. In International Conference on Sustainability and Emerging Technologies for Smart Manufacturing (pp. 727-735). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-97-7083-0_73.
46. Zeng, S., Chen, S. M., & Kuo, L. W. (2019). Multiattribute decision making based on novel score function of intuitionistic fuzzy values and modified VIKOR method. Information Sciences, 488, 76-92. https://doi.org/10.1016/j.ins.2019.03.018.
47. Zhai, C., & Wibowo, S. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence, 4, 100134. https://doi.org/10.1016/j.caeai.2023.100134.