Usability of Chatgpt in Human-Computer Interaction and Its Relationship With Artificial Intelligence Attitude

Abstract

This study examined the usability of ChatGPT, an AI-powered chatbot, and university students’ perceptions of its usability. It also investigated the relationship between participants’ views on ChatGPT’s usability and their general attitudes toward it. Scenario-based tasks were assigned to participants, and the data were analyzed using task analysis, user analysis, descriptive statistics, and correlation analysis. The findings revealed that ChatGPT performs well in areas such as information access, language translation, and academic content generation; however, its success rates decrease in more complex tasks such as system settings, data management, and sharing. Descriptive analyses indicated that participants generally perceived the system’s accessibility positively. Correlation analyses showed a strong and significant relationship between the system’s usability and positive attitudes toward artificial intelligence. Overall, the findings suggest that ChatGPT is functional for users; however, improvements are needed, particularly in contextual tracking and personalization.

Keywords

• Chatbot
• ChatGPT
• Usability
• Human-Computer Interaction
• General Attitude Towards Artificial Intelligence

İnsan-Bilgisayar Etkileşiminde Chatgpt'nin Kullanılabilirliği ve Yapay Zeka Tutumu ile İlişkisi

Abstract

Bu çalışma, yapay zekâ destekli bir sohbet robotu olan ChatGPT’nin kullanılabilirliğini ve üniversite öğrencilerinin bu kullanılabilirliğe ilişkin algılarını incelemiştir. Ayrıca, katılımcıların ChatGPT’nin kullanılabilirliğine yönelik görüşleri ile genel tutumları arasındaki ilişki ele alınmıştır. Katılımcılara senaryo temelli görevler verilmiş ve veriler; görev analizi, kullanıcı analizi, betimleyici istatistikler ve korelasyon analizi kullanılarak incelenmiştir. Bulgular, ChatGPT’nin bilgiye erişim, dil çevirisi ve akademik içerik üretimi gibi alanlarda iyi performans gösterdiğini; ancak sistem ayarları, veri yönetimi ve paylaşım gibi daha karmaşık görevlerde başarı oranlarının düştüğünü ortaya koymuştur. Betimleyici analizler, katılımcıların sistemin erişilebilirliğini genel olarak olumlu algıladığını göstermiştir. Korelasyon analizleri, sistemin kullanılabilirliği ile yapay zekâya yönelik olumlu tutumlar arasında güçlü ve anlamlı bir ilişki olduğunu göstermiştir. Bulgular, ChatGPT’nin kullanıcılar için işlevsel olduğunu; ancak özellikle bağlamsal izleme ve kişiselleştirme açısından geliştirmelere ihtiyaç duyulduğunu ortaya koymaktadır.

Keywords

• Sohbet robotu
• ChatGPT
• Kullanılabilirlik
• İnsan-Bilgisayar Etkileşimi
• Yapay Zekaya Karşı Genel Tutum

References

1. Akkaya, N., & Şengül, L. (2023). Sohbet robotları (chatbots) ve yabancı dil eğitimi. Buca Eğitim Fakültesi Dergisi, 56, 2988–2999. https://doi.org/10.53444/deubefd.1340781
2. Borsci, S., Malizia, A., Schmettow, M., van der Velde, F., Tariverdiyeva, G., Balaji, D., & Chamberlain, A. (2022). The chatbot usability scale: The design and pilot of a usability scale for interaction with AI-based conversational agents. Personal and Ubiquitous Computing, 26(1), 95–119. https://doi.org/10.1007/s00779-021-01582-9
3. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
4. Dix, A., Finlay, J., Abowd, G. D., & Beale, R. (2004). Human–computer interaction (3rd ed.). Pearson Education.
5. Eşitti, Ş. (2024). Yapay zekâ, deepfake (derin düzmece) ve dezenformasyon çağında gazetecilik. In Ş. Eşitti & S. Çoban (Eds.), Yapay zekâ çağında medya, kültür ve iletişim üzerine güncel araştırmalar (pp. 1–16). PA Paradigma Akademi Yayınları.
6. Faulkner, L. (2003). Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers, 35(3), 379–383. https://doi.org/10.3758/bf03195514
7. Janssen, A., Grützner, L., & Breitner, M. H. (2021). Why do chatbots fail? A critical success factors analysis. In Proceedings of the 42nd International Conference on Information Systems (ICIS 2021). AIS eLibrary. https://aisel.aisnet.org/icis2021/
8. Kahn, M. C. (2021). What influences attitudes about artificial intelligence adoption: Evidence from U.S. local officials. PLOS ONE, 16(10), e0257732. https://doi.org/10.1371/journal.pone.0257732

9. Kaya, F., Aydın, F., Schepman, A., Rodway, P., Yetişensoy, O., & Kaya, M. D. (2022). The roles of personality traits, AI anxiety, and demographic factors in attitudes toward artificial intelligence. International Journal of Human–Computer Interaction, 38(5), 497–514. https://doi.org/10.1080/10447318.2021.1958258
10. Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). Guilford Press.
11. Lan, Y., Li, X., Du, H., Lu, X., Gao, M., Qian, W., & Zhou, A. (2024). Survey of natural language processing for education: Taxonomy, systematic review, and future trends. arXiv. https://doi.org/10.48550/arXiv.2401.07518
12. Li, Y. (2024). Usability of ChatGPT in second language acquisition: Capabilities, effectiveness, applications, challenges, and solutions. Studies in Applied Linguistics & TESOL, 23(1), 24–37. https://doi.org/10.52214/salt.v24i1.12864
13. Liu, J. (2024). ChatGPT: Perspectives from human–computer interaction and psychology. Frontiers in Artificial Intelligence, 7, 1418869. https://doi.org/10.3389/frai.2024.1418869
14. Nielsen, J. (1994). Usability engineering. Academic Press.
15. OpenAI. (2022, November 30). ChatGPT: Optimizing language models for dialogue. https://openai.com/blog/chatgpt
16. Özsancaktar, O., & Karaca, M. F. (2024). Sohbet robotlarının (chatbot) kullanım alanları ve potansiyelleri. In R. Başar (Ed.), Yönetim bilişim sistemleri (pp. 20–50). Yaz Yayınları.
17. Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed.). McGraw-Hill.
18. Parasuraman, A., & Colby, C. L. (2015). An updated and streamlined technology readiness index: TRI 2.0. Journal of Service Research, 18(1), 59–74. https://doi.org/10.1177/1094670514539730
19. Rhee, C. S., & Rhee, H. (2019). Expectations and anxieties affecting attitudes toward artificial intelligence revolution. Journal of the Korea Contents Association, 19(9), 37–46. https://doi.org/10.5392/JKCA.2019.19.09.037
20. Saha, D., & Mandal, A. (2015). User interface design issues for easy and efficient human–computer interaction: An explanatory approach. International Journal of Computer Sciences and Engineering, 3(10), 127–135.
21. Sallam, M., Elsayed, W., Al-Shorbagy, M., Alwan, N., Hallit, S., & Malaeb, D. (2024). ChatGPT usage and attitudes are driven by perceptions of usefulness, ease of use, risks, and psycho-social impact: A study among university students in the UAE. Frontiers in Education, 9, 1356486. https://doi.org/10.3389/feduc.2024.1414758
22. Schepman, A., & Rodway, P. (2020). Initial validation of the general attitudes towards artificial intelligence scale. Computers in Human Behavior Reports, 1, 100014. https://doi.org/10.1016/j.chbr.2020.100014
23. Sevli, O. (2023). ChatGPT’nin eğitim alanında kullanımı. 4. Uluslararası Karadeniz Modern Bilimsel Araştırmalar Kongresi Bildiri Kitabı (ss. 59–64).
24. Shneiderman, B., & Plaisant, C. (2010). Designing the user interface: Strategies for effective human–computer interaction (5th ed.). Addison-Wesley.
25. Stahl, B. C., Schroeder, D., & Rodrigues, R. (2022). The ethics of artificial intelligence: An introduction. In T. Metzinger (Ed.), Ethics of artificial intelligence: Case studies and options for addressing ethical challenges (pp. 3–28). Springer. https://doi.org/10.1007/978-3-031-17040-9_1
26. Tripathi, S., & Rosak-Szyrocka, J. (2024). Impact of artificial intelligence on society. CRC Press.
27. Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39(2), 273–315. https://doi.org/10.1111/j.1540-5915.2008.00192.x
28. Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186–204. https://doi.org/10.1287/mnsc.46.2.186.11926
29. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478. https://doi.org/10.2307/30036540
30. Yavuz, M., Kayalı, B., Balat, Ş., & Çalışan, M. (2023). Chatbot usability scale: Adaptation to Turkish and validation and reliability analysis. i-manager’s Journal of Educational Technology, 20(1), 11–19. https://doi.org/10.26634/jet.20.1.19758