“Trust, but Verify”: An Examination of Perceptions of Trust in AI and ChatGPT in Reddit User Comments

Öz

In recent years, the rapid rise of large language models, such as ChatGPT, has brought about profound changes in information access and content generation, while simultaneously rendering the concept of “trust” in these systems increasingly complex and contested. The growing presence of AI systems in everyday life—through a wide range of applications, services, and products—has further highlighted the significance of user trust and distrust toward artificial intelligence. This study aims to investigate the nature, boundaries, and construction of trust that users establish with AI, focusing on ChatGPT. To this end, user comments posted under relevant threads on the Reddit platform, which capture natural user experiences and discussions, were analysed using thematic analysis, a qualitative research method. The findings revealed eight themes pertaining to users’ perceptions of trust in AI and ChatGPT. The results indicate that trust in AI is conditional, context-dependent, and process-oriented. Users don’t perceive AI as an infallible source of knowledge; rather, they position it as an “assistive tool” that requires supervision, verification, and guidance. The analysis demonstrates that trust is not contingent upon the system’s perfection, but on the user’s capacity to monitor and validate its outputs. Notably, distrust predominates in high-risk domains, such as psychological support, whereas in contexts that demand efficiency and speed, users continue to utilize the system despite potential errors. Overall, the study highlights that users do not delegate full responsibility to AI; instead, they integrate these technologies into their lives with pragmatic scepticism, adhering to a “trust, but verify” principle.

Anahtar Kelimeler

• New communication technologies
• communication studies
• artificial intelligence
• ChatGPT
• trustworthiness

“Güven, Ama Kontrol Et”: Reddit Kullanıcı Yorumlarında Yapay Zeka ve ChatGPT’ye Güven Algısı Üzerine Bir İnceleme

Öz

Son yıllarda ChatGPT gibi büyük dil modellerinin hızlı yükselişi, bilgiye erişim ve içerik üretimi süreçlerinde köklü değişimlere yol açmış, diğer yandan bu sistemlere duyulan “güven” kavramını da tartışmalı hale getirmiştir. Yapay zeka sistemlerinin çeşitli uygulama, hizmet ve ürünler aracılığıyla gündelik hayatta artan kullanımı, kullanıcıların yapay zekaya duyduğu güven ve güvensizliğin önemini daha da görünür kılmaktadır. Bu çalışma, kullanıcıların yapay zeka ve özellikle ChatGPT ile kurdukları güven ilişkisinin doğasını, sınırlarını ve bu güvenin nasıl inşa edildiğini anlamayı amaçlamaktadır. Bu doğrultuda, kullanıcıların doğal deneyimlerini ve tartışmalarını barındıran Reddit platformundaki ilgili başlıklar altında yer alan kullanıcı yorumları, nitel araştırma yöntemlerinden tematik analiz tekniği kullanılarak incelenmiştir. Araştırma sonucunda, yapay zeka ve ChatGPT’ye yönelik güven algısıyla ilgili 8 tema ortaya çıkmıştır. Araştırma bulguları, kullanıcıların yapay zekaya yönelik güveninin koşullu, bağlamsal ve süreç odaklı olduğunu ortaya koymaktadır. Kullanıcılar, yapay zekayı mutlak doğru bilgi kaynağı olarak değil, denetlenmesi, doğrulanması ve yönlendirilmesi gereken bir “yardımcı araç” olarak konumlandırmaktadır. Analiz sonucunda, güvenin sistemin kusursuzluğuna değil, kullanıcının "kontrol etme" ve "doğrulama" kapasitesine bağlı olduğu belirlenmiştir. Özellikle psikolojik destek gibi riskli alanlarda güvensizlik hakimken, verimlilik ve hız gerektiren durumlarda risklere rağmen kullanımın sürdüğü görülmüştür. Sonuç olarak çalışma, kullanıcıların tüm sorumluluğu yapay zekaya devretmediğini, bunun yerine “güven, ama kontrol et” ilkesini benimseyerek teknolojiyi “pragmatik bir şüphecilikle” hayatlarına entegre ettiklerini göstermektedir.

Anahtar Kelimeler

• Yeni iletişim teknolojileri
• iletişim çalışmaları
• yapay zeka
• ChatGPT
• güvenilirlik

Kaynakça

1. Adams, B. D., Bruyn, L. E., Houde, S., Angelopoulos, P., Iwasa-Madge, K., & McCann, C. (2003). Trust in automated systems. Ministry of National Defence.
2. Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024). Trust in AI: progress, challenges, and future directions. Humanities and Social Sciences Communications, 11, Article 1568.
3. Appelman, A., & Sundar, S. S. (2016). Measuring message credibility: Construction and validation of an exclusive scale. Journalism & Mass Communication Quarterly, 93(1), 59–79.
4. Ashoori, M., & Weisz, J. D. (2019). In AI we trust? Factors that influence trustworthiness of AI infused decision making processes.
5. Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., Do, Q. V., Xu, Y., & Fung, P. (2023). A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. CoRR.
6. Bao, Y., Cheng, X., de Vreede, T., & de Vreede, G. J. (2021). Investigating the relationship between AI and trust in human AI collaboration. In Proceedings of the Annual Hawaii International Conference on System Sciences (Vol. 2020 January, pp. 607–616). IEEE Computer Society.
7. Borji, A. (2023). A categorical archive of ChatGPT failures.
8. Botsman, R. (2017). Who can you trust?: How technology brought us together–and why it could drive us apart. Penguin UK.

9. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
10. Braun, V., & Clarke, V. (2021). Thematic analysis: A practical guide. SAGE Publications.
11. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., … Amodei, D. (2020). Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in Neural Information Processing Systems (Vol. 33, pp. 1877–1901). Curran Associates, Inc.
12. Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., … Zaremba, W. (2021). Evaluating large language models trained on code.
13. Cheng, X., Fu, S., & Druckenmiller, D. (2016). Trust development in globally distributed collaboration: A case of U.S. and Chinese mixed teams. Journal of Management Information Systems, 33(4), 978–1007.
14. Choudhury, A., & Shamszare, H. (2023). Investigating the impact of user trust on the adoption and use of ChatGPT: Survey analysis. Journal of Medical Internet Research, 25.
15. Coeckelbergh, M. (2009). Virtual moral agency, virtual moral responsibility: On the moral significance of the appearance, perception, and performance of artificial agents. AI & Society, 24(2), 181–189.
16. Coeckelbergh, M. (2012). Can we trust robots? Ethics and Information Technology, 14(1), 53–60.
17. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.
18. Dwivedi, Y. K., et al. (2023). Opinion paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71.
19. Flanagin, A. J., & Metzger, M. J. (2000). Perceptions of Internet information credibility. Journalism & Mass Communication Quarterly, 77(3), 515–540.
20. French, B., Duenser, A., & Heathcote, A. (2018). Trust in automation (CSIRO Report EP184082). CSIRO.
21. Glikson, E., & Woolley, A. W. (2020). Human trust in artificial intelligence: Review of empirical research. Academy of Management Annals, 14(2), 627–660.
22. Hawley, K. (2014). Trust, Distrust and Commitment. Noûs (48), pp. 1-20.
23. Hoff, K. A., & Bashir, M. (2015). Trust in automation: Integrating discrete findings into a unified theoretical framework. Human Factors, 57(3), 407–434.
24. Huschens, M., Briesch, M., Sobania, D., & Rothlauf, F. (2023). Do you trust ChatGPT? – Perceived credibility of human and AI generated content (CoRR, abs/2309.02524).
25. Jackob, N. (2008). Credibility effects. In W. Donsbach (Ed.), International encyclopedia of communication (pp. 1044–1047). Blackwell Publishing Ltd.
26. Kasneci, E., et al. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274.
27. Komiak, S. Y. X., & Benbasat, I. (2006). The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS Quarterly, 30(4), 941–960.
28. Kozinets, R. V. (2015). Netnography: Redefined (2nd ed.). SAGE Publications. Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80.
29. Lockey, S., Gillespie, N., Holm, D., & Someh, I. A. (2021). A review of trust in artificial intelligence: Challenges, vulnerabilities and future directions. Hawaii International Conference on System Sciences.
30. Madsen, M., & Gregor, S. (2000). Measuring human-computer trust. In 11th Australasian Conference on Information Systems Vol. 53, pp. 6–8.
31. Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734.
32. McLeod, C. (2021). Trust. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Fall 2021).
33. Mollering, G. (2006). Trust: Reason, routine, reflexivity (1st ed.). Bingley: Emerald Group Publishing Limited.
34. Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56(1), 81–103.
35. Nickel, P. J., Franssen, M., & Kroes, P. (2010). Can we make sense of the notion of trustworthy technology? Knowledge, Technology & Policy, 23(3), 429–444.
36. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., …
37. Leike, J. (2022). Training language models to follow instructions with human feedback. In H. Larochelle, M.
38. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in Neural Information Processing Systems (Vol. 35).
39. Puri, A. and Keymolen, E. (2023). Of ChatGPT and trustworthy AI. Journal of Human Technology Relations, 1(1), pp.1-10.
40. Rotter, J. B. (1967). A new scale for the measurement of interpersonal trust. Journal of Personality, 35(4), 651–665.
41. Schepers, J., & Wetzels, M. (2007). A meta-analysis of the technology acceptance model: Investigating subjective norm and moderation effects. Information & Management, 44(1), 90–103.
42. Seeber, I., Bittner, E., Briggs, R. O., de Vreede, T., de Vreede, G. J., Elkins, A., Maier, R., Merz, A. B., Oeste-Reiß, S.,
43. Randrup, N., & Schwabe, G. (2020). Machines as teammates: A research agenda on AI in team collaboration. Information & Management, 57(2).
44. Shen, X., Chen, Z., Backes, M., & Zhang, Y. (2023). In ChatGPT we trust? Measuring and characterizing the reliability of ChatGPT.
45. Siau, K. (2018). Building trust in artificial intelligence, machine learning, and robotics. Cutter Consortium.
46. Siau, K., & Wang, W. (2018). Building trust in artificial intelligence, machine learning, and robotics. Cutter Business Technology Journal, 31(3), 47–53.
47. Sobania, D., Briesch, M., & Rothlauf, F. (2022). Choose your programming copilot: A comparison of the program synthesis performance of GitHub Copilot and genetic programming. In Proceedings of the Genetic and Evolutionary Computation Conference (pp. 1019–1027). ACM.
48. Sobania, D., Briesch, M., Hanna, C., & Petke, J. (2023). An analysis of the automatic bug fixing performance of ChatGPT. In 2023 IEEE/ACM International Workshop on Automated Program Repair (APR) (pp. 23–30). IEEE Computer Society.
49. Srinivasan, A. V. (2019). Developing a model for improving trust in artificial intelligence. Technology, Policy and Management, TU Delft.
50. Srivastava, S. C., & Chandra, S. (2018). Social presence in virtual world collaboration: An uncertainty reduction perspective using a mixed methods approach. MIS Quarterly, 42(3), 779–803.
51. Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. F. (2020). Learning to summarize from human feedback. CoRR.
52. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R.
53. Garnett (Eds.), Advances in Neural Information Processing Systems (Vol. 30). Curran Associates, Inc. Xu, R., Feng, Y. & Chen, H. (2023). ‘Chatgpt vs. google: A comparative study of search performance and user experience’.