CAN ARTIFICIAL INTELLIGENCE LANGUAGE MODELS PERFORM FACT-CHECKING?
Öz
The aim of this study is to examine whether artificial intelligence (AI) language models can be effectively used as fact-checking tools and contribute to verification processes. The research was designed using a qualitative method with a case study approach, and thematic analysis was conducted through document analysis. ChatGPT and Gemini were selected using a similar sampling strategy. A total of 100 claims were examited 50 verified by Teyit.org concerning the Turkish agenda, and 50 verified by PolitiFact regarding the American and European agenda. Both AI models were asked to evaluate these claims based on the Truth-o-Meter rating scale. According to the findings, ChatGPT correctly identified 27 out of 50 claims related to Turkey, resulting in an accuracy rate of 52%, while Gemini correctly verified 38 claims, reaching 76%. For claims related to the Western agenda, ChatGPT again correctly answered 27 out of 50 (54%), while Gemini achieved 66% accuracy. Overall, Gemini demonstrated higher accuracy than ChatGPT across both regional contexts. These results indicate that although AI models can contribute to fact-checking efforts, their current performance is not yet reliable enough for standalone use in news verification. However, it is anticipated that with ongoing machine learning advancements and integration of real-time data, AI models will significantly improve. In the near future, such models could become dependable tools in the fact-checking ecosystem.
Anahtar Kelimeler
Artificial intelligence, Fact-checking, Allegation, News, ChatGPT, Gemini
Kaynakça
- Adair, B. (2017). Knight Foundation, Facebook and Craig Newmark provide funding to launch Duke Tech & Check Cooperative. Duke Reporters. https://sanford.duke.edu/story/knight-foundation-facebook-and-craig-newmark-provide-funding-launch-duke-tech-check/
- Alyukov, M. (2022). Making Sense of the News in an Authoritarian Regime: Russian Television Viewers’ Reception of the Russia–Ukraine Conflict. Europe-Asia Studies, 74(3), 337-359. https://ideas.repec.org//a/taf/ceasxx/v74y2022i3p337-359.html
- Augenstein, I., Baldwin, T., Cha, M., Chakraborty, T., Ciampaglia, G. L., Corney, D., DiResta, R., Ferrara, E., Hale, S., Halevy, A., Hovy, E., Ji, H., Menczer, F., Miguez, R., Nakov, P., Scheufele, D., Sharma, S., & Zagni, G. (2023). Factuality Challenges in the Era of Large Language Models. http://arxiv.org/abs/2310.05189
- Balmas, M. (2014). When fake news becomes real: Combined exposure to multiple news sources and political attitudes of inefficacy, alienation, and cynicism. Communication Research, 41(3), 430-454. https://doi.org/10.1177/0093650212453600
- Berger, B. K. (2001). Private Issues and Public Policy: Locating the Corporate Agenda in Agenda-Setting Theory. Journal of Public Relations Research, 13(2), 91-126. https://doi.org/10.1207/S1532754XJPRR1302_1
- Birks, J. (2019). Fact-Checking Journalism and Political Argumentation: A British Perspective. Springer International Publishing. https://doi.org/10.1007/978-3-030-30573-4
- Braun, V., & Clarke, V. (2019). Psikolojide tematik analizin kullanımı. Eğitimde Nitel Araştırmalar Dergisi- Journal of Qualitative Research in Education, 7(2), 873-898. https://doi.org/doi: 10.14689/issn.2148-2624.1.7c.2s.17m
- Cheng, Z., Golan ,Guy J., & and Kiousis, S. (2016). The Second-Level Agenda-Building Function of the Xinhua News Agency: Examining the role of government-sponsored news in mediated public diplomacy. Journalism Practice, 10(6), 744-762. https://doi.org/10.1080/17512786.2015.1063079
- DeVerna, M. R., Yan, H. Y., Yang, K.-C., & Menczer, F. (2023). Fact-checking information generated by a large language model can decrease news discernment. http://arxiv.org/abs/2308.10800
- PEN America., (2017, October 12) Faking News: Fraudulent News and the Fight for Truth. (2017, Ekim 12). PEN America. https://pen.org/report/faking-news/
