DRM LISTS CREATED WITH CHATGPT: ANALYSING RECOGNITION MEMORY WITH CLIMATE-CHANGE THEMED LISTS

Year 2024, , 1 - 17, 30.07.2024

Gökhan Şahin

https://doi.org/10.14783/maruoneri.1363643

Abstract

Machine learning and natural language processing have led to the development of powerful language models such as ChatGPT, which can generate consistent and human-like responses to a wide range of queries. In many domains, ChatGPT provides appropriate responses to given commands. One of the aims of this study is to investigate the use of these association lists, such as the Deese-Roediger-McDermott (DRM) lists popular in cognitive psychology studies, by ChatGPT by giving the necessary instructions. The same method was then used to create association lists around a specific topic (climate change). The results of the first study showed that participants gave more false answers when discriminating whether critical words were presented during the test phase than when related and unrelated words were presented. This finding shows that DRM lists generated by ChatGPT can be used to search for memory errors. In line with the literature, false answers for critical words were predominantly rated as ‘remember’. The results of the second study, which was applied to the lists created on the topic of climate change and compared the responses of the groups with the climate denial scores, show that there is no significant difference in the emergence of false memories between the two groups. The level of climate change denial did not significantly affect the participants’ responses to the critical words in the climate- related lists. The low level of climate denial in the sample is a limitation of this study. It is recommended that future studies compare memory performance across an appropriate sample.

Keywords

DRM, False memory, ChatGPT, Climate change, Remember/know

CHATGPT İLE OLUŞTURULAN DRM LİSTELERİ: İKLİM DEĞİŞİKLİĞİ TEMALI LİSTELERLE TANIMA BELLEĞİ ANALİZİ

Abstract

Makine öğrenimi ve doğal dil işleme, ChatGPT gibi çok çeşitli istemlere karşılık tutarlı ve insan benzeri yanıtlar üretebilen güçlü dil modellerinin geliştirilmesine yol açmıştır. Birçok alanda, ChatGPT verilen komutlara uygun yanıtlar sağlamaktadır. Bu çalışmanın amaçlarından biri, bilişsel psikolojide bellek yanılgıları çalışmalarında popüler olan Deese-Roediger-McDermott (DRM) listelerinin benzerlerinin, gerekli yönergeler verilerek ChatGPT tarafından oluşturulması ile bu çağrışım listelerini kullanarak bellek yanılgılarını incelemektir. Devamında aynı yöntem belirli bir tema çerçevesinde (iklim değişikliği) çağrışım listelerinin oluşturulması için kullanılmıştır. İlk çalışmanın sonuçları, katılımcıların test aşamasında kritik kelimelerin sunulup sunulmadığını ayırt etmekte ilişkili ve ilişkisiz kelimelere kıyasla daha fazla yanlış yanıt verdiklerini ortaya koymuştur. Bu bulgu ChatGPT ile oluşturulan DRM listelerinin de bellek yanılgılarını araştırmaya imkan verdiğini göstermektedir. Bununla birlikte kritik kelimeler için verilen yanlış yanıtlar, literatürle uyumlu şekilde, ağırlıklı olarak ‘hatırlıyorum’ şeklinde değerlendirilmiştir. İklim değişikliği temasında hazırlanan listelerle uygulanan ve iklim inkarı puanları ile gruplanan kişilerin yanıtlarının karşılaştırıldığı ikinci çalışmanın sonuçları, iki grup arasında yanlış yanıtların ortaya çıkmasında anlamlı bir fark olmadığını göstermektedir. İklim değişikliğini reddetme düzeyi, katılımcıların iklim temalı listelerdeki kritik kelimelere verdikleri yanıtları anlamlı düzeyde etkilememiştir. Örneklemin genelinde iklim inkarının düşük düzeyde olması bu çalışmanın bir sınırlılığıdır. İleride yapılacak çalışmalarda uygun örneklem üzerinden bellek performansının karşılaştırılması önerilmektedir.

Keywords

Bellek yanılması, İklim değişikliği, Hatırlıyorum/Biliyorum, DRM, ChatGPT

References

• Alshater, Muneer, Exploring the Role of Artificial Intelligence in Enhancing Academic Performance: A Case Study of ChatGPT (December 26, 2022). Available at SSRN: https://ssrn.com/abstract=4312358 or http://dx.doi.org/10.2139/ssrn.4312358.
• Aydın, Ö., Karaarslan, E. (2022). OpenAI ChatGPT Generated Literature Review: Digital Twin in Healthcare . In Ö. Aydın (Ed.), Emerging Computer Technologies 2 (pp. 22- 31). İzmir Akademi Derneği.
• Binz, M. & Schulz, E. (2022). Using cognitive psychology to understand GPT-3. 10.31234/osf.io/6dfgk.
• Brennen, T., Dybdahl, R., & Kapidžić, A. (2007, December). Trauma-related and neutral false memories in war- induced Posttraumatic Stress Disorder. Consciousness and Cognition, 16(4), 877–885. https://doi.org/10.1016/j.concog.2006.06.012.
• Buchanan, L., Brown, N. R., Cabeza, R., and Maitson, C. (1999). False memories and semantic lexicon arrangement. Brain Lang. 68, 172–177. doi: 10.1006/brln.1999.2072.
• Cann, D. R.,McRae, K., and Katz, A. N. (2011). False recall in the Deese-Roediger-McDermott paradigm: The roles of gist and associative strength. Q. J. Exp.Psychol. 64, 1515–1542. doi: 10.1080/17470218.2011.560272.
• ChatGPT (2022). ChatGPT: Optimizing Language Models for Dialogue. OpenAI. Retrieved from https://openai.com/blog/chatgpt/ , Access Date: 16.12.2022.
• Coane JH, McBride DM, Huff MJ, Chang K, Marsh EM and Smith KA (2021) Manipulations of List Type in the DRM Paradigm: A Review of How Structural and Conceptual Similarity Affect False Memory. Front. Psychol. 12:668550. doi: 10.3389/fpsyg.2021.668550.
• Dasse, M. N., Juback, S. K., Morissette, S. B., Dolan, S. L., & Weaver, C. A. (2015, September 28). False Memory Susceptibility in OEF/OIF Veterans With and Without PTSD. Military Psychology, 27(6), 354–365. https://doi.org/10.1037/mil0000094.
• Drori, I., Zhang, S., Shuttleworth, R., Tang, L., Lu, A., Elizabeth, K., Liu, K., Chen, L., Tran, S., Cheng, N., Wang, R., Singh, N., Patti, T. L., Lynch, J., Shporer, A., Verma, N., Wu, E., & Strang, G. (2022, August 2). A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.2123433119.
• Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., . . . Wright, R. (2023, August). Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
• Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/ BF03193146.
• Gallo, D. A., Roberts, M. J., and Seamon, J. G. (1997). Remembering words not presented in lists: can we avoid creating false memories? Psycho. Bull. Rev. 4, 271–276. doi: 10.3758/BF03209405.
• Goodman, G. S., Ogle, C. M., Block, S. D., Harris, L. S., Larson, R. P., Augusti, E. M., Cho, Y. I., Beber, J., Timmer, S., & Urquiza, A. (2011). False memory for trauma-related Deese-Roediger-McDermott lists in adolescents and adults with histories of child sexual abuse. Development and psychopathology, 23(2), 423–438. https://doi.org/10.1017/S0954579411000150
• Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York: Wiley. Reprinted 1974 by Krieger, Huntington, NY.
• Gunter, R. W., Bodner, G. E., & Azad, T. (2007). Generation and mnemonic encoding induce a mirror effect in the DRM paradigm. Memory & Cognition, 35, 1083–1092.
• Häkkinen, Kirsti, Nazar Akrami (2014). “Ideology and Climate Change Denial”. Personality and Individual Differences, S. 70, 62-65.
• Howe, M.L., Derbish, M.H., 2010. On the susceptibility of adaptive memory to false memory illusions. Cognition 115, 252–267.
• Huff, M. J., & Bodner, G. E. (2013). When does memory monitoring succeed versus fail? Comparing item-specific and relational encoding in the DRM paradigm. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39(4), 1246–1256.
• Huff, M. J., Bodner, G. E., & Gretz, M. R. (2020). Reducing false recognition in the Deese-Roediger/McDermott paradigm: Related lures reveal how distinctive encoding improves encoding and monitoring processes. Frontiers in Psychology, 11, 602347.
• Huff, M. J., Coane, J. H., Hutchison, K. A., Grasser, E. B., and Blais, J. E. (2012). Interpolated task effects on direct and mediated false recognition: effects of initial recall, recognition, and the ironic effect of guessing. J. Exp. Psychol. Learn. Mem. Cognit. 38, 1720–1730. doi: 10.1037/a0028476.
• Israel, L., and Schacter, D. L. (1997). Pictorial encoding reduces false recognition of semantic associates. Psycho. Bull. Rev. 4, 557–581. doi: 10.3758/BF03214352.
• Jiao, W., Wang, W., Huang, J. T., Wang, X., Shi, S., & Tu, Z. (2023). Is ChatGPT A Good Translator? Yes With GPT- 4 As The Engine. arXiv.org. https://arxiv.org/abs/2301.08745.
• Knott, L. M., Dewhurst, S. A., and Howe, M. L. (2012). What factors underlie associative and categorical memory illusions? The roles of backward associative strength and interitem connectivity. J. Exp. Psychol. Learn. Mem. Cognit. 38,229–239. doi: 10.1037/a002520.
• Kıral Uçar, G., Gezici Yalçın, M., & Özdemir, G. (2019). İklim değişikliği inkarının sosyal baskınlık yönelimi ve ekolojik adil dünya inancı ile ilişkisi. Uludağ Üniversitesi Fen-Edebiyat Fakültesi Sosyal Bilimler Dergisi, 20(37), 739-764.
• Klepper, D. (2023), It turns out that ChatGPT is really good at creating online propaganda: ‘I think what’s clear is that in the wrong hands there’s going to be a lot of trouble. January 24, (https://fortune.com/2023/01/24/chatgpt-open-ai-online- propaganda/).
• Licklider, J. C. (1960). Man-computer symbiosis. IRE transactions on human factors in electronics, (1), 4-11.
• Maulina, D., Irwanda, D. Y., Sekarmewangi, T. H., Putri, K. M. H., & Otgaar, H. (2021). How accurate are memories of traffic accidents? Increased false memory levels among motorcyclists when confronted with accident-related word lists. Transportation Research Part F: Traffic Psychology and Behaviour, 80, 275–294. https://doi. org/10.1016/j.trf.2021.04.015.
• McCabe, D. P., Presmanes, A. G., Robertson, C. L., & Smith, A. D. (2004). Item-specific processing reduces false memories.Psychonomic Bulletin & Review, 11, 1074–1079.
• McCabe, D. P., & Smith, A. D. (2006). The distinctiveness heuristic in false recognition and false Recall. Memory, 14(5), 570–583
• Namias, J. M., Huff, M. J., Smith, A., & Maxwell, N. P. (2022). Drawing individual images benefits recognition accuracy in the Deese–Roediger–McDermott paradigm. Quarterly Journal of Experimental Psychology, 75, 1571–1582.
• Noever, D. (2020, August 2). The Chess Transformer: Mastering Play using Generative Language Models. arXiv.org. http://arxiv.org/abs/2008.04057
• Oliver, M. C., Bays, R. B., & Zabrucky, K. M. (2016). False memories and the DRM paradigm: Effects of imagery, list and test type. Journal of General Psychology, 143(1), 33–48.
• Park, L., Shobe, K. K., and Kihlstrom, J. F. (2005). Associative and categorical relations in the associative memory illusion. Psychol. Sci. 16, 792–797. doi: 10.1111/j.1467-9280.2005.01616.x.
• Payne, D. G., Elie, C. J., Blackwell, J. M., and Neuschatz, J. S. (1996). Memory illusions: recalling, recognizing, and recollecting events that never occurred. J. Mem. Lang. 35, 261–285. doi: 10.1006/jmla.1996.0015.
• Reed, L. (2022). ChatGPT for Automated Testing: From conversation to code. Sauce Labs. (https://saucelabs.com/blog/chatgpt-automated-testing-conversation-to-cod)Accessed: February 20, 2023
• Robin, F. (2010). Imagination and false memories. Imagination, Cognition, and Personality, 30, 407–423.
• Roediger, H. L., & McDermott, K. B. (1995a). Creating false memories: Remembering words not presented in lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(4), 803–814. https://doi.org/10.1037/0278-7393.21.4.803.
• Schacter, D. L., Koutstaal, W., and Norman, K. A. (1997). False memories and aging. Trends Cognit. Sci. 1, 229–236. doi: 10.1016/S1364-6613(97)01068-1.
• Smith, S. M., Gerkens, D. R., Pierce, B. H., and Choi, H. (2002). The roles of associative responses at study and semantically guided recollection at test in false memory: the Kirkpatrick and Deese hypotheses. J. Mem. Lang. 47, 436–447. doi: 10.1016/S0749-596X(02)00012-8.
• Şahin, G. (2022). False Memories and Cognitive Flexibility. Studia Psychologica, 64(3), 283-294. 0039-3320. DOI: https://doi.org/10.31577/sp.2022.03.854.
• Şahin, G., & Tekman, H. G. (2019). Sahte Hatıralar¬da Zorunlu Seçim Yöntemi ile Hatırlıyorum/ Bili¬yorum ve Modalite Etkileri. Psikoloji Çalışmaları/ Studies in Psychology, 39, 179–193. https://doi. org/10.26650/sp2019-0016 Tulving, E. (1985). Memory and consciousness. Canadian Psychology/Psychologie canadienne, 26(1), 1–12. https://doi.org/10.1037/h0080017.
• Tung, L. (2023). ChatGPT can write code. Now researchers say it’s good at fixing bugs, too. ZDNet. (https://www.zdnet.com/article/chatgpt-can-write-code-now-researc hers-say-its-good-at-fixing-bugs-too) Accessed: February 29, 2023
• Wagner, R. L., Lyon, B. A., & AuBuchon, A. (2022). False Memory for Words in Noise: An At-Home DeeseRoediger- McDermott (DRM) Experiment Across Adulthood. Psi Chi Journal of Psychological Research, 27(2), 154– 165. https://doi.org/10.24839/2325-7342.JN27.2.154
• Zwaan, R. A., Pecher, D., Paolacci, G., Bouwmeester, S., Verkoeijen, P., Dijkstra, K., et al. (2018). Participant nonnaiveté and the reproducibility of cognitive psychology. Psycho. Bull. Rev. 25, 1968–1972. doi: 10.3758/s13423-017-1348-y