Artificial Intelligence Literacy: An Adaptation Study

Year 2023, Volume: 4 Issue: 2, 291 - 306, 31.12.2023

Celalettin Çelebi , Fatih Yılmaz , Uğur Demir , Ferhat Karakuş

https://doi.org/10.52911/itall.1401740

Cited By: 4

Abstract

The purpose of this research is to adapt the Artificial Intelligence Literacy Scale (AILS) developed by Wang et al. (2022) into Turkish and study its validity and reliability. The scale aims to measure the artificial intelligence literacy levels of non-expert adults. The research data were gathered from 402 participants, and the researchers did Confirmatory Factor Analysis (CFA) to test the validity of the adapted scale, and to test the reliability, they adopted Cronbach’s alpha technique. The adapted scale consists of 12 items and 4 factors, as is the case in the original version. CFA results indicate that X^2/df =1.82, RMSEA = 0.04, RMR = 0.03, NFI = 0.95, CFI = 0.98, GFI = 0.96 and AGFI = 0.94. Considering CFA results, it is concluded that the adapted scale is a good fit. As for reliability, as far as the factors are concerned, the internal consistency results are 0.72, 0.74, 0.76, and 0.72, respectively. Additionally, α=0.85 for the whole scale. Consideringly, the scale and its factors are adequately reliable, and the adapted scale can be used in Turkish culture.

Keywords

Artificial Intelligence, AI literacy, Digital literacy, AI literacy scale

Yapay Zeka Okuryazarlığı: Bir Ölçek Uyarlama Çalışması

Abstract

Bu çalışmada Wang ve diğerleri (2022) tarafından geliştirilmiş Yapay Zeka Okuryazarlık Ölçeği’ni Türkçe diline uyarlayarak güvenirlik ve geçerliliğinin incelenmesi amaçlanmıştır. Ölçek yapay zeka konusunda uzman olmayan yetişkin bireylerin yapay zeka okuryazarlık düzeylerini ölçmeyi amaçlamaktadır. Çalışma kapsamında 402 katılımcının oluşturduğu yetişkin bireylerden veri toplanmıştır. Ölçeğin geçerliliğini test etmek amacıyla doğrulayıcı faktör analizi yapılmıştır. Güvenirliği için ise Cronbach Alpha iç tutarlılık katsayısı hesaplanmıştır. Dört boyut ve 12 maddeden oluşan Yapay Zeka Okuryazarlığı Ölçeği için yapılan doğrulayıcı faktör analizinde; x2/df için 1.82, RMSEA için 0.04, RMR için 0.03, NFI için 0.95, CFI için 0.98, GFI için 0.96 ve AGFI için 0.94 değerlerine ulaşılmıştır. Elde edilen uyum indeksleri değeri sonucunda modelin iyi bir uyuma sahip olduğu ortaya konulmuştur. Güvenlik analizi için yapılan Cronbach’s Alpha iç tutarlılık katsayısının hesaplanmasında ölçeğin alt boyutları için sırasıyla 0.72, 0.74, 0.76, 0.72 değerlerine ulaşılmıştır. Ölçeğin tümü için 0.85 iç tutarlılık katsayısı hesaplanmıştır. Buna göre ölçeğin hem boyutları hem de tamamı için elde edilen değerler ölçeğin güvenirliğine yönelik yeterli kanıtlar sunmaktadır. Türkçe diline uyarlanan Yapay Zeka Okuryazarlık Ölçeği’nin, yapay zeka konusunda uzman olmayan yetişkin bireylerin yapay zeka okuryazarlık düzeylerini ölçmek için geçerli ve güvenilir bir ölçme aracı olduğu sonucuna ulaşılmıştır.

Keywords

Yapay Zeka, Yapay Zeka Okuryazarlığı, Dijital Okuryazarklık, Yapay Zeka Okuryazarlığı Ölçeği

References

• Akkaya, B., Özkan, A., & Özkan, H. (2021). Artificial intelligence anxiety (AIA) scale: adaptation to Turkish, validity and reliability study. Alanya Academic Review, 5(2), 1125 - 1146. https://doi.org/10.29023/alanyaakademik.833668
• Alpaydın, E. (2004). Introduction to machine learning. The MIT Press.
• Amirrudin, M., Nasution, K., & Supahar, S. (2020). Effect of variability on Cronbach alpha reliability in research practice. Jurnal Matematika, Statistika Dan Komputasi, 17(2), 223–230. https://doi.org/10.20956/jmsk.v17i2.11655
• Balfe, N., Sharples, S., & Wilson, J. R. (2018). Understanding Is Key: An Analysis of Factors Pertaining to Trust in a Real-World Automation System. Human Factors, 60(4), 477-495. https://doi.org/10.1177/0018720818761256
• Bollen, K. A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53(1), 605–634. https://doi.org/10.1146/annurev.psych.53.100901.135239
• Calvani, A., Cartelli, A., Fini, A., & Ranieri, M. (2008). Models and instruments for assessing digital competence at school. Journal of E-learning and Knowledge Society, 4(3), 183-193.
• Calvani, A., Fini, A., & Ranieri, M. (2009). Assessing digital competence in secondary education. Issues, models and instruments. M. Leaning içinde, Issues in information and media literacy: Education, practice and pedagogy (s. 153-172). Informing Science Press.
• Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., & Wienrich, C. (2023). MAILS - Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies. Computers in Human Behavior: Artificial Humans, 1(2). https://doi.org/10.1016/j.chbah.2023.100014
• Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen and J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
• Büyüköztürk, Ş., Akgün, Ö. E., Özkahveci, Ö., & Demirel, F. (2004). The validity and reliability study of the Turkish version of the motivated strategies for learning questionnaire. Educational Sciences: Theory & Practice, 4(2), 231–237.
• Cihangir Çankaya, Z. (2009). Autonomy support, basic psychological need satisfaction and subjective well-being: Self-determination theory. Turkish Psychological Counseling and Guidance Journal, 4(31), 23-31.
• Çelebi, C., Demir, U., & Karakuş, F. (2023). Artificial intelligence literacy: A systematic review. Journal of Necmettin Erbakan University Ereğli Faculty of Education, 5(2), 536-560. https://doi.org/10.51119/ereegf.2023.67
• Eshet, Y. (2004). Digital literacy: A conceptual framework for survival skills in the digital era. Journal of educational multimedia and hypermedia, 13(1), 93-106.
• Ferrari, A. (2012). Digital competence in practice: An analysis of frameworks. Publications Office of the European Union.
• Gapski, H. (2007). Some reflections on digital literacy. Proceedings of the 3rd International Workshop on Digital Literacy (s. 49-55). Crete, Greece: CEUR Workshop Proceedings. Retrieved from http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-310/paper05.pdf
• Elrehail, H. (2018). The relationship among leadership, innovation and knowledge sharing: A guidance for analysis. Data in Brief, 19, 128–133. https://doi.org/10.1016/j.dib.2018.04.138
• Ferikoğlu, D., & Akgün, E. (2022). An ınvestigation of teachers’ artificial intelligence awareness: A scale development study. Malaysian Online Journal of Educational Technology, 10(3), 215–231. https://doi.org/10.52380/mojet.2022.10.3.407
• Harrington, D. (2009). Confirmatory factor analysis. Oxford University Press.
• Gerbing, D. W., & Hamilton, J. G. (1996). Viability of exploratory factor analysis as a precursor to confirmatory factor analysis. Structural Equation Modeling: A Multidisciplinary Journal, 3(1), 62–72. https://doi.org/10.1080/10705519609540030
• Gillaspy, J. A. Jr. (1996). A primer on confirmatory factor analysis. The Annual Meeting of the Southwest Educational Research Association.
• Goretzko, D., Siemund, K., & Sterner, P. (2023). Evaluating model fit of measurement models in confirmatory factor analysis. Educational and Psychological Measurement, https://doi.org/10.1177/00131644231163813
• Grassini, S. (2023). Development and validation of the AI attitude scale (AIAS-4): a brief measure of general attitude toward artificial intelligence. Frontiers in Psychology, 14, 1191628. https://doi.org/10.3389/fpsyg.2023.1191628
• Gür, Y. E., Ayden, C., & Yücel, A. (2019). Effects to human resources managements of developments in artificial intelligence. International Journal of Economics and Administrative Sciences, 3(2), 137-158.
• Haenlein, M., & Kaplan, A. (2019). A brief history of artificial intelligence: On the past, present, and future of artificial intelligence. California Management Review, 61(4), 5-14. https://doi.org/10.1177/0008125619864925
• Hox, J. (2021). Confirmatory factor analysis. J. Barnes, & D. R. Forde içinde, The Encyclopedia of Research Methods in Criminology and Criminal Justice. John Wiley & Sons, Inc.
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
• Hurley, A. E., Scandura, T. A., Schriesheim, C. A., Brannick, M. T., Seers, A., Vandenberg, R. J., & Williams, L. J. (1997). Exploratory and confirmatory factor analysis: guidelines, issues, and alternatives. Journal of Organizational Behavior, 18(6), 667–683. https://doi.org/10.1002/(SICI)1099-1379(199711)18:6<667::AID-JOB874>3.0.CO;2-T
• IBM Corp. (2020). IBM SPSS Statistics for Windows (Version 27.0) [Computer software]. IBM Corp. Jöreskog, K. G., & Sörbom, D. (1993). LISREL 8: Structural Equation Modeling with the SIMPLIS Command Language. Scientific Software International.
• Jöreskog, K. G. & Sörbom, D. (2019). LISREL 8.80. Mooresville, Ind. : Scientific Software,
• Karpunina, E. K., Dedov, S. V., Kholod, M. V., Ponomarev, S. V., & Gorlova, E. A. (2020). Artificial intelligence and its impact on economic security: trends, estimates and forecasts. E. G. Popkova, & B. S. Sergi içinde, Scientific and Technical Revolution: Yesterday, Today and Tomorrow (p. 213-225). Springer. https://doi.org/10.1007/978-3-030-47945-9_23
• Kalafat, S. (2012). The influence of personality traits of high school teachers on their teacher qualifications. Journal of Higher Education and Science, 2(3), 193. https://doi.org/10.5961/jhes.2012.050
• Karagöz, B., & İrge, N. T. (2023). The effect of personnel empowerment on organizational commitment and individual work performance: A research in the health sector. Eurasian Academy of Sciences Social Sciences Journal, (48), 46-53.
• Kaya, F., Aydın, F., Schepman, A., Rodway, P., Yetişensoy, O., & Kaya, M. D. (2022). The roles of personality traits, AI anxiety, and demographic factors in attitudes toward artificial intelligence. International Journal of Human–Computer Interaction. https://doi.org/10.1080/10447318.2022.2151730
• Kieslich, K., Lünich, M. & Marcinkowski, F. The Threats of Artificial Intelligence Scale (TAI). Int J of Soc Robotics 13, 1563–1577 (2021). https://doi.org/10.1007/s12369-020-00734-w
• Kim, S. W., & Lee, Y. (2022). The Artificial Intelligence Literacy Scale for Middle School Students., 27(3), 225-238. https://doi.org/10.9708/jksci.2022.27.03.225
• Kurudayıcıoğlu, M., & Tüzel, S. (2010). The types of literacy of the 21st century, changing text comprehension and Turkish teaching. Türklük Bilimi Araştırmaları, 28, 281-298.
• Laupichler, M. C., Aster, A., & Raupach, T. (2023a). Delphi study for the development and preliminary validation of an item set for the assessment of non-experts' AI literacy. Computers and Education: Artificial Intelligence, 4. https://doi.org/10.1016/j.caeai.2023.100126
• Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023b). Development of the “Scale for the assessment of non-experts’ AI literacy” – An exploratory factor analysis. Computers in Human Behavior Reports, 12. https://doi.org/10.1016/j.chbr.2023.100338
• Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410-425. https://doi.org/10.3390/educsci13040410
• Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Honolulu, USA: Association for Computing Machinery. https://doi.org/10.1145/3313831.3376727
• McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021). Artificial intelligence explainability: the technical and ethical dimensions. Philosophical Transactions: Mathematical, Physical and Engineering Sciences, 379(2207), Article 20200363. Advance online publication. https://doi.org/10.1098/rsta.2020.0363
• Mohajan, H. K. (2017). Two criteria for good measurements in research: Validity and reliability. Annals of Spiru Haret University. Economic Series, 17(4), 59–82. https://doi.org/10.26458/1746
• Muenjohn, Dr. N., & Armstrong, Prof. A. (2008). Evaluating the structural validity of the Multifactor Leadership Questionnaire (MLQ), capturing the leadership factors of transformational-transactional leadership. Contemporary Management Research, 4(1). https://doi.org/10.7903/cmr.704
• Mulaik, S. A., James, L. R., Van Alstine, J., Bennett, N., Lind, S., & Stilwell, C. D. (1989). Evaluation of goodness-of-fit indices for structural equation models. Psychological Bulletin, 105(3), 430–445. https://doi.org/10.1037/0033-2909.105.3.430
• Öztürk, K., & Şahin, M. E. (2018). A general view of artificial neural networks and artificial intelligence. Takvim-i Vekayi, 6(2), 25-36.
• Panayides, P. (2013). Coefficient alpha: Interpret with caution. Europe’s Journal of Psychology, 9(4), 687–696. https://doi.org/10.5964/ejop.v9i4.653
• Pedroso, R., Zanetello, L., Guimarães, L., Pettenon, M., Gonçalves, V., Scherer, J., Kessler, F., & Pechansky, F. (2016). Confirmatory factor analysis (CFA) of the Crack Use Relapse Scale (CURS). Archives of Clinical Psychiatry (São Paulo), 43(3), 37–40. https://doi.org/10.1590/0101-60830000000081
• Pinski, M., & Benlian, A. (2023). AI literacy-towards measuring human competency in artificial intelligence. https:// scholarspace.manoa.hawaii.edu/handle/10125/102649
• Precedence Research. (2023). Artificial intelligence (AI) market. https://www.precedenceresearch.com/artificial-intelligence-market#:~:text=The%20global%20artificial%20intelligence%20(AI,19%25%20from%202023%20to%202032.
• Rosemann, A., & Zhang, X. (2022). Exploring the social, ethical, legal, and responsibility dimensions of artificial intelligence for health-a new column in Intelligent Medicine. Intelligent Medicine, 2(2), 103-109. https://doi.org/10.1016/j.imed.2021.12.002
• Schepman, A., & Rodway, P. (2020). Initial validation of the general attitudes towards Artificial Intelligence Scale. Computers in Human Behavior Reports, 1, 1-18. https://doi.org/10.1016/j.chbr.2020.100014
• Schermelleh-Engel, K., & Moosbrugger, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74.
• Suh, W., & Ahn, S. (2022). Development and Validation of a Scale Measuring Student Attitudes Toward Artificial Intelligence. SAGE Open, 12(2). https://doi.org/10.1177/21582440221100463
• Taasoobshirazi, G., & Wang, S. (2016). The performance of the SRMR, RMSEA, CFI, and TLI: An examination of sample size, path size, and degrees of freedom. Journal of Applied Quantitative Methods, 11(3), 31–39.
• Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
• Terzi, R. (2020). An adaptation of Artificial Intelligence Anxiety Scale into Turkish: Reliability and validity study, International Online Journal of Education and Teaching, 7(4), 1501-1515.
• Thanasegaran, G. (2009). Reliability and validity issues in research. Integration and Dissemination Research Bulletin, 4, 35–40.
• Topal, Ç. (2017). Sociological imagination of Alan Turing: artificial intelligence as social imaginary. Ankara University Journal of the Faculty of Languages and History-Geography, 57(2), 1340-1364. https://doi.org/10.1501/Dtcfder_0000001565
• Turing, A. (1950). Computing machinery and intelligence. Mind, LIX(236), 433-460. https://doi.org/10.1093/mind/LIX.236.433
• Yavuz Aksakal, N., & Ülgen, B. (2021). Artificial intelligence and jobs of the future. TRT Akademi, 6(13), 834-852. https://doi.org/10.37679/trta.969285
• Wang, B., Rau, P.-L. P., & Yuan, T. (2022). Measuring user competence in using artificial intelligence: validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology, 42(9), 1324–1337. https://doi.org/10.1080/0144929X.2022.2072768
• Wilson, M., Scalise, K., & Gochyyev, P. (2015). Rethinking ICT literacy: From computer skills to social network settings. Thinking Skills and Creativity, 18, 65-80. https://doi.org/10.1016/j.tsc.2015.05.001
• Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. https://doi.org/10.3758/s13428-018-1055-2