Artificial Intelligence Literacy Scale: Latent Profile Analysis

Year 2024, Volume: 14 Issue: 3, 582 - 599

Ali Kırksekiz , Mehmet Yıldız , Mübin Kıyıcı , Metin Yıldız

Abstract

Artificial intelligence literacy is vital for individuals' adaptation to the future workforce and societal changes by enabling them to understand and effectively use AI technologies and critically evaluate their impact on society. In this study, the validity and reliability of the artificial intelligence literacy scale in Turkish language were tested and the latent profiles of the students were determined. This methodological study was carried out with a total of 729 students between December 2023 and February 2024. Validity and reliability analyses were conducted with SPSS 27 and AMOS 24, and latent profile analysis was handled with R programming language. According to the results of the CFA analysis of the Artificial Intelligence Literacy Scale, the fit indices were found to be significant (X²/sd= 3.832, RMSEA=.062, CFI=.949, AGFI=.933, GFI=.960, NFI=.949, TLI=.928, IFI=.916). Considering the Cronbach Alpha value of the scale consisting of 4 sub-dimensions and 12 items, the internal consistency coefficient was found to be 0.814. Since the lowest BIC value in the latent profile analysis was found in the VVV model, the VVV model was considered as the appropriate one in the study, and the class analyses were carried out through this model. With the LPA analysis, it was designated that the scale was divided into 3 classes. It was determined that the Artificial intelligence literacy scale is a valid and reliable measurement tool. After latent profile analysis, it was found out that the scale was divided into 3 classes.

Keywords

Artificial Intelligence, Literacy, Scale Adaptation, Latent Profile Analysis

References

• Akogul, S., & Erisoglu, M. (2017). An approach for determining the number of clusters in a model-based cluster analysis. Entropy, 19(9), 452. https://doi.org/10.3390/e19090452
• Ali, S., Payne, B. H., Williams, R., Park, H. W., & Breazeal, C. (2019). Constructionism, ethics, and creativity: Developing primary and middle school artificial intelligence education. In International workshop on education in artificial intelligence K-12 (EDUAI’19) (pp. 1–4).
• AL-Tkhayneh, K., Alghazo, E., & Tahat, D. (2023). The advantages and disadvantages of using artificial intelligence in education. Journal of Educational and Social Research, 13(4), 105. https://doi.org/10.36941/jesr-2023-0094
• Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411
• Bauer, J. (2022). A primer to latent profile and latent class analysis. In Methods for researching professional learning and development: Challenges, applications and empirical illustrations (pp. 243-268). Cham: Springer International Publishing.
• Brendel, A. B., Mirbabaie, M., Lembcke, T. B., & Hofeditz, L. (2021). Ethical management of artificial intelligence. Sustainability, 13(4), 1–18. https://doi.org/10.3390/su13041974.
• Bruderer, H. (2016). The Birth of Artificial Intelligence: First Conference on Artificial Intelligence in Paris in 1951?. In: Tatnall, A., Leslie, C. (eds) International Communities of Invention and Innovation. HC 2016. IFIP Advances in Information and Communication Technology, vol 491. Springer, Cham. https://doi.org/10.1007/978-3-319-49463-0_12
• Bryman, A., & Cramer, D. (2001). Quantitative data analysis with SPSS release 10 for Windows. Routledge.
• Büyüköztürk, Ş. (2002). Faktör analizi: Temel kavramlar ve ölçek geliştirmede kullanımı [Factor analysis: Basic concepts and use in scale development]. Kuram ve Uygulamada Eğitim Yönetimi, 32(32), 470-483. Retrieved from https://dergipark.org.tr/en/pub/kuey/issue/10365/126871
• Büyüköztürk, Ş., Çakmak, E., Akgün, Ö., Karadeniz, Ş., & Demirel, F. (2013). Bilimsel araştırma yöntemleri [Scientific research methods]. Ankara: Pegem Akademi Yayınları.
• Canbay, P., & Demircioğlu, Z. (2021). Endüstri 5.0’a doğru: zeki otonom sistemlerde etik ve ahlaki sorumluluklar [Towards Industry 5.0: ethics and moral responsibilities in intelligent autonomous systems]. Ajit-E Online Academic Journal of Information Technology, 12(45), 106-123. https://doi.org/10.5824/ajite.2021.02.006.x
• Cole, D. A. (1987). Utility of confirmatory factor analysis in test validation research. Journal of Consulting and Clinical Psychology, 55, 1019-1031. https://psycnet.apa.org/doi/10.1037/0022-006X.55.4.584
• Danry, V., Leong, J., Pataranutaporn, P., Tandon, P., Liu, Y., Shilkrot, R., ... & Sra, M. (2022, April). AI-Generated Characters: putting Deepfakes to Good Use. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (pp. 1-5). https://doi.org/10.1145/3491101.3503736
• Davenport, T., & Kalakota, R. (2019). The potential for artificial intelligence in healthcare. Future Healthcare Journal, 6(2), 94-98. https://doi.org/10.7861/futurehosp.6-2-94
• Defeng, Q., & Xiaojie, Q. (2020). Curriculum and teaching reform from the perspective of media history. Philosophy Study, 10(10). https://doi.org/10.17265/2159-5313/2020.10.005
• Drach, I., Petroye, O., Borodiyenko, O., Reheilo, I., Bazeliuk, O., Bazeliuk, N., & Slobodianiuk, O. (2023). The Use of Artificial Intelligence in Higher Education. International Scientific Journal of Universities and Leadership, 15, 66-82. https://doi.org/10.31874/2520-6702-2023-15-66-82
• Ece, N. (2024). Yapay zeka: deni̇zci̇li̇k sektöründe kullanimi ve swot anali̇zi̇ [Artificial intelligence: its use in the maritime industry and swot analysis]. Mersin Üniversitesi Denizcilik Ve Lojistik Araştırmaları Dergisi, 6(1), 30-51. https://doi.org/10.54410/denlojad.1491372
• Fornell, C., & Larcker, D. F. (1981). Structural equation models with unobservable variables and measurement error: Algebra and statistics. Journal of Marketing Research, 18(3), 382-388. https://doi.org/10.1177/002224378101800313
• Ferikoğlu, D., & Akgün, E. (2022). An Investigation of Teachers’ Artificial Intelligence Awareness: A Scale Development Study. Malaysian Online Journal of Educational Technology, 10(3), 215–231. https://doi.org/10.52380/mojet.2022.10.3.407
• Ferguson, S. L., Moore, E. W., & Hull, D. M. (2020). Finding latent groups in observed data: A primer on latent profile analysis in Mplus for applied researchers. International Journal of Behavioral Development, 44(5), 458-468. https://doi.org/10.1177/0165025419881721
• Garingan, D., & Pickard, A. (2021). Artificial intelligence in legal practice: exploring theoretical frameworks for algorithmic literacy in the legal information profession. Legal Information Management, 21(2), 97-117. https://doi.org/10.1017/s1472669621000190
• Hooper, D., Coughlan, J., & Mullen, M. (2008). Evaluating model fit: a synthesis of the structural equation modelling literature. In 7th European Conference on research methodology for business and management studies (pp. 195-200).
• Hornberger, M., Bewersdorff, A., & Nerdel, C. (2023). What do university students know about Artificial Intelligence? Development and validation of an AI literacy test. Computers and Education: Artificial Intelligence, 5, 100165. https://doi.org/10.1016/j.caeai.2023.100165
• Hwang, H. S., Zhu, L. C., & Cui, Q. (2023). Development and Validation of a Digital Literacy Scale in the Artificial Intelligence Era for College Students. KSII Transactions on Internet and Information Systems (TIIS), 17(8), 2241-2258. https://doi.org/10.3837/tiis.2023.08.016
• Ianculescu, M., Balog, A., Cristescu, I., Iordache, D. D., & Bajenaru, L. (2019). Latent profile analysis in health research: a case study. In 2019 22nd International Conference on Control Systems and Computer Science (CSCS) (pp. 649-654). IEEE.
• Katenova, M. (2024). Artificial intelligence and business school students’ performance. International Journal of Religion, 5(8), 96-101. https://doi.org/10.61707/6wjvxp71
• Khawlah, M., AL-Tkhayneh, Hasan, A., Al-Tarawneh, Enas, Said, Ali, Abulibdeh, Moath, Alomery. (2023). Social and Legal Risks of Artificial Intelligence: An Analytical Study. Academic Journal of Interdisciplinary Studies, 12(3), 308. https://doi.org/10.36941/ajis-2023-0079
• Kline, R. B. (2005). Principles and Practice of Structural Equation Modeling. New York: The Guilford Press.
• Kong, S. C., Cheung, W. M. Y., & Zhang, G. (2021). Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds. Computers and Education: Artificial Intelligence, 2, 100026. https://doi.org/10.1016/j.caeai.2021.100026
• Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the “scale for the assessment of non-experts’ AI literacy”–An exploratory factor analysis. Computers in Human Behavior Reports, 12, 100338.
• Law, N., Woo, D., de la Torre, J., & Wong, G. (2018). A global framework of reference on digital literacy skills for indicator 4.4.2. UNESCO Institute for Statistics. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000265403
• Liu, N., Shapira, P., & Yue, X. (2021). Tracking developments in artificial intelligence research: constructing and applying a new search strategy. Scientometrics, 126(4), 3153-3192. https://doi.org/10.1007/s11192-021-03868-4
• McMillan, C. T. (2021). Posthumanism in digital culture: Cyborgs, Gods and Fandom. Emerald Publishing Limited.
• McMillan, J. H., & Schumacher, S. (2006). Research in education: Evidence-based inquiry. Pearson.
• Minbaleev, A. (2022). The concept of "artificial intelligence" in law. Bulletin of Udmurt University Series Economics and Law, 32(6), 1094-1099. https://doi.org/10.35634/2412-9593-2022-32- 6-1094-1099
• Moloi, T., & Marwala, T. (2021). A High-Level Overview of Artificial Intelligence: Historical Overview and Emerging Developments. In Artificial Intelligence and the Changing Nature of Corporations. Springer, Cham. https://doi.org/10.1007/978-3-030-76313-8_2
• Munro, B. H. (2005). Statistical methods for health care research. Lippincott Williams & Wilkins.
• Muthén, B. O. (2001). Latent variable mixture modeling. In New developments and techniques in structural equation modeling (pp. 21-54). Psychology Press.
• Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535-569. https://doi.org/10.1080/10705510701575396
• Pastor, D. A., Barron, K. E., Miller, B. J., & Davis, S. L. (2007). A latent profile analysis of college students' achievement goal orientation. Contemporary Educational Psychology, 32(1), 8-47. https://doi.org/10.1016/j.cedpsych.2006.10.003
• Puspitaningsih, S., Irhadtanto, B., & Puspananda, D. (2022). The role of artificial intelligence in children's education for a digital future. Kne Social Sciences, 5th International Conference on Education and Social Science Research (ICESRE), 642-647. https://doi.org/10.18502/kss.v7i19.12483
• Rosenberg, J. M., van Lissa, C. J., Beymer, P. N., Anderson, D. J., Schell, M. J., & Schmidt, J. A. (2019). tidyLPA: Easily carry out latent profile analysis (LPA) using open-source or commercial software [R package]. Retrieved from https://data-edu.github.io/tidyLPA/
• Ruiz-Real, J., Uribe-Toril, J., Arriaza, J., & Valenciano, J. (2020). A look at the past, present and future research trends of artificial intelligence in agriculture. Agronomy, 10(11), 1839. https://doi.org/10.3390/agronomy10111839
• Scrucca, L., Fraley, C., Murphy, T. B., & Raftery, A. E. (2023). Model-based clustering, classification, and density estimation using mclust in R. Chapman and Hall/CRC.
• Seçer, E. (2024). Technostress levels of physiotherapy and rehabilitation students, related factors and awareness of the use of artificial intelligence in health: a cross-sectional study. Turkiye Klinikleri Journal of Health Sciences, 9(1), 127-136. https://doi.org/10.5336/healthsci.2023-100746
• Seçer, İ. (2015). Psikolojik test geliştirme ve uyarlama süreci: SPSS ve LISREL uygulamaları [Psychological test development and adaptation process: SPSS and LISREL applications]. Anı Yayıncılık.
• Seong-Won Kim, Youngjun Lee. (2022). The artificial intelligence literacy scale for middle school students. 국컴퓨터정보학회논문지, 27(3), 225-238. https://doi.org/10.9708/jksci.2022.27.03.225
• Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74. Retrieved from https://www.stats.ox.ac.uk/~snijders/mpr_Schermelleh.pdf
• Spurk, D., Hirschi, A., Wang, M., Valero, D., & Kauffeld, S. (2020). Latent profile analysis: A review and “how to” guide of its application within vocational behavior research. Journal of Vocational Behavior, 120, 103445. https://doi.org/10.1016/j.jvb.2020.103445
• Şimşek, Ö. F. (2007). Yapısal eşitlik modellemesine giriş: temel ilkeler ve LISREL uygulamaları [Introduction to structural equation modeling: basic principles and LISREL applications.]. Ekinoks.
• Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Allyn & Bacon.
• Vermunt, J. K. (2022). Latent class analysis. In International Encyclopedia of Education, Fourth Edition. Oxford: Elsevier.
• Voulgari, I., Zammit, M., Stouraitis, E., Liapis, A., & Yannakakis, G. (2021, June). Learn to machine learn: designing a game based approach for teaching machine learning to primary and secondary education students. In Proceedings of the 20th Annual ACM Interaction Design and Children Conference (pp. 593-598). https://doi.org/10.1145/3459990.3465176
• Wang, B., Rau, P., & Yuan, T. (2023). Measuring user competence in using artificial intelligence: Validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology, 42(9), 1324–1337. https://doi.org/10.1080/0144929x.2022.2072768
• Wang, W., & Siau, K. (2019). Artificial intelligence, machine learning, automation, robotics, future of work and future of humanity: A review and research agenda. Journal of Database Management, 30(1), 61–79. https://doi.org/10.4018/JDM.2019010104
• Wardenaar, K. J. (2021). Latent profile analysis in R: A tutorial and comparison to Mplus. Retrieved from https://psyarxiv.com/wzftr/download
• WHO (2015). Process of translation and adaptation of instruments. Retrieved from http://www.who.int/substance_abuse/research_tools/translation/en/
• Williams, R., Park, H., Oh, L., & Breazeal, C. (2019). Popbots: designing an artificial intelligence curriculum for early childhood education. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 9729-9736. https://doi.org/10.1609/aaai.v33i01.33019729
• Xu, F. (2023). The link between artificial intelligence and computer technology. Sixth International Conference on Intelligent Computing, Communication, and Devices (ICCD 2023). https://doi.org/10.1117/12.2683091
• Yin, R. K., & Moore, G. B. (1987). The use of advanced technologies in special education: Prospects from robotics, artificial intelligence, and computer simulation. Journal of Learning Disabilities, 20(1), 60-63. https://doi.org/10.1177/002221948702000111
• Yeşilyurt, S., & Çapraz, C. (2018). Ölçek geliştirme çalışmalarında kullanılan kapsam geçerliği için bir yol haritası [A roadmap for content validity used in scale development studies]. Erzincan Üniversitesi Eğitim Fakültesi Dergisi, 20(1), 251-264. https://doi.org/10.17556/erziefd.297741