Adaptation of Artificial Intelligence Literacy Scale: Latent Profile Analysis

Abstract

Artificial intelligence literacy is vital for individuals' adaptation to the future workforce and societal changes by enabling them to understand and effectively use AI technologies and critically evaluate their impact on society. In this study, the validity and reliability of the artificial intelligence literacy scale in Turkish language were tested and the latent profiles of the students were determined. This methodological study was carried out with a total of 729 students between December 2023 and February 2024. Validity and reliability analyses were conducted with SPSS 27 and AMOS 24, and latent profile analysis was handled with R programming language. According to the results of the CFA analysis of the Artificial Intelligence Literacy Scale, the fit indices were found to be significant (X²/sd= 3.832, RMSEA=.062, CFI=.949, AGFI=.933, GFI=.960, NFI=.949, TLI=.928, IFI=.916). Considering the Cronbach Alpha value of the scale consisting of 4 sub-dimensions and 12 items, the internal consistency coefficientwas found to be 0.814. Since the lowest BIC value in the latent profile analysis was found in the VVV model, the VVV model was considered as the appropriate one in the study, and the class analyses were carried out through this model. With the LPA analysis, it was designated that the scale was divided into 3 classes. It was determined that the Artificial intelligence literacy scale is a valid and reliable measurement tool. After latent profile analysis, it was found out that the scale was divided into 3 classes.

Keywords

• Artificial Intelligence
• Literacy
• Scale Adaptation
• Latent Profile Analysis

References

1. Akogul, S., & Erisoglu, M. (2017). An approach for determining the number of clusters in a model-based cluster analysis. Entropy, 19(9), 452. https://doi.org/10.3390/e19090452
2. Ali, S., Payne, B. H., Williams, R., Park, H. W., & Breazeal, C. (2019). Constructionism, ethics, and creativity: Developing primary and middle school artificial intelligence education. In International workshop on education in artificial intelligence K-12 (EDUAI’19) (pp. 1–4).
3. AL-Tkhayneh, K., Alghazo, E., & Tahat, D. (2023). The advantages and disadvantages of using artificial intelligence in education. Journal of Educational and Social Research, 13(4), 105. https://doi.org/10.36941/jesr-2023-0094
4. Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411
5. Bauer, J. (2022). A primer to latent profile and latent class analysis. In Methods for researching professional learning and development: Challenges, applications and empirical illustrations (pp. 243-268). Cham: Springer International Publishing.
6. Brendel, A. B., Mirbabaie, M., Lembcke, T. B., & Hofeditz, L. (2021). Ethical management of artificial intelligence. Sustainability, 13(4), 1–18. https://doi.org/10.3390/su13041974.
7. Bruderer, H. (2016). The Birth of Artificial Intelligence: First Conference on Artificial Intelligence in Paris in 1951?. In: Tatnall, A., Leslie, C. (eds) International Communities of Invention and Innovation. HC 2016. IFIP Advances in Information and Communication Technology, vol 491. Springer, Cham. https://doi.org/10.1007/978-3-319-49463-0_12
8. Bryman, A., & Cramer, D. (2001). Quantitative data analysis with SPSS release 10 for Windows. Routledge.

9. Büyüköztürk, Ş. (2002). Faktör analizi: Temel kavramlar ve ölçek geliştirmede kullanımı [Factor analysis: Basic concepts and use in scale development]. Kuram ve Uygulamada Eğitim Yönetimi, 32(32), 470-483. Retrieved from https://dergipark.org.tr/en/pub/kuey/issue/10365/126871
10. Büyüköztürk, Ş., Çakmak, E., Akgün, Ö., Karadeniz, Ş., & Demirel, F. (2013). Bilimsel araştırma yöntemleri [Scientific research methods]. Ankara: Pegem Akademi Yayınları.
11. Canbay, P., & Demircioğlu, Z. (2021). Endüstri 5.0’a doğru: Zeki otonom sistemlerde etik ve ahlaki sorumluluklar [Towards Industry 5.0: ethics and moral responsibilities in intelligent autonomous systems]. Ajit-E Online Academic Journal of Information Technology, 12(45), 106-123. https://doi.org/10.5824/ajite.2021.02.006.x
12. Cole, D. A. (1987). Utility of confirmatory factor analysis in test validation research. Journal of Consulting and Clinical Psychology, 55, 1019-1031. https://psycnet.apa.org/doi/10.1037/0022-006X.55.4.584
13. Danry, V., Leong, J., Pataranutaporn, P., Tandon, P., Liu, Y., Shilkrot, R., ... & Sra, M. (2022, April). AI-Generated Characters: Putting Deepfakes to Good Use. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (pp. 1-5). https://doi.org/10.1145/3491101.3503736
14. Davenport, T., & Kalakota, R. (2019). The potential for artificial intelligence in healthcare. Future Healthcare Journal, 6(2), 94-98. https://doi.org/10.7861/futurehosp.6-2-94
15. Defeng, Q., & Xiaojie, Q. (2020). Curriculum and teaching reform from the perspective of media history. Philosophy Study, 10(10). https://doi.org/10.17265/2159-5313/2020.10.005
16. Drach, I., Petroye, O., Borodiyenko, O., Reheilo, I., Bazeliuk, O., Bazeliuk, N., & Slobodianiuk, O. (2023). The Use of Artificial Intelligence in Higher Education. International Scientific Journal of Universities and Leadership, 15, 66-82. https://doi.org/10.31874/2520-6702-2023-15-66-82
17. Ece, N. (2024). Yapay zeka: Denı̇zcı̇lı̇k sektöründe kullanimi ve swot analı̇zı̇ [Artificial intelligence: its use in the maritime industry and swot analysis]. Mersin Üniversitesi Denizcilik ve Lojistik Araştırmaları Dergisi, 6(1), 30-51. https://doi.org/10.54410/denlojad.1491372
18. Fornell, C., & Larcker, D. F. (1981). Structural equation models with unobservable variables and measurement error: Algebra and statistics. Journal of Marketing Research, 18(3), 382-388. https://doi.org/10.1177/002224378101800313
19. Ferikoğlu, D., & Akgün, E. (2022). An Investigation of Teachers’ Artificial Intelligence Awareness: A Scale Development Study. Malaysian Online Journal of Educational Technology, 10(3), 215–231. https://doi.org/10.52380/mojet.2022.10.3.407
20. Ferguson, S. L., Moore, E. W., & Hull, D. M. (2020). Finding latent groups in observed data: A primer on latent profile analysis in Mplus for applied researchers. International Journal of Behavioral Development, 44(5), 458-468. https://doi.org/10.1177/0165025419881721
21. Garingan, D., & Pickard, A. (2021). Artificial intelligence in legal practice: Exploring theoretical frameworks for algorithmic literacy in the legal information profession. Legal Information Management, 21(2), 97-117. https://doi.org/10.1017/s1472669621000190
22. Hooper, D., Coughlan, J., & Mullen, M. (2008). Evaluating model fit: A synthesis of the structural equation modelling literature. In 7th European Conference on research methodology for business and management studies (pp. 195-200).
23. Hornberger, M., Bewersdorff, A., & Nerdel, C. (2023). What do university students know about Artificial Intelligence? Development and validation of an AI literacy test. Computers and Education: Artificial Intelligence, 5, 100165. https://doi.org/10.1016/j.caeai.2023.100165
24. Hwang, H. S., Zhu, L. C., & Cui, Q. (2023). Development and Validation of a Digital Literacy Scale in the Artificial Intelligence Era for College Students. KSII Transactions on Internet and Information Systems (TIIS), 17(8), 2241-2258. https://doi.org/10.3837/tiis.2023.08.016
25. Ianculescu, M., Balog, A., Cristescu, I., Iordache, D. D., & Bajenaru, L. (2019). Latent profile analysis in health research: A case study. In 2019 22nd International Conference on Control Systems and Computer Science (CSCS) (pp. 649-654). IEEE.
26. Katenova, M. (2024). Artificial intelligence and business school students’ performance. International Journal of Religion, 5(8), 96-101. https://doi.org/10.61707/6wjvxp71
27. AL-Tkhayneh, K. M., Al-Tarawneh, H. A., Abulibdeh, E. S. A., & Alomery, M. (2023). Social and Legal Risks of Artificial Intelligence: An Analytical Study. Academic Journal of Interdisciplinary Studies, 12(3), 308. https://doi.org/10.36941/ajis-2023-0079
28. Kline, R. B. (2005). Principles and Practice of Structural Equation Modeling. New York: The Guilford Press.
29. Kong, S. C., Cheung, W. M. Y., & Zhang, G. (2021). Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds. Computers and Education: Artificial Intelligence, 2, 100026. https://doi.org/10.1016/j.caeai.2021.100026
30. Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the “scale for the assessment of non-experts’ AI literacy”–An exploratory factor analysis. Computers in Human Behavior Reports, 12, 100338.
31. Law, N., Woo, D., de la Torre, J., & Wong, G. (2018). A global framework of reference on digital literacy skills for indicator 4.4.2. UNESCO Institute for Statistics. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000265403
32. Liu, N., Shapira, P., & Yue, X. (2021). Tracking developments in artificial intelligence research: Constructing and applying a new search strategy. Scientometrics, 126(4), 3153-3192. https://doi.org/10.1007/s11192-021-03868-4
33. McMillan, C. T. (2021). Posthumanism in digital culture: Cyborgs, Gods and Fandom. Emerald Publishing Limited.
34. McMillan, J. H., & Schumacher, S. (2006). Research in education: Evidence-based inquiry. Pearson.
35. Minbaleev, A. (2022). The concept of "artificial intelligence" in law. Bulletin of Udmurt University Series Economics and Law, 32(6), 1094-1099. https://doi.org/10.35634/2412-9593-2022-32- 6-1094-1099
36. Moloi, T., & Marwala, T. (2021). A High-Level Overview of Artificial Intelligence: Historical Overview and Emerging Developments. In Artificial Intelligence and the Changing Nature of Corporations. Springer, Cham. https://doi.org/10.1007/978-3-030-76313-8_2
37. Munro, B. H. (2005). Statistical methods for health care research. Lippincott Williams & Wilkins.
38. Muthén, B. O. (2001). Latent variable mixture modeling. In New developments and techniques in structural equation modeling (pp. 21-54). Psychology Press.
39. Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535-569. https://doi.org/10.1080/10705510701575396
40. Pastor, D. A., Barron, K. E., Miller, B. J., & Davis, S. L. (2007). A latent profile analysis of college students' achievement goal orientation. Contemporary Educational Psychology, 32(1), 8-47. https://doi.org/10.1016/j.cedpsych.2006.10.003
41. Puspitaningsih, S., Irhadtanto, B., & Puspananda, D. (2022). The role of artificial intelligence in children's education for a digital future. Kne Social Sciences, 5th International Conference on Education and Social Science Research (ICESRE), 642-647. https://doi.org/10.18502/kss.v7i19.12483
42. Rosenberg, J. M., van Lissa, C. J., Beymer, P. N., Anderson, D. J., Schell, M. J., & Schmidt, J. A. (2019). tidyLPA: Easily carry out latent profile analysis (LPA) using open-source or commercial software [R package]. Retrieved from https://data-edu.github.io/tidyLPA/
43. Ruiz-Real, J., Uribe-Toril, J., Arriaza, J., & Valenciano, J. (2020). A look at the past, present and future research trends of artificial intelligence in agriculture. Agronomy, 10(11), 1839. https://doi.org/10.3390/agronomy10111839
44. Scrucca, L., Fraley, C., Murphy, T. B., & Raftery, A. E. (2023). Model-based clustering, classification, and density estimation using mclust in R. Chapman and Hall/CRC.
45. Seçer, E. (2024). Technostress levels of physiotherapy and rehabilitation students, related factors and awareness of the use of artificial intelligence in health: A cross-sectional study. Turkiye Klinikleri Journal of Health Sciences, 9(1), 127-136. https://doi.org/10.5336/healthsci.2023-100746
46. Seçer, İ. (2015). Psikolojik test geliştirme ve uyarlama süreci: SPSS ve LISREL uygulamaları [Psychological test development and adaptation process: SPSS and LISREL applications]. Anı Yayıncılık.
47. Seong-Won Kim, Youngjun Lee. (2022). The artificial intelligence literacy scale for middle school students. 국컴퓨터정보학회논문지, 27(3), 225-238. https://doi.org/10.9708/jksci.2022.27.03.225
48. Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74. Retrieved from https://www.stats.ox.ac.uk/~snijders/mpr_Schermelleh.pdf
49. Spurk, D., Hirschi, A., Wang, M., Valero, D., & Kauffeld, S. (2020). Latent profile analysis: A review and “how to” guide of its application within vocational behavior research. Journal of Vocational Behavior, 120, 103445. https://doi.org/10.1016/j.jvb.2020.103445
50. Şimşek, Ö. F. (2007). Yapısal eşitlik modellemesine giriş: Temel ilkeler ve LISREL uygulamaları [Introduction to structural equation modeling: basic principles and LISREL applications.]. Ekinoks.
51. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Allyn & Bacon.
52. Vermunt, J. K. (2022). Latent class analysis. In International Encyclopedia of Education, Fourth Edition. Oxford: Elsevier.
53. Voulgari, I., Zammit, M., Stouraitis, E., Liapis, A., & Yannakakis, G. (2021, June). Learn to machine learn: Designing a game based approach for teaching machine learning to primary and secondary education students. In Proceedings of the 20th Annual ACM Interaction Design and Children Conference (pp. 593-598). https://doi.org/10.1145/3459990.3465176
54. Wang, B., Rau, P., & Yuan, T. (2023). Measuring user competence in using artificial intelligence: Validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology, 42(9), 1324–1337. https://doi.org/10.1080/0144929x.2022.2072768
55. Wang, W., & Siau, K. (2019). Artificial intelligence, machine learning, automation, robotics, future of work and future of humanity: A review and research agenda. Journal of Database Management, 30(1), 61–79. https://doi.org/10.4018/JDM.2019010104
56. Wardenaar, K. J. (2021). Latent profile analysis in R: A tutorial and comparison to Mplus. Retrieved from https://psyarxiv.com/wzftr/download
57. WHO (2015). Process of translation and adaptation of instruments. Retrieved from http://www.who.int/substance_abuse/research_tools/translation/en/
58. Williams, R., Park, H., Oh, L., & Breazeal, C. (2019). Popbots: Designing an artificial intelligence curriculum for early childhood education. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 9729-9736. https://doi.org/10.1609/aaai.v33i01.33019729
59. Xu, F. (2023). The link between artificial intelligence and computer technology. Sixth International Conference on Intelligent Computing, Communication, and Devices (ICCD 2023). https://doi.org/10.1117/12.2683091
60. Yin, R. K., & Moore, G. B. (1987). The use of advanced technologies in special education: Prospects from robotics, artificial intelligence, and computer simulation. Journal of Learning Disabilities, 20(1), 60-63. https://doi.org/10.1177/002221948702000111
61. Yeşilyurt, S., & Çapraz, C. (2018). Ölçek geliştirme çalışmalarında kullanılan kapsam geçerliği için bir yol haritası [A roadmap for content validity used in scale development studies]. Erzincan Üniversitesi Eğitim Fakültesi Dergisi, 20(1), 251-264. https://doi.org/10.17556/erziefd.297741