Development of Artificial Intelligence Awareness Scale (AIAS): Validity and Reliability Study
Öz
This study presents the development, validation, and reliability assessment of the Artificial Intelligence Awareness Scale (AIAS), specifically designed to evaluate university students' awareness of artificial intelligence (AI). Despite significant advances in AI technologies and their increasing integration into daily life and educational environments, there is currently no existing measurement instrument that comprehensively evaluates AI awareness among students, incorporates the most recent AI developments, and takes non-technical users into consideration. To address this gap, the present study employed an exploratory sequential mixed-methods design. An initial literature review and expert consultations guided the formation of a preliminary item pool, which was refined through qualitative evaluations. The finalized 20-item draft scale was administered to 361 university students enrolled in social science departments. Exploratory factor analysis (EFA) resulted in an 11-item scale with two distinct factors: "Literacy" (7 items, factor loadings ranging from .48 to .72) and "Threats" (4 items, factor loadings ranging from .56 to .70). Confirmatory factor analysis (CFA) confirmed the two-factor structure, yielding acceptable fit indices (RMSEA = .084, SRMR = .074, GFI = .93). Internal consistency reliability was acceptable (Cronbach's alpha = .64 overall; .67 for Literacy and .59 for Threats; McDonald’s Omega = .65 overall; .67 for Literacy and .61 for Threats), and test-retest stability over a four-week period was high (r = .80). Results demonstrate that AIAS is a valid and reliable instrument for measuring university students' AI awareness. The scale is particularly valuable for identifying specific areas where educational interventions are needed and informing policy decisions.
Keywords: Artificial Intelligence, Artificial Intelligence Awareness, Artificial Intelligence literacy, Artificial Intelligence Threats
Anahtar Kelimeler
Artificial Intelligence, Artificial Intelligence Awareness, Artificial Intelligence literacy, Artificial Intelligence Threats, Artificial Intelligence Awareness Scale
Destekleyen Kurum
Proje Numarası
Etik Beyan
Kaynakça
- Ahmadi Fatalaki, J., Saeedi, Z., Sheikhzadeh Marand, E., & Nasiri Firuz, A. R. (2024). Artificial Intelligence in Education and Efl Teachers’ Attitudes: Scale Development and Validation. Elsevier BV. https://doi.org/10.2139/ssrn.4687460
- Alan, B., Keçeci, G., & Kırbağ Zengin, F. (2024). Öğretmenler için Yapay Zekâ Tutum Ölçeği: Geçerlik ve Güvenirlik Çalışması. Pamukkale University Journal of Education. https://doi.org/10.9779/pauefd.1418456
- Alan, B., Keçeci, G., & Kırbağ Zengin, F. (2025). Öğretmenler için Yapay Zekâ Tutum Ölçeği: Geçerlik ve Güvenirlik Çalışması. Pamukkale University Journal of Education. https://doi.org/10.9779/pauefd.1418456
- Almatrafi, O., Johri, A., & Lee, H. (2024). A systematic review of AI literacy conceptualization, constructs, and implementation and assessment efforts (2019–2023). Computers and Education Open, 6, 100173. https://doi.org/10.1016/j.caeo.2024.100173
- Belanger, A. (2025, June 24). Key fair use ruling clarifies when books can be used for AI training. Ars Technica. https://arstechnica.com/tech-policy/2025/06/key-fair-use-ruling-clarifies-when-books-can-be-used-for-ai-training/
- Burgsteiner, H., Kandlhofer, M., & Steinbauer, G. (2016). IRobot: Teaching the Basics of Artificial Intelligence in High Schools. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). https://doi.org/10.1609/aaai.v30i1.9864
- Büyüköztürk, Ş. (2010). Sosyal bilimler için veri analizi el kitabı. (4th edn). Pegem Akademi.
- Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., & Wienrich, C. (2023). MAILS - Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies. Computers in Human Behavior: Artificial Humans, 1(2), 100014. https://doi.org/10.1016/j.chbah.2023.100014
- Casal-Otero, L., Catala, A., Fernández-Morante, C., Taboada, M., Cebreiro, B., & Barro, S. (2023). AI literacy in K-12: A systematic literature review. International Journal of STEM Education, 10(1). https://doi.org/10.1186/s40594-023-00418-7
- Cetindamar, D., Kitto, K., Wu, M., Zhang, Y., Abedin, B., & Knight, S. (2024). Explicating AI Literacy of Employees at Digital Workplaces. IEEE Transactions on Engineering Management, 71, 810–823. https://doi.org/10.1109/tem.2021.3138503