TY  - JOUR
T1  - Turkish adaptation of the digital literacy scale: A rasch analysis
TT  - Turkish adaptation of the digital literacy scale: A rasch analysis
AU  - Yang, Hongwei
AU  - Alanoğlu, Müslim
AU  - Karabatak, Songül
AU  - Bradley, Kelly
PY  - 2025
DA  - February
Y2  - 2024
DO  - 10.21449/ijate.1545220
JF  - International Journal of Assessment Tools in Education
JO  - Int. J. Assess. Tools Educ.
PB  - İzzet KARA
WT  - DergiPark
SN  - 2148-7456
SP  - 93
EP  - 112
VL  - 12
IS  - 1
LA  - en
AB  - The study took a Rasch measurement theory approach to validating the 10-item Digital Literacy Scale (DLS) using the unidimensional rating scale model (RSM). To that end, the study used the data from a sample of online Turkish university students. The study began the Rasch analysis with all 10 items in the scale and, to improve in the local independence assumption, identified and eliminated two items which did not adequately fit the RSM. Under the eight-item DLS, the assumptions of undimensionality and local independence were both satisfied and the fit of all individual items to the RSM was adequate. Next, the psychometric properties of the eight-item DLS were examined including rating scale effectiveness, relative endorsability of the items, differential item functioning (DIF) by each of three demographic variables: (a) gender, (b) connection device, and (c) grade level. Through the analysis, evidence of reliability and validity was identified which generally supports the use of the DLS instrument among the population of online Turkish university students from which the sample was obtained. The study also identified items which demonstrated either misfit to the model or DIF by the demographic variables, and recommends they be further reviewed and revised for future use.
KW  - Differential item functioning
KW  - Digital literacy scale
KW  - Scale validation
KW  - Rasch analysis
KW  - University students
N2  - The study took a Rasch measurement theory approach to validating the 10-item Digital Literacy Scale (DLS) using the unidimensional rating scale model (RSM). To that end, the study used the data from a sample of online Turkish university students. The study began the Rasch analysis with all 10 items in the scale and, to improve in the local independence assumption, identified and eliminated two items which did not adequately fit the RSM. Under the eight-item DLS, the assumptions of undimensionality and local independence were both satisfied and the fit of all individual items to the RSM was adequate. Next, the psychometric properties of the eight-item DLS were examined including rating scale effectiveness, relative endorsability of the items, differential item functioning (DIF) by each of three demographic variables: (a) gender, (b) connection device, and (c) grade level. Through the analysis, evidence of reliability and validity was identified which generally supports the use of the DLS instrument among the population of online Turkish university students from which the sample was obtained. The study also identified items which demonstrated either misfit to the model or DIF by the demographic variables, and recommends they be further reviewed and revised for future use.
CR  - Adroher, N.D., Prodinger, B., Fellinghauer, C.S., & Tennant, A. (2018) All metrics are equal, but some metrics are more equal than others: A systematic search and review on the use of the term metric'. PloS ONE, 13(3), e0193861. https://doi.org/10.1371/journal.pone.0193861
CR  - Ala Mutka, K. (2011). Mapping digital competence: Towards a conceptual understanding (Technical Note: JRC 67075). https://doi.org/10.13140/RG.2.2.18046.00322
CR  - Amin, H., Malik, M.A., & Akkaya, B. (2022). Development and validation of Digital Literacy Scale (DLS) and its Implication for higher education. International Journal of Distance Education and E Learning, 7(1), 24 43. https://doi.org/10.36261/ijdeel.v7i1.2224
CR  - Andrich, D., & Marais, I. (2019). A course in Rasch measurement theory: Measuring in the educational, social and health sciences. Springer.
CR  - Anthonysamy, L., Koo, A.C., & Hew, S.H. (2020). Self-regulated learning strategies in higher education: Fostering digital literacy for sustainable lifelong learning. Education and Information Technologies, 25, 2393-2414. https://doi.org/10.1007/s10639-020-10201-8
CR  - Anwar, Z., Hanurawan, F., Chusniyah, T., & Setiyowati, N. (2023). Adaptation of the Academic Digital Literacy Scale for college students: A validity and reliability study. Psychological Science and Education, 28(4), 98 111. https://doi.org/10.17759/pse.2023280406
CR  - Aryadoust, V., Ng, L.Y., & Sayama, H. (2021). A comprehensive review of Rasch measurement in language assessment: Recommendations and guidelines for research. Language Testing, 38(1), 6-40. https://doi.org/10.1177/0265532220927487
CR  - Aydınlar, A., Mavi, A., Kütükçü, E., Kırımlı, E.E., Alış, D., Akın, A., & Altıntaş, L. (2024). Awareness and level of digital literacy among students receiving health-based education. BMC Medical Education, 24-38. https://doi.org/10.1186/s12909-024-05025-w
CR  - Bartholomew, D.J., Steele, F., Moustaki, I., & Galbraith, J. (2008). Analysis of multivariate social science data (2nd ed.). Routledge.
CR  - Bond, T.G., & Fox, C.M. (2015). Applying the Rasch model (3rd ed.). Routledge.
CR  - Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge University Press.
CR  - Cameron, I.M., Scott, N.W., Adler, M., & Reid, I.C. (2014). A comparison of three methods of assessing Differential Item Functioning (DIF) in the hospital anxiety depression scale: Ordinal logistic regression, Rasch analysis and the Mantel chi-square procedure. Quality of Life Research, 23(10), 2883-2888. https://doi.org/10.1007/s11136-014-0719-3
CR  - Cape, J., Whittington, C., Buszewicz, M., & Underwood, L. (2010). Brief psychological therapies for anxiety and depression in primary care: Meta-analysis and meta-regression. BMC Medicine, 8, 38. https://doi.org/10.1186/1741-7015-8-38
CR  - Capinding, A.T. (2024). Development and validation of instruments for assessing the impact of artificial intelligence on students in higher education. International Journal of Educational Methodology, 10(2), 997 – 1011. https://doi.org/10.12973/ijem.10.2.997
CR  - Ceylan, G., Eken, M.O., Yuruk, S., & Emir, F. (2023). Examining the influence of self-esteem and digital literacy on professional competence factors in dental education: A cross sectional study. Applied Sciences, 13(16), 9411. https://doi.org/10.3390/app13169411
CR  - Chandra, S., Ghadei, K., Chennamadhava, M., & Ali, W. (2024). Development and validation of a farmer’s focused Digital Literacy Scale. Indian Journal of Extension Education, 60(1), 111-115. https://doi.org/10.48165/IJEE.2024.601RT1
CR  - Chen, F.F., Hayes, A., Carver, C.S., Laurenceau, J.P., & Zhang, Z. (2012). Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches. Journal of Personality, 80(1), 219–251. https://doi.org/10.1111/j.1467-6494.2011.00739.x
CR  - Da Dalt, L., Anselmi, P., Bressan, S., Carraro, S., Baraldi, E., Robusto, E., & Perilongo, G. (2013). A short questionnaire to assess pediatric resident’s competencies: the validation process. Italian Journal of Pediatrics, 39(41), 1 8. https://doi.org/10.1186/1824-7288-39-41
CR  - Da Dalt, L., Anselmi, P., Furlan, S., Carraro, S., Baraldi, E., Robusto, E. & Perilongo, G. (2015). Validating a set of tools designed to assess the perceived quality of training of pediatric residency programs. Italian Journal of Pediatrics, 41(2), 1 7. https://doi.org/10.1186/s13052-014-0106-2
CR  - Durak, H.Y., & Seferoğlu, S.S. (2020). Antecedents of social media usage status: Examination of predictiveness of digital literacy, academic performance, and fear of missing out variables. Social Science Quarterly, 101(3), 1056-1074. https://doi.org/10.1111/ssqu.12790
CR  - Erol, S., & Aydin, E. (2021). Digital literacy status of Turkish teachers. International Online Journal of Educational Sciences, 13(2), 620 633. https://doi.org/10.15345/iojes.2021.02.020
CR  - Fan, J., & Bond T. (2019). Applying Rasch measurement in language assessment: Unidimensionality and local independence. In V. Aryadoust & M. Raquel (Eds.), Quantitative data analysis for language assessment, Vol. I: Fundamental techniques (pp. 83–102). Routledge. https://doi.org/10.4324/9781315187815
CR  - Frampton, G.K., & Shepherd, J. (2011). Patient-reported outcomes in clinical trials of inhaled asthma medications: Systematic review and research needs. Quality of Life Research, 20, 343–357. https://doi.org/10.1007/s11136-010-9750-1
CR  - Garzon, J., & Garzon, J. (2023). Teachers’ digital literacy and self-efficacy in blended learning. International Journal of Multidisciplinary Educational Research and Innovation. 1 (4), 162-174. https://doi.org/10.17613/cmjv-1386
CR  - Gignac, G.E., & Kretzschmar, A. (2017). Evaluating dimensional distinctness with correlated-factor models: Limitations and suggestions. Intelligence, 62, 138 147. https://doi.org/10.1016/j.intell.2017.04.001
CR  - Gillen, J., & Barton, D. (2010). Digital literacies: A research briefing by the technology enhanced learning phase of the Teaching and Learning Research Programme. London Knowledge Lab, Institute of Education.
CR  - Gilster, P. (1997). Digital literacy. John Wiley & Sons.
CR  - Gocen, A., & Sen, S. (2021). A validation of Servant Leadership Scale on multinational sample. Psychological Reports, 124(2), 752 770. https://doi.org/10.1177/0033294120957246
CR  - Gutiérrez-Ángel, N., Sánchez-García, J.-N., Mercader-Rubio, I., García-Martín, J., & Brito-Costa, S. (2022). Digital literacy in the university setting: A literature review of empirical studies between 2010 and 2021. Frontiers in Psychology, 13, 896800. https://doi.org/10.3389/fpsyg.2022.896800
CR  - Hagquist, C. (2019). Explaining differential item functioning focusing on the crucial role of external information–an example from the measurement of adolescent mental health. BMC Medical Research Methodology, 19(185), 1 9. https://doi.org/10.1186/s12874-019-0828-3
CR  - Hagquist, C., Bruce, M., & Gustavsson, J.P. (2009). Using the Rasch model in nursing research: An introduction and illustrative example. International Journal of Nursing Studies, 46(3), 380-393. https://doi.org/10.1016/j.ijnurstu.2008.10.007
CR  - Hamilton, C.B., & Chesworth, B.M. (2013). A Rasch-validated version of the upper extremity functional index for interval-level measurement of upper extremity function. Physical Therapy, 93(11), 1507–1519. https://doi.org/10.2522/ptj.20130041
CR  - Hamutoğlu, N.B., Canan Güngören, Ö., Kaya Uyanık, G., & Gür Erdoğan, D. (2017). Adapting digital literacy scale into Turkish. Ege Journal of Education, 18(1), 408–29. https://dergipark.org.tr/tr/download/article-file/326304
CR  - Horn, J.L., & McArdle, J.J. (1992). A practical and theoretical to measurement invariance in Aging research. Experimental Aging Research, 18, 117 144. https://doi.org/10.1080/03610739208253916
CR  - Hwang, H.S., Zhu, L.C., & Cui, Q. (2023). Development and validation of a Digital Literacy Scale in the artificial intelligence era for college students. KSII Transactions on Internet and Information Systems, 17(8), 2241-2258. https://doi.org/10.3837/tiis.2023.08.016
CR  - Lazonder, A.W., Walraven, A., Gijlers, H., & Janssen, N. (2020). Longitudinal assessment of digital literacy in children: Findings from a large Dutch single-school study. Computers & Education, 143, 103681. https://doi.org/10.1016/j.compedu.2019.103681
CR  - Lee, Y.W. (2004). Examining passage-related local item dependence (LID) and measurement construct using Q3 statistics in an EFL reading comprehension test. Language Testing, 21(1), 74–100. https://doi.org/10.1191/0265532204lt260oa
CR  - Linacre, J.M. (n.d.). Table 23.0 Variance components for items. https://www.winsteps.com/winman/table23_0.htm
CR  - Linacre, J.M. (2018, September 2). Detecting multidimensionality in Rasch data using Winsteps Table 23 [Video]. Youtube. https://www.youtube.com/watch?v=sna19QemE50
CR  - Linacre, J.M. (2023). Winsteps® Rasch measurement computer program user's guide. Version 5.6.4. Winsteps.com
CR  - Liu, Y., Millsap, R.E., West, S.G., Tein, J.-Y., Tanaka, R., & Grimm, K.J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22(3), 486–506. https://doi.org/10.1037/met0000075
CR  - Liza, K., & Andriyanti, E. (2020). Digital literacy scale of English pre-service teachers and their perceived readiness toward the application of digital technologies. Journal of Education and Learning, 14(1), 74-79. https://doi.org/10.11591/edulearn.v14i1.13925
CR  - Meredith, W. (1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 525-543. https://doi.org/10.1007/BF02294825
CR  - Miot, H.A. (2020). Analysis of ordinal data in clinical and experimental studies. Jornal Vascular Brasileiro, 19, e20200185. https://doi.org/10.1590/1677-5449.200185
CR  - Mundfrom, D.J., Shaw, D.G., & Ke, T.L. (2005). Minimum sample size recommendations for conducting factor analyses. International Journal of Testing, 5(2), 159–168. https://doi.org/10.1207/s15327574ijt0502_4
CR  - Ng, W. (2012a). Can we teach digital natives digital literacy?. Computers & Education, 59(3), 6881065-1078. https://doi.org/10.1016/j.compedu.2012.04.016
CR  - Ng, W. (2012b). Empowering scientific literacy through digital literacy and multiliteracies. Nova Science Publishers, Inc.
CR  - Nguyen, L.A.T., & Habók, A. (2024). Tools for assessing teacher digital literacy: A review. Journals of Computers in Education, 11, 305-346). https://doi.org/10.1007/s40692-022-00257-5
CR  - Ningsih, S.K., Mulyono, H., Rahmah, R.A., & Fitriani, N.A. (2021). A Rasch-based validation of EFL teachers’ received online social support scale. Cogent Education, 8(1), 1-13. https://doi.org/10.1080/2331186X.2021.1957529
CR  - Noorrizki, R.D., Abadi, D., Siwi, N.S.W., Sa'id, M., Mantara, A.Y., & Ramadhani, F. (2022). Factors affecting digital literacy in young adults. Proceedings of the International Conference of Psychology 2022 (ICoPsy 2022), 308-315, KnE Social Sciences. https://doi.org/10.18502/kss.v7i18.12396
CR  - Nunnally, J.C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
CR  - Olur, B., & Ocak, G. (2021). Digital literacy self-efficacy scale: A scale development study. African Educational Research Journal, 9(2), 581 590. https://doi.org/10.30918/AERJ.92.21.074
CR  - Pangrazio, L., Godhe, A.-L., & Ledesma, A.G.L. (2020). What is digital literacy? A comparative review of publications across three language contexts. E-Learning and Digital Media, 17(6), 442-459. https://doi.org/10.1177/2042753020946291
CR  - Perera, M.U., Gardner, L., & Peiris, A. (2016). Investigating the Interrelationship between undergraduates’ digital literacy and self-regulated learning skills. Proceedings of the 2016 International Conference on Information Systems (ICIS), Dublin, Ireland.
CR  - Reddy, P., Chaudhary, K., Sharma, B., & Chand, D. (2021). Contextualized game-based intervention for digital literacy for the Pacific Islands. Education and Information Technologies, 26, 5535–5562. https://doi.org/10.1007/s10639-021-10534-y
CR  - Reddy, P., Chaudhary, K., Sharma, B., & Hussein, S. (2023). Essaying the design, development and validation processes of a new digital literacy scale. Online Information Review, 47(2), 371-397. https://doi.org/10.1108/oir-10-2021-0532
CR  - Reddy, P., Chaudhary, K., & Hussein, S. (2023). A digital literacy model to narrow the digital literacy skills gap. Heliyon, 9, e14878. https://doi.org/10.1016/j.heliyon.2023.e14878
CR  - Reise, S.P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555
CR  - Reise, S.P., Morizot, J., & Hays, R.D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16(1), 19–31. https://doi.org/10.1007/s11136-007-9183-7
CR  - Rodriguez, A., Reise, S.P., & Haviland, M.G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98(3), 223–237. https://doi.org/10.1080/00223891.2015.1089249
CR  - Rodriguez, A., Reise, S.P., & Haviland, M.G. (2016b). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045
CR  - Setiyowati N., & Razak, A.Z.A. (2020). Perceived leadership styles and academicians’ job performance: Teaching, research, and community services in Indonesia. Malaysia Online Journal of Psychology and Counseling, 7(1), 11 26. https://jupidi.um.edu.my/index.php/MOJC/article/view/24179/11654
CR  - Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. Chapman Hall/CRC.
CR  - Tor, D.L., Başaran, S.D., & Arık, E. (2022). Examining the digital Literacy level of teacher candidates. Journal of Kırşehir Education Faculty, 23(2), 2027 2064. https://doi.org/10.29299/kefad.1047590
CR  - UNESCO. (2018). A global framework of reference on digital literacy skills for indicator 4.4.2. https://uis.unesco.org/sites/default/files/documents/ip51-global-framework-reference-digital-literacy-skills-2018-en.pdf
CR  - Urbancikova, N., Manakova, N., & Ganna, B. (2017). Socio-economic and regional factors of digital literacy related to prosperity. Quality Innovation Prosperity, 21(2), 124-141. https://doi.org/10.12776/qip.v21i2.942
CR  - Üstündağ, M.T., Güneş, E., & Bahçivan, E. (2017). Turkish adaptation of Digital Literacy Scale and investigating pre-service science teachers' digital literacy. Journal of Education and Future, 12, 19-29. https://dergipark.org.tr/tr/download/article-file/332115
CR  - Wright, B.D. (1996). Local dependency, correlations and principal components. Rasch Measurement Transactions, 10(3), 509–511. https://www.rasch.org/rmt/rmt103b.htm
CR  - Wu, M., DeWitt1, D., Alias, N., & Nazry, N.N.M. (2022). Digital literacy from 2018-2021: A scientometric study of the literature. https://doi.org/10.21203/rs.3.rs-1848300/v1
CR  - Yen, W.M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125-145. https://doi.org/10.1177/014662168400800201
CR  - Zwick, R., Thayer, D.T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1), 1 28. https://doi.org/10.1111/j.1745-3984.1999.tb00543.x
CR  - Zwick, R. (2012). A review of ETS Differential Item Functioning assessment procedures: Flagging rules, minimum sample size requirements, and criterion refinement. ETS Research Report Series, i-30. https://doi.org/10.1002/j.2333-8504.2012.tb02290.x
UR  - https://doi.org/10.21449/ijate.1545220
L1  - https://dergipark.org.tr/en/download/article-file/4197773
ER  -