Development and Validation of the Teachers’ Digital Competence Scale (TDiCoS)

Abstract

The competences expected from teachers are described in several international reports. An investigation of teacher competences has a potential to analyze and delineate the current situation. Assessment of any construct necessitates utilization of appropriate scales with established validity and reliability. Currently available validated digital competence scales have examined digital competences of citizens, teachers, students, but most are based on only one digital competence framework. In this study a valid, reliable, and comprehensive scale entitled “Teachers’ Digital Competence Scale” (TDiCoS) for teachers of different subjects has been obtained by relying on standards/policy reports from several countries. TDiCoS items measure teachers' self-assessment and report of their use of digital technologies in their classrooms. TDiCoS was validated on a sample of 288 in-service teachers. In the final form TDiCoS is unidimensional and composed of 19 items. The model fit criteria, factor loadings, internal validity, and reliability of TDiCoS were examined and found to be good. Hence, it can inform design of new policies and teacher professional development programs targeting digital competences.

Keywords

• Digital competence
• Scale development
• Teacher
• Digital technologies

References

1. (Author, 2020)
2. Alarcón, R., Del Pilar Jiménez, E., & de Vicente‐Yagüe, M. I. (2020). Development and validation of the DIGIGLO, a tool for assessing the digital competence of educators. British Journal of Educational Technology, 51(6), 2407-2421. https://doi.org/10.1111/bjet.12919
3. Bartolomé, J., de Soria, I. M., Jakobsone, M., Fernández, A., Ruseva, G., Koutoudis, P., ... & Vaquero, M. (2018, March). Developing a digital competence assessment and accreditation platform for digital profiles. In Proceedings of the 12th International Technology, Education and Development Conference (INTED), Valencia, Spain (pp. 5-7).
4. Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: a primer. Frontiers in Public Health, 6, 149. https://doi.org/10.3389/fpubh.2018.00149
5. Brečko, B. & Ferrari, A. (2016). The digital competence framework for consumers. European Commission Joint Research Centre. https://doi.org/10.2791/838886
6. Bryman, A., & Cramer, D. (1999). Quantitative data analysis with SPSS release 8 for windows. Routledge.
7. Caena, F., & Redecker, C. (2019). Aligning teacher competence frameworks to 21st century challenges: The case for the European Digital Competence Framework for Educators (DigCompedu). European Journal of Education, 54(3), 356–369. https://doi.org/10.1111/ejed.12345
8. Calvani, A., Cartelli, A., Fini, A., & Ranieri, M. (2008). Models and instruments for assessing digital competence at school. Journal of E-learning and Knowledge Society, 4(3), 183-193. https://doi.org/10.20368/1971-8829/288

9. Carretero, S.; Vuorikari, R. and Punie, Y. (2017). DigComp 2.1: The digital competence framework for citizens with eight proficiency levels and examples of use, Joint Research Centre. https://doi.org/10.2760/38842
10. Cartelli, A. (2010, June). Frameworks for digital competence assessment: Proposals, instruments, and evaluation. In Proceedings of Informing Science & IT Education Conference (InSITE) (pp. 561-573). https://doi.org/10.28945/1274
11. Cattell, R. B. (1966). The Scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245-276. https://doi.org/10.1207/s15327906mbr0102_10
12. Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319. https://doi.org/10.1037/1040-3590.7.3.309
13. Cohen, R. J., & Swerdlik, M. E., (2009). Psychological testing and assessment: An introduction to tests and measurement (7th Edition). The McGraw-Hill Companies.
14. Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis. (2nd Edition). Psychology press.
15. Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(7), 1-9. https://doi.org/10.7275/jyj1-4868
16. DeCoster, J. (1998). Overview of Factor Analysis. http://www.stat-help.com/notes.html. Accessed 08 May 2022.
17. DeVellis, R. F. (2017). Scale development: Theory and applications. (4th Edition). Sage.
18. Dziuban, C. D., & Shirkey, E. C. (1974). When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin, 81(6), 358. https://doi.org/10.1037/h0036316
19. Ferrari, A. (2012). Digital Competence in Practice: An Analysis of Frameworks. European Commission Joint Research Centre. https://doi.org/10.2791/82116
20. Ferrari, A., & Punie, Y. (2013). DIGCOMP: A framework for developing and understanding digital competence in Europe. European Commission Joint Research Centre. https://doi.org/10.2788/52966
21. Fontaine, J. R. (2005). Equivalence. Encyclopedia of Social Measurement, 1, 803-813.
22. Fowler, F. J. (1995). Improving survey questions design and evaluation. Sage.
23. Guadagnoli, E., & Velicer, W. F. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265-275. https://doi.org/10.1037//0033-2909.103.2.265
24. Guskey, T. R. (2002). Professional development and teacher change. Teachers and Teaching: Theory and Practice, 8(3), 381-391. https://doi.org/10.1080/135406002100000512
25. Gümüş, M. M., & Kukul, V. (2022). Developing a digital competence scale for teachers: validity and reliability study. Education and Information Technologies, 1-19. https://doi.org/10.1007/s10639-022-11213-2
26. Hambleton, R. K., & Traub, R. E. (1973). Analysis of empirical data using two logistic latent trait models. British Journal of Mathematical and Statistical Psychology, 26(2), 195–211. https://doi.org/10.1111/j.2044-8317.1973.tb00517.x
27. Harari, Y. N. (2018). 21. yüzyıl için 21 ders [21 lessons for the 21st century] (in Turkish). Kolektif Kitap.
28. He, T., & Li, S. (2019). A comparative study of digital informal learning: The effects of digital competence and technology expectancy. British Journal of Educational Technology, 50(4), 1744-1758. https://doi.org/10.1111/bjet.12778
29. Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34(3), 177-189. https://doi.org/10.1080/07481756.2002.12069034
30. Hurley, A. E., Scandura, T. A., Schriesheim, C. A., Brannick, M. T., Seers, A., Vandenberg, R. J., & Williams, L. J. (1997). Exploratory and confirmatory factor analysis: Guidelines, issues, and alternatives. Journal of Organizational Behavior, 667-683. https://doi.org/10.1002/(SICI)1099-1379(199711)18:6%3C667::AID-JOB874%3E3.0.CO;2-T
31. Ilomäki, L., Paavola, S., Lakkala, M., & Kantosalo, A. (2016). Digital competence–an emergent boundary concept for policy and educational research. Education and Information Technologies, 21(3), 655-679. https://doi.org/10.1007/s10639-014-9346-4
32. ISTE (2000). ISTE national educational technology standards (NETS) and performance indicators for teachers. http://www.iste.org/docs/pdfs/nets_for_teachers_2000.pdf Accessed 18 May 2022.
33. ISTE (2017). ISTE standards for educators. International Society for Technology in Education. https://www.iste.org/standards/iste-standards-for-teachers. Accessed 18 May 2022. Janssen, J., Stoyanov, S., Ferrari, A., Punie, Y., Pannekeet, K., & Sloep, P. (2013). Experts' views on digital competence: Commonalities and differences. Computers & Education, 68, 473-481. https://doi.org/10.1016/j.compedu.2013.06.008
34. Kim, M., & Choi, D. (2018). Development of youth digital citizenship scale and implication for educational setting. Journal of Educational Technology & Society, 21(1), 155-171.
35. Kline, P. (1993). The handbook of psychological testing (2nd edition). Routledge.
36. Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). The Guilford.
37. Kluzer, S., & Pujol Priego, L. (2018). DigComp into action: get inspired, make it happen. Publications Office of the European Union. https://doi.org/10.2760/112945
38. Kuran, E. (2018). Telgraftan tablete [From telegram to tablet] (in Turkish). Destek.
39. Kuzminska, O., Mazorchuk, M., Morze, N., Pavlenko, V., & Prokhorov, A. (2018). Digital competency of the students and teachers in Ukraine: measurement, analysis, development prospects. In V. Ermolayev et al. (Eds.), Proceedings of the 14th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer (ICTERI 2018) (pp. 366–379).
40. Law, N., Woo, D., de la Torre, J., & Wong, G. (2018). A global framework of reference on digital literacy skills for indicator 4.4.2. Unesco Institute for Statistics. http://uis.unesco.org/sites/default/files/documents/ip51-global-framework-reference-digital-literacy-skills-2018-en.pdf Accessed 18 May 2022.
41. Lucas, M., Bem-Haja, P., Siddiq, F., Moreira, A., & Redecker, C. (2021). The relation between in-service teachers' digital competence and personal and contextual factors: What matters most? Computers & Education, 160, 104052. https://doi.org/10.1016/j.compedu.2020.104052
42. MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84–99. https://doi.org/10.1037/1082-989X.4.1.84
43. Mattar, J., Ramos, D. K., & Lucas, M. R. (2022). DigComp-based digital competence assessment tools: literature review and instrument analysis. Education and Information Technologies, 1-25. https://doi.org/10.1007/s10639-022-11034-3
44. MNE (2006). Öğretmenlik mesleği genel yeterlikleri [General competencies of the teaching profession] (in Turkish). https://oygm.meb.gov.tr/meb_iys_dosyalar/2017_12/13161921_YYretmenlik_MesleYi_Genel__YETERLYKLERi_onaylanan.pdf Accessed 18 May 2022.
45. MNE (2017). Öğretmenlik mesleği genel yeterlikleri [General competencies of the teaching profession] (in Turkish). https://oygm.meb.gov.tr/meb_iys_dosyalar/2017_12/11115355_YYRETMENLYK_MESLEYY_GENEL_YETERLYKLERY.pdf Accessed 18 May 2022.
46. MNE (2020). Dijital okuryazarlık öğretmen klavuzu [Digital literacy teacher's guide] (in Turkish). http://cdn.eba.gov.tr/kitap/digital/#p=1 Accessed 18 May 2022.
47. NETS-S. (2007). National educational technology standards for students, NET-S. Washington.D.C.
48. NETS-T. (2008). National educational technology standards for teachers, NET-T. Washington.D.C.
49. Niess, M. L. (2011). Investigating TPACK: knowledge growth in teaching with technology. Journal of Educational Computing Research, 44(3) 299-317. https://doi.org/10.2190/EC.44.3.c
50. Oberländer, M., Beinicke, A., & Bipp, T. (2020). Digital competencies: A review of the literature and applications in the workplace. Computers & Education, 146, 103752. https://doi.org/10.1016/j.compedu.2019.103752
51. OEDC. (2022). OECD education and skills today. https://oecdedutoday.com/coronavirus/. Accessed 18 May 2022.
52. Prensky, M. (2001). Digital natives, digital immigrants part 1. On the horizon, 9(5), 1-6. https://doi.org/10.1108/10748120110424816
53. Redecker, C. (2017). European framework for the digital competence of educators-DigCompEdu. Publications Office of the European Union. https://doi.org/10.2760/159770 Revuelta-Domínguez, F. I., Guerra-Antequera, J., González-Pérez, A., Pedrera-Rodríguez, M. I., & González-
54. Fernández, A. (2022). Digital teaching competence: a systematic review. Sustainability, 14(11), 6428. https://doi.org/10.3390/su14116428
55. Roberts, K., & Kruse, J. (2022). Investigating changes in preservice teachers’ conceptions of technological literacy. International Journal of Technology and Design Education, 1-14. https://doi.org/10.1007/s10798-021-09726-x
56. Rowston, K., Bower, M., & Woodcock, S. (2021). The impact of prior occupations and initial teacher education on post-graduate pre-service teachers’ conceptualization and realization of technology integration. International Journal of Technology and Design Education, 1-39. https://doi.org/10.1007/s10798-021-09710-5
57. Scherer, R., Siddiq, F., & Tondeur, J. (2017). On the quest for validity: Testing the factor structure and measurement invariance of the technology-dimensions in the Technological, Pedagogical, and Content Knowledge (TPACK) model. Computers & Education, 112, 1–17. https://doi.org/10.1016/j.compedu.2017.04.012
58. Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74.
59. Schleicher, A. (2020). TALIS 2018: Insights and interpretations, OECD. https://www.oecd.org/education/talis/TALIS2018_insights_and_interpretations.pdf. Accessed 18 May 2022.
60. Schumacker, R. E., & Lomax, R. G. (2010). A beginner's guide to structural equation modeling (3rd ed.). Routledge.
61. Siddiq, F., Scherer, R., & Tondeur, J. (2016). Teachers' emphasis on developing students' digital information and communication skills (TEDDICS): A new construct in 21st century education. Computers & Education, 92, 1-14. https://doi.org/10.1016/j.compedu.2015.10.006
62. Sillat, L. H., Tammets, K., & Laanpere, M. (2021). Digital competence assessment methods in higher education: A systematic literature review. Education Sciences, 11(8), 402. https://doi.org/10.3390/educsci11080402
63. Skantz-Åberg, E., Lantz-Andersson, A., Lundin, M., & Williams, P. (2022). Teachers’ professional digital competence: an overview of conceptualisations in the literature. Cogent Education, 9(1), 2063224. https://doi.org/10.1080/2331186X.2022.2063224
64. Tabachnick, B., & Fidell, L. (2013). Using multivariate statistics. Pearson Education.
65. Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273-1296. https://doi.org/10.1007/s11165-016-9602-2
66. TEDMEM (2021). Teachers’ digital competences [Öğretmen dijital yeterlikleri] (in Turkish) https://tedmem.org/wp-content/uploads/2021/09/ogretmen-dijital-yeterlikleri.pdf. Accessed 18 May 2022.
67. Toker, T., Akgün, E., Cömert, Z., & Sultan, E. (2021). Eğitimciler için dijital yeterlilik ölçeği: uyarlama, geçerlik ve güvenirlik çalışması. Milli Eğitim Dergisi, 50(230), 301-328. https://doi.org/10.37669/milliegitim.801607
68. Tondeur, J., Aesaert, K., Pynoo, B., van Braak, J., Fraeyman, N., & Erstad, O. (2017). Developing a validated instrument to measure preservice teachers’ ICT competencies: Meeting the demands of the 21st century. British Journal of Educational Technology, 48(2), 462–472. https://doi.org/10.1111/bjet.12380
69. Twenge, J. M. (2017). iGen: Why today's super-connected kids are growing up less rebellious, more tolerant, less happy--and completely unprepared for adulthood--and what that means for the rest of us. Simon and Schuster.
70. Turkish Education Association (2009). Öğretmen yeterlikleri [Teacher competencies] (in Turkish). Adım Okan. Tzafilkou, K., Perifanou, M., & Economides, A. A. (2022). Development and validation of students’ digital competence scale (SDiCoS). International Journal of Educational Technology in Higher Education, 19(1), 1-20. https://doi.org/10.1186/s41239-022-00330-0
71. UNESCO (2011). UNESCO ICT competency framework for teachers. https://iite.unesco.org/pics/publications/en/files/3214694.pdf. Accessed 18 May 2022.
72. UNESCO (2018). UNESCO ICT competency framework for teachers. https://unesdoc.unesco.org/ark:/48223/pf0000265721. Accessed 18 May 2022.
73. Van Driel, J. H., Beijaard, D., & Verloop, N. (2001). Professional development and reform in science education: The role of teachers' practical knowledge. Journal of Research in Science Teaching, 38(2), 137-158.https://doi.org/10.1002/1098-2736(200102)38:2<137::AID-TEA1001>3.0.CO;2-U
74. Vuorikari, R., Punie, Y., Carretero Gomez S., Van den Brande, G. (2016). DigComp 2.0: The digital competence framework for citizens. Luxembourg Publication Office of the European Union. https://doi.org/10.2791/11517
75. Yılmaz, E., Aktürk, A., & Çapuk, S. (2021). Dijital öğretmen yeterlilik ölçeği geliştirme: geçerlik ve güvenirlik çalışması. Adıyaman Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 14(38), 35-68. https://doi.org/10.14520/adyusbd.950728