Reliability and validity of the Turkish version of the teachers’ basic ICT competence beliefs scale

Öz

The present study seeks to adapt the Teachers’ Basic Information Communication Technology (ICT) Competence Beliefs Scale, developed by Rubach and Lazarides (2021), into Turkish and test the adapted scale's validity and reliability. The initial step involved conducting a linguistic equivalence of the scale from English to Turkish with 62 English language teachers in a pre-test. Subsequently, the Turkish version of the scale was administered to 356 teachers (69.7% female, 30.3% male) in Turkey to assess its validity and reliability. Participating teachers were from different subjects (e.g., 9.8% science, 7.9% mathematics, 3.7% social science) and school types (27.5% primary school, 55.3% secondary school, 17.1% others). Results of confirmatory factor analysis indicated the original six-factor structure with three first-order and three second-order factors that best fitted the data. The same competence dimensions were indicated in the Turkish contexts as in the original instrument, i.e., information and data literacy; communication and collaboration; digital content creation; safety and security; problem-solving; analyzing and reflecting. The correlations between all six first-order factors were between .58 ≥ r ≥.79. All factors showed good reliability indices, i.e., α > .83, ω > .83 and CR > .72. The adapted instrument was found to be invariant across gender. Mean-level differences among gender groups point to one difference with male teachers reporting higher competence beliefs for digital content creation compared to female teachers. In conclusion, the results of this replication study support the cross-cultural transferability of the original Teachers' Basic ICT Competence Beliefs instrument developed by Rubach and Lazarides (2019).

Anahtar Kelimeler

• Cross-cultural validation
• Pedagogical issues
• ICT competence beliefs
• Teacher professional development

References

1. Anderson, J., Van Weert, T. (Eds.), & Duchâteau, C. (2002). Information and communication technology in education: A curriculum for schools and program of teacher development. UNESCO. https://pure.unamur.be/ws/portalfiles/portal/256022/129538f.pdf
2. Anagün, Ş.S., Atalay, N., Kılıç, Z., & Yaşar, S. (2016). Öğretmen adaylarına yönelik 21. yüzyıl becerileri yeterlilik algıları ölçeğinin geliştirilmesi: Geçerlik ve güvenirlik çalışması [The development of a 21st century skills and competences scale directed at teaching candidates: Validity and reliability study]. Pamukkale University Journal of Education, 40(40), 160-175. http://dx.doi.org/10.9779/PUJE768
3. Ananiadou, K., & Claro, M. (2009). 21st Century Skills and Competences for New Millennium Learners in OECD Countries (No. 41). OECD Education Working Papers, No. 41.OECD Publishing. http://dx.doi.org/10.1787/218525261154
4. Angeli, C., & Valanides, N. (2009). Epistemological and methodological issues for the conceptualization, development, and assessment of ICT–TPCK: Advances in technological pedagogical content knowledge (TPCK). Computers & Education, 52(1), 154-168. https://doi.org/10.1016/j.compedu.2008.07.006
5. Asri, A.A.S.M.A.N., & Darma, G.S. (2020). Revealing the digital leadership spurs in 4.0 industrial revolution. International Journal of Business, Economics & Management, 3(1), 93-100. https://doi.org/10.31295/ijbem.v3n1.135.
6. Ay, Y., Karadağ, E., & Acat, M.B. (2015). The Technological Pedagogical Content Knowledge-practical (TPACK-Practical) model: Examination of its validity in the Turkish culture via structural equation modeling. Computers & Education, 88, 97-108. https://doi.org/10.1016/j.compedu.2015.04.017
7. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral changes. Psychol. Rev. 84, 191–215. https://doi.org/10.1037/0033-295X.84.2.191
8. Bacon, D.R., Sauer, P.L., & Young, M. (1995). Composite reliability in structural equations modeling. Educational and psychological measurement, 55(3), 394-406.

9. Baş, M., & Balaman, F. (2021). Yenilikçi iş davranışı ölçeği'nin Türkçeye uyarlanması: Geçerlik-güvenirlik çalışması [Adaptation of ınnovative work behavior scale to Turkish: A Validity-reliability study]. Mersin University Journal of the Faculty of Education, 17(3).
10. Başusta, N.B., & Gelbal, S. (2015). Gruplararası karşılaştırmalarda ölçme değişmezliğinin test edilmesi: PISA öğrenci anketi örneği [Examination of measurement ınvariance at groups’ comparisons: A Study on PISA student questionnaire]. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 30(4), 80-90.
11. Bayazıt, A., & Seferoğlu, S.S. (2009). Türkiye’deki teknoloji politikalarında eğitimin yeri ve öğretmen yetiştirme politikaları [The place of education in technology policies in Turkey and teacher training policies]. TBD 26. National Informatics Congress, 12. Education Congress in the Light of Information Technologies (BTIE'2009), Proceedings Book, 7-11. Türkiye Bilişim Derneği. http://yunus.hacettepe.edu.tr/~sadi/yayin/BTIE09_Bayazit-Seferoglu_TeknolojiPolitika.pdf
12. Buabeng-Andoh, C. (2012). Factors influencing teachers adoption and integration of information and communication technology into teaching: A review of the literature. International Journal of Education and Development using ICT, 8(1), 136-155. https://www.learntechlib.org/p/188018/
13. Büyüköztürk, Ş. (2011). Sosyal bilimler için veri analizi el kitabı istatistik, araştırma deseni, SPSS uygulamaları ve yorum [Data analysis handbook for social sciences statistics, research design, SPSS applications and interpretation] (13th edition). Pegem Akademi.
14. Cai, Z., Fan, X., & Du, J. (2017). Gender and attitudes toward technology use: A meta-analysis. Computers & Education, 105, 1 13. https://doi.org/10.1016/j.compedu.2016.11.003
15. Chen, F.F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
16. Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. https://link.springer.com/article/10.1007/bf02310555
17. Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2010). Çok değişkenli istatistik SPSS ve LISREL uygulamaları. Ankara: Pegem Akademi Yayınları.
18. Dancey, C.P., & Reidy, J. (2007). Statistics without maths for psychology. Pearson education.
19. Daoud, J.I. (2017, December). Multicollinearity and regression analysis. In Journal of Physics: Conference Series (Vol. 949, No. 1, p. 012009). IOP Publishing.
20. Dündar, S., Ekşi. H, & Yıldız, A. (2008). Aksiyonda değerler ölçeği dilsel eşdeğerlik geçerlik ve güvenirlik çalışması [Modified Values in Action Questionnaire (VIA)]. Değerler Eğitimi Dergisi, 6(15), 89-110.
21. Eğitim Bilişim Ağı (EBA). (2020). Education Information Network. http://www.eba.gov.tr/
22. Eickelmann, B., & Vennemann, M. (2017). Teachers ‘attitudes and beliefs regarding ICT in teaching and learning in European countries. European Educational Research Journal, 16(6), 733-761. https://doi.org/10.1177/1474904117725899
23. European Commission (EACEA/Eurydice). (2019). Digital education at school in Europe. Eurydice Report. Luxembourg: Publications Office of the European Union. https://eacea.ec.europa.eu/national policies/eurydice/sites/default/files/en_digital_education_n.pdf
24. Ferrari, A. (2012). Digital competence in practice: An analysis of frameworks. Sevilla: JRC IPTS. http://doi.org/10.2791/82116
25. Ferrari, A. (2013). DigComp: A framework for developing and understanding digital competence in Europe. EUR, scientific and technical research series (Vol. 26035). Luxembourg Publications Office. http://digcomp.org.pl/wp content/uploads/2016/07/DIGCOMP-1.0-2013.pdf
26. Field, A. (2018). Discovering statistics using IBM SPSS Statistics (5th Ed.). Sage.
27. Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in a digital age: The IEA International Computer and Information Literacy Study international report (p. 308). Springer Nature. https://doi.org/10.1007/978-3-319-14222-7
28. Galanouli, D., Murphy, C., & Gardner, J. (2004). Teachers' perceptions of the effectiveness of ICT competence training. Computers & Education, 43(1 2), 63 79. https://doi.org/10.1016/j.compedu.2003.12.005
29. George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Allyn & Bacon.
30. Gerick, J., Eickelmann, B., & Bos, W. (2017). School-level predictors for the use of ICT in schools and students’ CIL in international comparison. Large-scale Assessments in Education, 5(1), 1-13. https://doi.org/10.1186/s40536-017-0037-7
31. Gil-Flores, J., Rodríguez-Santero, J., & Torres-Gordillo, J.J. (2017). Factors that explain the use of ICT in secondary-education classrooms: The role of teacher characteristics and school infrastructure. Computers in Human Behavior, 68(1), 441-449. https://doi.org/10.1016/j.chb.2016.11.057
32. Gökçearslan, Ş., Karademir Coşkun, T., & Şahin, S. (2019). Öğretmen adayı bilgi ve iletişim teknolojisi yeterlikleri ölçeğinin Türkçe’ye uyarlanması [Adaptation of information and communication technology competency scale to Turkish for pre-service teachers]. Kastamonu Education Journal, 27(4), 1435 1444. https://doi.org/10.24106/kefdergi.2828
33. Guggemos, J., & Seufert, S. (2021). Teaching with and teaching about technology–Evidence for professional development of in-service teachers. Computers in Human Behavior, 115, 106613. https://doi.org/10.1016/j.chb.2020.106613
34. Hatlevik, O.E. (2017). Examining the relationship between teachers’ self-efficacy, their digital competence, strategies to evaluate information, and use of ICT at school. Scandinavian Journal of Educational Research, 61(5), 555 567. https://doi.org/10.1080/00313831.2016.1172501
35. Hatlevik, I.K., & Hatlevik, O.E. (2018). Examining the relationship between teachers’ ICT self-efficacy for educational purposes, collegial collaboration, lack of facilitation and the use of ICT in teaching practice. Frontiers in psychology, 9, 935. https://doi.org/10.3389/fpsyg.2018.00935
36. Hatlevik, O.E., Throndsen, I., Loi, M., & Gudmundsdottir, G.B. (2018). Students’ ICT self-efficacy and computer and information literacy: Determinants and relationships. Computers & Education, 118, 107-119. https://doi.org/10.1016/j.compedu.2017.11.011
37. Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
38. Hutcheson, G.D., & Sofroniou, N. (1999). The Multivariate Social Scientist: Introductory Statistics Using Generalized Linear Models. SAGE.
39. Ilomäki, L., Paavola, S., Lakkala, M. et al. (2016). (Digital competence – an emergent boundary concept for policy and educational research. Educ Inf Technol., 21(1), 655–679 https://doi.org/10.1007/s10639-014-9346-4
40. International Journal of Assessment Tools in Education (IJATE). (2014). Guidelines for Translating and Adapting Tests. https://dergipark.org.tr/tr/pub/ijate
41. International Society for Technology in Education (ISTE). (2008). Essential conditions: Necessary conditions to effectively leverage technology for learning. https://www.iste.org/standards/for-educators
42. İzmirli, Ö.Ş. (2015). Öğretim Sürecine BİT Entegrasyonunu Etkinlik Kuramı Çerçevesinde Anlama: Bir Durum Çalışması [Understanding ICT integration into instructional processes within the scope of activity system theory: A case study]. Education and Science, 40(180), 307-325. http://dx.doi.org/10.15390/EB.2015.4725
43. Kapur, R., Byfield, V., Del Frate, F., Higgins, M., & Jagannathan, S. (2018). The digital transformation of education. Earth Observation Open Science and Innovation, 15, 25-41. http://dx.doi.org/10.1007/978-3-319-65633-5
44. Kılıç, S. (2016). Cronbach’s alpha reliability coefficient. Journal of Mood Disorders, 6(1), 47-48. http://dx.doi.org/10.5455/jmood.20160307122823
45. Kılıç, C., & Alcı, B. (2022). Sosyal duygusal öğrenme ölçme aracının Türkçe ’ye uyarlanması: geçerlilik ve güvenilirlik çalışması [Validity and reliability study of Turkish form of social emotional learning questionnaire]. e-Uluslararası Eğitim Araştırmaları Dergisi, 13 (1), 38-50. https://doi.org/10.19160/e-ijer.975137
46. Kim, H.Y. (2013). Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restorative Dentistry & Endodontics, 38(1), 52-54. https://doi.org/10.5395/rde.2013.38.1.52
47. Kocaman Karoğlu, A., Bal, K., & Çimşir, E. (2020). Toplum 5.0 sürecinde Türkiye’de eğitimde dijital dönüşüm [Digital transformation of education in Turkey in society 5.0]. Journal of University Research, 3(3), 147-158. https://doi.org/10.26701/uad.815428
48. Koehler, M.J., & Mishra, P. (2005). What happens when teachers design educational technology? The development of technological pedagogical content knowledge. Journal of Educational Computing Research, 32(2), 131-152. https://doi.org/10.2190/0EW7-01WB-BKHL-QDYV
49. Koh, J.H.L., Chai, C.S., & Tsai, C.C. (2013). Examining practicing teachers’ perceptions of technological pedagogical content knowledge (TPACK) pathways: A structural equation modeling approach. Instructional Science, 41(4), 793 809. https://doi.org/10.1007/s11251-012-9249-y
50. Kultusministerkonferenz (2016). Bildung in der digitalen Welt: Strategie der Kultusministerkonferenz [Education in digital environment: Strategy of the Kultusministerkonferenz]. https://www.kmk.org/fileadmin/Dateien/veroeffentlichungen_beschluesse/2018/Strategie_Bildung_in_der_digitalen_Welt_idF._vom_07.12.2017.pdf
51. Kutluca, T., Arslan, S., & Özpinar, İ. (2010). Developing a scale to measure information and communication technology utilization levels. Journal of Turkish Science Education, 7(4), 37-45. http://tused.org/index.php/tused/article/view/535
52. Krumsvik, R.J. (2014). Teacher educators’ digital competence. Scandinavian Journal of Educational Research, 58(3), 269–280. https://doi.org/10.1080/00313831.2012.726273
53. McDonald, R.P. (1999). Test theory: A unified treatment. Lawrence Erlbaum.
54. Ministary of National Education (MoNE) [Milli Eğitim Bakanlığı]. (2006). Temel eğitime destek projesi “öğretmen eğitimi bileşeni” öğretmenlik mesleği genel yeterlikleri [Basic education support project "teacher training component" general competencies of the teaching profession]. Tebliğler Dergisi, 2590, 1491 1540. http://oygm.meb.gov.tr/meb_iys_dosyalar/2017_12/13161921_YYretmenlik_MesleYi_Genel__YETERLYKLERi_onaylanan.pdf
55. Ministary of National Education (MoNE) [Milli Eğitim Bakanlığı]. (2017). Öğretmenlik Mesleği Genel Yeterlikleri [General Competencies of Teaching Profession]. Tebliğler Dergisi. https://oygm.meb.gov.tr/meb_iys_dosyalar/2017_12/11115355_YYRETMENLYK_MESLEYY_GENEL_YETERLYKLERY.pdf
56. Mishra, P., & Koehler, M.J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teachers’ knowledge. Teachers College Record, 108(6), 1017-1054. https://www.learntechlib.org/p/99246/.
57. Muenks, K., Wigfield, A., & Eccles, J.S. (2018). I can do this! The development and calibration of children’s expectations for success and competence beliefs. Developmental Review, 48, 24-39. https://doi.org/10.1016/j.dr.2018.04.001
58. Muth´en, L.K., & Muth´en, B.O. (1998-2016). Mplus 8.1 [computer software]. Los Angeles, CA: Muth´en & Muth´en.
59. Organisation for Economic Cooperation and Development (OECD). (2018). The future of education and skills: Education 2030. OECD Education Working Papers. http://www.oecd.org/education/2030/oecd-education-2030-position-paper.pdf
60. Öğretmen Bilişim Ağı (ÖBA). (2022). Teacher Information Network. https://www.oba.gov.tr/
61. Palvia, S., Aeron, P., Gupta, P., Mahapatra, D., Parida, R., Rosner, R., & Sindhi, S. (2018). Online education: Worldwide status, challenges, trends, and implications. Journal of Global Information Technology Management, 21(4). 233 241. https://doi.org/10.1080/1097198X.2018.1542262
62. Parlak, B. (2017). Dijital çağda eğitim: olanaklar ve uygulamalar üzerine bir analiz [Education in digital age: An analysis on opportunities and applications] Suleyman Demirel University the Journal of Faculty of Economics and Administrative Sciences, Special Issue on Kayfor, 15, 1741-1759.
63. Patton, M.Q. (2018). Nitel araştırma ve değerlendirme yöntemleri [Qualitative research and evaluation methods]. (Demir,S.B. & Bütün, M., trans.). Pegem Akademi
64. Quast, J., Rubach, C., & Lazarides, R. (2021). Lehrkräfteeinschätzungen zu Unterrichtsqualität mit digitalen Medien: Zusammenhänge zur wahrgenommenen technischen Schulausstattung, Medienunterstützung,digitalen Kompetenzselbsteinschätzungen und Wertüberzeugungen [Teachers' perceptions of quality teaching with ICT: Associations with perceived school ICT equipment, ICT support, ICT competence beliefs and value beliefs]. Zeitschrift für Bildungsforschung. https://doi.org/10.1007/s35834-021-00313-7
65. Redecker, C., & Punie, Y. (2017). Digital Competence of Educators. JRC Science for Policy Report. https://core.ac.uk/download/pdf/132627227.pdf
66. Roemer, E., Schuberth, F., & Henseler, J. (2021). HTMT2–an improved criterion for assessing discriminant validity in structural equation modeling. Industrial management & data systems.
67. Rubach, C., & Lazarides, R. (2019). Eine Skala zur Selbsteinsch¨atzung digitaler Kompetenzen bei Lehramtsstudierenden: Entwicklung eines Instrumentes und die Validierung durch Konstrukte zur Mediennutzung und Werteüberzeugungen zur Nutzung digitaler Medien im Unterricht [A scale for self-percieved digital competence for student teachers: Development and validation with constructs like ICT, ICT values]. Zeitschrift Für Bildungsforschung, 2(78), 4. https://doi.org/10.1007/s35834-019-00248-0
68. Rubach, C., & Lazarides, R. (2021). Addressing 21st-century digital skills in schools–Development and validation of an instrument to measure teachers' basic ICT competence beliefs. Computers in Human Behavior, 118, 106636. https://doi.org/10.1016/j.chb.2020.106636
69. Seufert, S., Guggemos, J., & Sailer, M. (2021). Technology-related knowledge, skills, and attitudes of pre-and in-service teachers: The current situation and emerging trends. Computers in Human Behavior, 115, 106552. https://doi.org/10.1016/j.chb.2020.106552
70. Scherer, R., Tondeur, J., & Siddiq, F. (2017). On the quest for validity: Testing the factor structure and measurement invariance of the technology-dimensions in the Technological, Pedagogical, and Content Knowledge (TPACK) model. Computers & Education, 112, 1–17. https://doi.org/10.1016/j.compedu.2017.04.012
71. Seçer, İ. (2018). Psikolojik test geliştirme ve uyarlama süreci SPSS ve LISRELL uygulamaları [Psychological test development and adaptation process SPSS and LISRELL applications]. (2th edition). Anı Yayıncılık.
72. Şad, S.N., & Nalçacı, Ö.İ. (2015). Öğretmen Adaylarının Eğitimde Bilgi ve İletişim Teknolojilerini Kullanmaya İlişkin Yeterlilik Algıları [Prospective Teachers’ Perceived Competencies about Integrating Information and Communication Technologies into Education]. Mersin University Journal of the Faculty of Education, 11(1). http://abakus.inonu.edu.tr/xmlui/handle/123456789/17255
73. Tabachnick, B.G., & Fidell, L.S. (2013). Using multivariate statistics, 6th Ed. Pearson.
74. Tavşancıl, E. (2014). Tutumların ölçülmesi ve SPSS ile veri analizi. Ankara: Nobel Yayın Dağıtım
75. Tekindal, S. (2015). Duyuşsal özelliklerin ölçülmesi için araç oluşturma [Creating a tool for measuring affective traits] (3rd ed.). Pegem A Akademi.
76. Tezbaşaran, A. (2008). Likert tipi ölçek hazırlama kılavuzu. https://www.academia.edu/1288035/Likert_Tipi_Ölçek_Hazırlama_Kılavuzu
77. Tondeur, J., Aesaert, K., Prestridge, S., & Consuegra, E. (2018). A multilevel analysis of what matters in the training of pre-service teacher's ICT competencies. Computers & Education, 122, 32-42. https://doi.org/10.1016/j.compedu.2018.03.002
78. Tondeur, J., Van Braak, J., & Valcke, M. (2007). Curricula and the use of ICT in education: Two worlds apart? British Journal of educational technology, 38(6), 962-976. https://doi.org/10.1111/j.1467-8535.2006.00680.x
79. Tondeur, J., Valcke, M., & Van Braak, J. (2008). A multidimensional approach to determinants of computer use in primary education: Teacher and school characteristics. Journal of computer assisted learning, 24(6), 494-506. https://doi.org/10.1111/j.1365-2729.2008.00285.x
80. Trochim, W.M. & Donnelly, J.P. (2006). The research methods knowledge base. 3rd Edition, Atomic Dog, Cincinnati, OH.
81. Turkish Academy of Sciences (TÜBA) (Türkiye Bilimler Akademisi) (2002). Bilimsel araştırmada etik ve sorunları [Ethics and problems in scientific research]. TÜBA. http://www.tuba.gov.tr/tr/yayinlar/suresiz-yayinlar/raporlar/tuba-bilimsel-arastirmada-etik-ve-sorunlari
82. Türel, Y.K., Özdemir, T.Y., & Varol, F. (2017). Öğretmenlerin bilgi ve iletişim teknolojileri becerileri ölçeği: Güvenirlik ve geçerlik [Teachers’ ICT skills scale (TICTS): Reliability and validity]. Çukurova University Faculty of Educational Journal, 46(2), 503-516. http://doi.org/10.14812/cuefd.299864
83. United Nations Educational, Scientific and Cultural Organization (UNESCO). (2011). ICT Competency Framework for Teachers. https://unesdoc.unesco.org/ark:/48223/pf0000213475.
84. Vanderlinde, R., & Van Braak, J. (2010). The e-capacity of primary schools: Development of a conceptual model and scale construction from a school improvement perspective. Computers & Education, 55(2), 541 553. https://doi.org/10.1016/j.compedu.2010.02.016
85. Voogt, J., & Roblin, N.P. (2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299-321. https://doi.org/10.1080/00220272.2012.668938
86. Wang, H.Y., Sigerson, L., & Cheng, C. (2019). Digital nativity and information technology addiction: Age cohort versus individual difference approaches. Computers in Human Behavior, 90, 1-9. https://doi.org/10.1016/j.chb.2018.08.031
87. Wicherts, J.M. (2007). Group differences in intelligence test performance. Unpublished dissertation, University of Amsterdam.
88. Yurdakul, K.I., Dönmez, O., Altınok, A., & Odabaşı, H.F. (2013). Dijital ebeveynlik ve değişen roller [Digital parenting and changing roles]. Gaziantep University Journal of Social Sciences, 12(4), 883 896.
89. Yurdakul, I.K., Odabasi, H.F., Kilicer, K., Coklar, A.N., Birinci, G., & Kurt, A.A. (2012). The development, validity and reliability of TPACK-deep: A technological pedagogical content knowledge scale. Computers & Education, 58(3), 964 977. https://doi.org/10.1016/j.compedu.2011.10.012
90. Zhong, L. (2017). Indicators of digital leadership in the context of K-12 education. Journal of Educational Technology Development and Exchange (JETDE), 10(1), 27- 40.