Validity and reliability study of metacognitive listening strategies teaching self-efficacy scale for teachers

Öz

In this study, an attempt was made to develop a valid and reliable measurement tool to determine teachers' self-efficacy levels for teaching metacognitive listening strategies. The study group consisted of 205 teachers for EFA and 248 teachers for CFA. As a result of the analyzes, a scale consisting of 16 items with 4 factors was developed. It was determined that the scale explained 74.10 of the total variances. For EFA, Kaiser-Mayer-Olkin (KMO) test, Barlett test, total variance, item-total correlation, common factor variance, factor loadings, χ2/df RMSEA, SRMR, RMR, NFI, NNFI, CFI, GFI, AGFI, IFI, RFI, CR, AVE, MSV and ASV values and Cronbach Alpha statistics were performed. The KMO value of the scale is 0.915, the result of Barlett's Test of Sphericity is significant (p=0.000) and the Cronbach Alpha value is 0.932. Five of the fit indices showed excellent fit and six of them showed acceptable fit. The CR, AVE, MSV and ASV values showed that it provided divergent and convergent validity. After the analysis, it was concluded that the Self-Efficacy Scale for Teaching Metacognitive Listening Strategies for Teachers is valid and reliable.

Anahtar Kelimeler

• Metacognition
• Metacognitive strategies
• Listening
• Self-efficacy
• Teacher training

References

1. Akin, A. (2006). Relationships between achievement goal orientations and metacognitive awareness, parental attitudes and academic achievement [Unpublished master's thesis]. Sakarya University.
2. Anderson, J.C., & Gerbing, D.W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood con-firmatory factor analysis. Psychometrika, 49(2), 155-173.
3. Anderson, N.J. (2002). The role of metacognition in second language teaching and learning. ERIC Digest.
4. Arnold, C.L. (2014). Listening: The Forgotten Communication Skill. OMICS Publishing Group, 04(10). https://doi.org/10.4172/2165-7912.1000e155
5. Aypay, A. (2010). The adaptation study of General Self-Efficacy (GSE) Scale to Turkish. Inonu University Journal of the Faculty of Education, 11(2), 113-132.
6. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychologi̇cal Revi̇ew, 84(2), 191. https://doi.org/10.1037/0033-295X.84.2.191
7. Baumgartner, H., & Homburg, C. (1996). Applications of structural equation modeling in marketing and consumer research: A review. International Journal of Research in Marketing, 13(2), 139-161. https://doi.org/10.1016/0167-8116(95)00038-0
8. Bentler, P.M. (1980). Multivariate analysis with latent variables: Causal modeling. Annual Review of Psychology, 31(1), 419-456.

9. Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588-606. https://doi.org/10.1037/0033-2909.88.3.588
10. Berman, R. (1994). Learners’ transfer of writing skills between languages. TESL Canada Journal, 12(1), 29-46.
11. Birjandi, P., & Rahimi, A.H. (2012). The effect of metacognitive strategy instruction on the listening performance of EFL students. International Journal of Linguistics, 4(2), 495-517.
12. Boomsma, A. (1985). Nonconvergence, improper solutions, and starting values in LISREL maximum likelihood estimation. Psychometrika, 50(2), 229-242.
13. Bozorgian, H. (2012). Metacognitive instruction does improve listening comprehension. ISRN Education, 1-6.
14. Brown, A.L. (1978). Knowing when, where, and how to remember: A problem of metacognition. In R. Glaser (Ed.), Advances in instructional psychology, (Vol. 7, pp. 55–111). Academic Press.
15. Brown, A.L., & Palincsar, A.S. (1982). Inducing strategic learning from texts by means of informed, self-control training. Technical report No. 262.
16. Brown, A.L., Bransford, J.D., Ferrara, R.A., & Campione, J.C. (1982). Learning, Remembering and Understanding. Technical Report No. 244.
17. Brown, T.A. (2006). Confirmatory factor analysis for applied research. The Guilford Press.
18. Bryman, A., & Cramer, D. (2004). Quantitative data analysis with SPSS 12 and 13: A guide for social scientists. Routledge. https://doi.org/10.4324/9780203498187
19. Büyüköztürk, Ş. (2020). Sosyal bilimler için veri analizi kitabi (28. basım) [Handbook of data analysis for social sciences (28th Edition)]. Pegem Akademi Publishing.
20. Byrne, B.M. (2001). Structural equation modeling with AMOS, EQS, and LISREL: Comparative approaches to testing for the factorial validity of a measuring instrument. International Journal of Testing, 1(1), 55-86. https://doi.org/10.1207/S15327574IJT0101_4
21. Chamot, A. U., & Robbins, J. (2005). The CALLA Model: Strategies for ELL student success. Workshop for Region 10. New York City Board of Education.
22. Chamot, A.U., O'Malley, M.J., Kupper, L. & Impink-Hernandez, M.V. (1987). A study of learning strategies in foreign language instruction: First Year Report. ERIC Number: ED352824.
23. Cohen, A.D. (1996). Second language learning and use strategies: Clarifying the issues: Center for Advanced Research in Language Acquisition.
24. Cohen, A., & Brooks-Carson, A. (2001). Research on direct versus translated writing: students’ strategies and their results. The Modern Language Journal, 85, 169 188. https://doi.org/10.1111/0026-7902.00103
25. Cohen, A.D. (1996). Second language learning and use strategies: Clarifying the issues.
26. Costa, A.L. (2008). Describing the habits of mind. In A.L. Costa, B. Kallick (Eds.), Learning and leading with habits of mind (15-41). Association for Supervision and Curriculum Development.
27. Coşkun, A. (2010). The effect of metacognitive strategy training on the listening performance of beginner students. Novitas-ROYAL (Research on Youth and language), 4(1), 35-50.
28. Cross, J. (2010). Metacognitive instruction for helping less-skilled listeners. ELT Journal, 65(4), 408-416. https://doi.org/10.1093/elt/ccq073
29. Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2010). Sosyal bilimler için çok değişkenli 497 istatistik. SPSS ve LISREL uygulamaları [Multivariate statistics for social sciences. SPSS and LISREL applications]. Pegem Academy.
30. De Winter, J.C.F, Dodou, D., & P.A. Wieringa (2009). Exploratory factor analysis with small sample sizes. Multivariate Behavioral Research, 44, 147-181.
31. Desoete, A., & Roeyers, H. (2002). Off-line metacognition—A domain-specific retardation in young children with learning disabilities? Learning Disability Quarterly, 25(2), 123-139. https://doi.org/10.2307/1511279
32. DeVellis, R.F., & Thorpe, C.T. (2021). Scale development: Theory and applications. Sage publications.
33. Doğanay, A, & Kara, Z. (1995). Dimensions of thinking. Çukurova University Journal of Faculty of Education, 1(11), 25–38.
34. Ermiş, M. (2019). The relationship between teachers' self-efficacy levels and institutional commitment [Unpublished master's thesis]. Amasya University.
35. Field, A. (2013). Discovering statistics using IBM SPSS statistics. Sage.
36. Flavell, J.H. (1976). Metacognitive aspects of problem solving. The Nature of Intelligence. Lawrence Erbaum.
37. Flavell, J.H. (1979). Metacognitive and cognitive monitoring: A new area of cognitive devolopmental inquiry. American Psychologyst, 34(10), 906 911. https://doi.org/10.1037/0003-066X.34.10.906
38. Fornell, C., & Larcker, D.F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. https://doi.org/10.1177/002224378101800104
39. Ghapanchi, Z., & Taheryan, A. (2012). Roles of linguistic knowledge, metacognitive knowledge and metacognitive strategy use in speaking and listening proficiency of ıranian efl learners. World Journal of Education, 2(4), 64-75.
40. Goh, C., & Taib, Y. (2006). Metacognitive instruction in listening for young learners. ELT Journal, 60(3), 222-232. https://doi.org/10.1093/elt/ccl002
41. Gülebağlan, C. (2003). A research on the comparison of teachers' tendency to postpone the work to the last moment in terms of their perceptions of professional competence, professional experience and branches [Unpublished master's thesis]. Ankara University.
42. Gürbüz, S. (2021). Amos ile yapısal eşitlik modellemesi. Seçkin Yayıncılık.
43. Hacker, D.J., & Dunlosky, J. (2003). Not all metacognition is created equal. New Directions for Teaching and Learning, 95, 73-79.
44. Haghighi, M., Rashtchi, M., & Birjandi, P. (2019). Developing and validating a questionnaire to assess strategic competence in EFL listening performance: A structural equation modeling approach. Research in English Language Pedagogy, 7(2), 336 362. https://doi.org/10.30486/relp.2019.665886
45. Hair Jr, J.F., Hult, G.T.M., Ringle, C.M., & Sarstedt, M. (2014). A primer on partial least squares structural equation modeling (PLS-SEM). Sage publications.
46. Hameed, H.A., & Cheruvalath, R. (2021). Metacognitive skills inventory (MSI): Development and validation. International Journal of Testing, 21(3 4), 154 181. https://doi.org/10.1080/15305058.2021.1986051
47. Hattingh, S. (2014). The importance of teaching listening. OIU Journal of International Studies, 27(3), 97-110.
48. Hauck, M. (2005). Metacognitive knowledge, metacognitive strategies, and CALL. CALL Research Perspectives, 65-86.
49. Iacobucci, D. (2010). Structural equations modeling: Fit indices, sample size, and advanced topics. Journal of Consumer Psychology, 20(1), 90-98.
50. Imhof, M. (2000). How to Listen More Efficiently: Self-monitoring Strategies in Listening. International Journal of Listening, 15(1), 2 19. https://doi.org/10.1080/10904018.2001.10499042
51. İlhan, M., & Çetin, B. (2014). Comparing the Analysis Results of the Structural Equation Models (SEM) Conducted Using LISREL and AMOS. Journal of Measurement and Evaluation in Education and Psychology, 5(2), 26-42.
52. Jackson, D.L. (2001). Sample size and number of parameter estimates in maximum likeli-hood confirmatory factor analysis: A Monte Carlo investigation. Structural Equa-tion Modeling, 8, 205-223.
53. Kaplan, A., & Duran, M. (2016). Mathematical metacognition awareness inventory towards middle school students: Validity and reliability. Journal of Kazım Karabekir Education Faculty, 32, 1-17.
54. Katrancı, M., & Yangın, B. (2012). Effects of teachıng metacognıtıon strategıes to lıstenıng omprehensıon skılls and attıtude toward lıstenıng. Adiyaman University Journal of Social Sciences, (11), 733-771.
55. Kim, S.H. (2013). Metacognitive knowledge in second language writing. [Unpublished master's thesis] Michigan State University.
56. Klassen, R.M., & Tze, V.M. (2014). Teachers’ self-efficacy, personality, and teaching effectiveness: A meta analysis. Educational Research Review, 12, 59 76. https://doi.org/10.1016/j.edurev.2014.06.001
57. Kline, P. (1994). An easy guide to factor analysis. Routledge. https://doi.org/10.4324/9781315788135
58. Kline, R.B. (2015). Principles and practice of structural equation modeling. Guilford publications.
59. Kurita, T. (2012). Issues in second language listening comprehension and the pedagogical implications. Accents Asia, 5(1), 30-44.
60. Küçükahmet, L. (Ed.) (2001). Dinleme Becerisi [Listening skills]. Nobel Publication Distribution.
61. Liu, J. (2010). Language learning strategies and its training model. International Education Studies, 3(3), 100-104.
62. MacCallum, R.C., Widaman, K.F., Preacher, K.J., & Hong, S. (2001). Sample size in factor analysis: The role of model error. Multivariate behavioral research, 36(4), 611-637. https://doi.org/10.1207/S15327906MBR3604_06
63. Malureanu, F., & Enachi-Vasluianu, L. (2016). The ımportance of elements of actıve lıstenıng ın dıdactıc communıcatıon: a student’s perspectıve. In CBU International Conference Proceedings., 4, 332-335. https://doi.org/10.12955/cbup.v4.776
64. Manchon, R.M., Murphy, L., & de Larios, J.R. (2007). Lexical retrieval processes and strategies in second language writing: A synthesis of empirical research. International Journal of English Studies, 7(2), 149-174.
65. Marsh, H.W., Hau, K.T., Artelt, C., Baumert, J., & Peschar, J.L. (2006). OECD's brief self-report measure of educational psychology's most useful affective constructs: Cross-cultural, psychometric comparisons across 25 countries. International Journal of Testing, 6(4), 311-360. https://doi.org/10.1207/s15327574ijt0604_1
66. McDonald, R.P., & Ho, M.H.R. (2002). Principles and practice in reporting structural equation analyses. Psychological methods, 7(1), 64.
67. Melanlıoğlu, D. (2011). Impact of the metacognitive strategy instruction on secondary school students listening skill [Unpublished Phd dissertation]. Gazi University.
68. Nix, J.M.L. (2016). Measuring latent listening strategies: Development and validation of the EFL listening strategy inventory. System, 57, 79 97. https://doi.org/10.1016/j.system.2016.02.001
69. Nunnally, J.C. (1978). An overview of psychological measurement. Clinical diagnosis of mental disorders: A handbook. 97-146.
70. O’Malley, M.J., Chamot, A.U., Walker, C., Russo, R.P. & Kupper, L. (1987). The role of learning strategies in second language acquisition: A selected literature review. ARI. U.S. Army Research Institute for the Behavioral and Social Sciences, Technical Report 744.
71. Okur, A., & Azizoğlu, N.İ. (2016). Metacognitive Listening Strategies Instrument: Validity and Reliability Study. Mehmet Akif Ersoy University Journal of Education Faculty, 40, 113-124.
72. O'Neil Jr, H.F., & Abedi, J. (1996). Reliability and validity of a state metacognitive inventory: Potential for alternative assessment. The journal of educational research, 89(4), 234-245. https://doi.org/10.1080/00220671.1996.9941208
73. Osborne, J.W. (2015). What is rotating in exploratory factor analysis? Practical Assessment, Research, and Evaluation, 20(1), 1-7. https://doi.org/10.7275/hb2g-m060
74. Raykov, T., & Marcoulides, G.A. (2006). On multilevel model reliability estimation from the perspective of structural equation modeling. Structural Equation Modeling, 13(1), 130-141. https://doi.org/10.1207/s15328007sem1301_7
75. Rost, M. (2001). Listening. Cambridge University Press, 7 13. https://doi.org/10.1017/cbo9780511667206.002
76. Rost, M. (2013). Teaching and researching: Listening. Routledge.
77. Rubin, J. (2001). Language learner self-management. Journal of Asian Pacific Communication, 11(1), 25–37. https://doi.org/10.1075/japc.11.1.05rub
78. Sarıkaya, B. (2021). Investigation of 7th grade elementary school students' metacognitive awareness levels and vocabulary learning strategies they use in English lessons. [Unpublished Phd dissertation]. İnönü University.
79. Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74.
80. Schraw, G. (1998). Promoting General Metacognitive Awareness. Instructional Science. 26(1 2), 113–125. https://doi.org/10.1023/A:1003044231033
81. Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational psychology review, 7, 351-371. http://dx.doi.org/10.1007/s10648-017-9413-7
82. Schreiber, J.B., Nora, A., Stage, F.K., Barlow, E.A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323-338. https://doi.org/10.3200/JOER.99.6.323-338
83. Stevens, J. (2002). Applied multivariate statistics for the social sciences (Vol. 4). Lawrence Erlbaum Associates.
84. Taylor, S. (1999). Better learning through better thinking: Developing students' metacognitive abilities. Journal of College Reading and Learning, 30(1), 34 45. https://doi.org/10.1080/10790195.1999.10850084
85. Thomas, G., Anderson, D., & Nashon, S. (2008). Development of an instrument designed to investigate elements of science students’ metacognition, self‐efficacy and learning processes: The SEMLI‐S. International Journal of Science Education, 30(13), 1701-1724.
86. Topaç, E. (2019). The effect of metacognitive listening strategy instruction on EFL learnerslistening comprehension and awareness levels and the role of ted talks as a listening resource on studentsperceptions [Unpublished master's thesis]. Yeditepe University.
87. Tosun, A., & Irak, M. (2008). Adaptation, validity, and reliability of the metacognition questionnaire-30 for the turkish population, and its relationship to anxiety and obsessive-compulsive symptoms. Turkish Journal of Psychiatry, 19(1), 67-80.
88. Vandergrift, L. (2003). Orchestrating strategy use: Toward a model of the skilled second language listener. Language Learning, 53(4), 463-496. https://doi.org/10.1111/1467-9922.00232
89. Vandergrift, L., Goh, C.C.M., Mareschal, C.J., & Tafaghodtari, M.H. (2006). The metacognitive awareness listening questionnaire: Development and validation. Language Learning, 56(3), 431–462. https://doi.org/10.1111/j.1467-9922.2006.00373.x
90. Wenden, A.L. (1998). Metacognitive knowledge and language learning. Applied Linguistics, 19(4), 515-537. https://doi.org/10.1093/applin/19.4.515
91. Weston, R., & Gore Jr, P.A. (2006). A brief guide to structural equation modeling. The Counseling Psychologist, 34(5), 719-751. https://doi.org/10.1177/0011000006286345
92. Wey, S. (1998). The effects of goal orientations, metacognition, self-efficacy and effort on writing achievement. University of Southern California.
93. Wolfersberger, M. (2003). L1 to L2 writing process and strategy transfer: a look at lower proficiency writers. TESL-EJ, 7(2), 1-12.
94. Yaşlıoğlu, M.M. (2017). Factor analysis and validity in social sciences: Application of exploratory and confirmatory factor analyses. Istanbul Business Research, 46, 74-85.
95. Yıldırım A., Şimşek, H. (2008). Sosyal bilimlerde nitel araştırma yöntemleri [Qualitative research methods in social sciences]. Seçkin Yayıncılık.
96. Yurdubakan, İ. (2010) Eğitimde kullanılan ölçme araçların nitelikleri [Qualifications of measurement tools used in education]. In M. Gömleksiz & S. Erkan (Eds.). Eğitimde ölçme ve değerlendirme [Measurement and evaluation in education]. (pp. 33-66). Nobel.
97. Yurdugül, H. (2005). Ölçek geliştirme çalışmalarında kapsam geçerliği için kapsam geçerlik indekslerinin kullanılması [Using content validity indices for content validity in scale development studies]. XIV. National Congress of Educational Sciences. Denizli.
98. Zhang, Z., & Zhang, L.J. (2011). Developing a listening comprehension problem scale for university students’ metacognitive awareness. The Journal of Asia TEFL, 8(3), 161-189.