Validity and reliability study of metacognitive listening strategies teaching self-efficacy scale for teachers

Year 2025, Volume: 12 Issue: 1, 147 - 164

Murat Ermiş , Şafak Uluçınar Sağır

https://doi.org/10.21449/ijate.1403112

Abstract

In this study, an attempt was made to develop a valid and reliable measurement tool to determine teachers' self-efficacy levels for teaching metacognitive listening strategies. The study group consisted of 205 teachers for EFA and 248 teachers for CFA. As a result of the analyzes, a scale consisting of 16 items with 4 factors was developed. It was determined that the scale explained 74.10 of the total variances. For EFA, Kaiser-Mayer-Olkin (KMO) test, Barlett test, total variance, item-total correlation, common factor variance, factor loadings, χ2/df RMSEA, SRMR, RMR, NFI, NNFI, CFI, GFI, AGFI, IFI, RFI, CR, AVE, MSV and ASV values and Cronbach Alpha statistics were performed. The KMO value of the scale is 0.915, the result of Barlett's Test of Sphericity is significant (p=0.000) and the Cronbach Alpha value is 0.932. Five of the fit indices showed excellent fit and six of them showed acceptable fit. The CR, AVE, MSV and ASV values showed that it provided divergent and convergent validity. After the analysis, it was concluded that the Self-Efficacy Scale for Teaching Metacognitive Listening Strategies for Teachers is valid and reliable.

Keywords

Metacognition, Metacognitive strategies, Listening, Self-efficacy, Teacher training

Ethical Statement

Amasya University, E-30640013-108.01-108629.

References

• Akin, A. (2006). Relationships between achievement goal orientations and metacognitive awareness, parental attitudes and academic achievement [Unpublished master's thesis]. Sakarya University.
• Anderson, J.C., & Gerbing, D.W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood con-firmatory factor analysis. Psychometrika, 49(2), 155-173.
• Anderson, N.J. (2002). The role of metacognition in second language teaching and learning. ERIC Digest.
• Arnold, C.L. (2014). Listening: The Forgotten Communication Skill. OMICS Publishing Group, 04(10). https://doi.org/10.4172/2165-7912.1000e155
• Aypay, A. (2010). The adaptation study of General Self-Efficacy (GSE) Scale to Turkish. Inonu University Journal of the Faculty of Education, 11(2), 113-132.
• Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychologi̇cal Revi̇ew, 84(2), 191. https://doi.org/10.1037/0033-295X.84.2.191
• Baumgartner, H., & Homburg, C. (1996). Applications of structural equation modeling in marketing and consumer research: A review. International Journal of Research in Marketing, 13(2), 139-161. https://doi.org/10.1016/0167-8116(95)00038-0
• Bentler, P.M. (1980). Multivariate analysis with latent variables: Causal modeling. Annual Review of Psychology, 31(1), 419-456.
• Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588-606. https://doi.org/10.1037/0033-2909.88.3.588
• Berman, R. (1994). Learners’ transfer of writing skills between languages. TESL Canada Journal, 12(1), 29-46.
• Birjandi, P., & Rahimi, A.H. (2012). The effect of metacognitive strategy instruction on the listening performance of EFL students. International Journal of Linguistics, 4(2), 495-517.
• Boomsma, A. (1985). Nonconvergence, improper solutions, and starting values in LISREL maximum likelihood estimation. Psychometrika, 50(2), 229-242.
• Bozorgian, H. (2012). Metacognitive instruction does improve listening comprehension. ISRN Education, 1-6.
• Brown, A.L. (1978). Knowing when, where, and how to remember: A problem of metacognition. In R. Glaser (Ed.), Advances in instructional psychology, (Vol. 7, pp. 55–111). Academic Press.
• Brown, A.L., & Palincsar, A.S. (1982). Inducing strategic learning from texts by means of informed, self-control training. Technical report No. 262.
• Brown, A.L., Bransford, J.D., Ferrara, R.A., & Campione, J.C. (1982). Learning, Remembering and Understanding. Technical Report No. 244.
• Brown, T.A. (2006). Confirmatory factor analysis for applied research. The Guilford Press.
• Bryman, A., & Cramer, D. (2004). Quantitative data analysis with SPSS 12 and 13: A guide for social scientists. Routledge. https://doi.org/10.4324/9780203498187
• Büyüköztürk, Ş. (2020). Sosyal bilimler için veri analizi kitabi (28. basım) [Handbook of data analysis for social sciences (28th Edition)]. Pegem Akademi Publishing.
• Byrne, B.M. (2001). Structural equation modeling with AMOS, EQS, and LISREL: Comparative approaches to testing for the factorial validity of a measuring instrument. International Journal of Testing, 1(1), 55-86. https://doi.org/10.1207/S15327574IJT0101_4
• Chamot, A. U., & Robbins, J. (2005). The CALLA Model: Strategies for ELL student success. Workshop for Region 10. New York City Board of Education.
• Chamot, A.U., O'Malley, M.J., Kupper, L. & Impink-Hernandez, M.V. (1987). A study of learning strategies in foreign language instruction: First Year Report. ERIC Number: ED352824.
• Cohen, A.D. (1996). Second language learning and use strategies: Clarifying the issues: Center for Advanced Research in Language Acquisition.
• Cohen, A., & Brooks-Carson, A. (2001). Research on direct versus translated writing: students’ strategies and their results. The Modern Language Journal, 85, 169 188. https://doi.org/10.1111/0026-7902.00103
• Cohen, A.D. (1996). Second language learning and use strategies: Clarifying the issues.
• Costa, A.L. (2008). Describing the habits of mind. In A.L. Costa, B. Kallick (Eds.), Learning and leading with habits of mind (15-41). Association for Supervision and Curriculum Development.
• Coşkun, A. (2010). The effect of metacognitive strategy training on the listening performance of beginner students. Novitas-ROYAL (Research on Youth and language), 4(1), 35-50.
• Cross, J. (2010). Metacognitive instruction for helping less-skilled listeners. ELT Journal, 65(4), 408-416. https://doi.org/10.1093/elt/ccq073
• Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2010). Sosyal bilimler için çok değişkenli 497 istatistik. SPSS ve LISREL uygulamaları [Multivariate statistics for social sciences. SPSS and LISREL applications]. Pegem Academy.
• De Winter, J.C.F, Dodou, D., & P.A. Wieringa (2009). Exploratory factor analysis with small sample sizes. Multivariate Behavioral Research, 44, 147-181.
• Desoete, A., & Roeyers, H. (2002). Off-line metacognition—A domain-specific retardation in young children with learning disabilities? Learning Disability Quarterly, 25(2), 123-139. https://doi.org/10.2307/1511279
• DeVellis, R.F., & Thorpe, C.T. (2021). Scale development: Theory and applications. Sage publications.
• Doğanay, A, & Kara, Z. (1995). Dimensions of thinking. Çukurova University Journal of Faculty of Education, 1(11), 25–38.
• Ermiş, M. (2019). The relationship between teachers' self-efficacy levels and institutional commitment [Unpublished master's thesis]. Amasya University.
• Field, A. (2013). Discovering statistics using IBM SPSS statistics. Sage.
• Flavell, J.H. (1976). Metacognitive aspects of problem solving. The Nature of Intelligence. Lawrence Erbaum.
• Flavell, J.H. (1979). Metacognitive and cognitive monitoring: A new area of cognitive devolopmental inquiry. American Psychologyst, 34(10), 906 911. https://doi.org/10.1037/0003-066X.34.10.906
• Fornell, C., & Larcker, D.F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. https://doi.org/10.1177/002224378101800104
• Ghapanchi, Z., & Taheryan, A. (2012). Roles of linguistic knowledge, metacognitive knowledge and metacognitive strategy use in speaking and listening proficiency of ıranian efl learners. World Journal of Education, 2(4), 64-75.
• Goh, C., & Taib, Y. (2006). Metacognitive instruction in listening for young learners. ELT Journal, 60(3), 222-232. https://doi.org/10.1093/elt/ccl002
• Gülebağlan, C. (2003). A research on the comparison of teachers' tendency to postpone the work to the last moment in terms of their perceptions of professional competence, professional experience and branches [Unpublished master's thesis]. Ankara University.
• Gürbüz, S. (2021). Amos ile yapısal eşitlik modellemesi. Seçkin Yayıncılık.
• Hacker, D.J., & Dunlosky, J. (2003). Not all metacognition is created equal. New Directions for Teaching and Learning, 95, 73-79.
• Haghighi, M., Rashtchi, M., & Birjandi, P. (2019). Developing and validating a questionnaire to assess strategic competence in EFL listening performance: A structural equation modeling approach. Research in English Language Pedagogy, 7(2), 336 362. https://doi.org/10.30486/relp.2019.665886
• Hair Jr, J.F., Hult, G.T.M., Ringle, C.M., & Sarstedt, M. (2014). A primer on partial least squares structural equation modeling (PLS-SEM). Sage publications.
• Hameed, H.A., & Cheruvalath, R. (2021). Metacognitive skills inventory (MSI): Development and validation. International Journal of Testing, 21(3 4), 154 181. https://doi.org/10.1080/15305058.2021.1986051
• Hattingh, S. (2014). The importance of teaching listening. OIU Journal of International Studies, 27(3), 97-110.
• Hauck, M. (2005). Metacognitive knowledge, metacognitive strategies, and CALL. CALL Research Perspectives, 65-86.
• Iacobucci, D. (2010). Structural equations modeling: Fit indices, sample size, and advanced topics. Journal of Consumer Psychology, 20(1), 90-98.
• Imhof, M. (2000). How to Listen More Efficiently: Self-monitoring Strategies in Listening. International Journal of Listening, 15(1), 2 19. https://doi.org/10.1080/10904018.2001.10499042
• İlhan, M., & Çetin, B. (2014). Comparing the Analysis Results of the Structural Equation Models (SEM) Conducted Using LISREL and AMOS. Journal of Measurement and Evaluation in Education and Psychology, 5(2), 26-42.
• Jackson, D.L. (2001). Sample size and number of parameter estimates in maximum likeli-hood confirmatory factor analysis: A Monte Carlo investigation. Structural Equa-tion Modeling, 8, 205-223.
• Kaplan, A., & Duran, M. (2016). Mathematical metacognition awareness inventory towards middle school students: Validity and reliability. Journal of Kazım Karabekir Education Faculty, 32, 1-17.
• Katrancı, M., & Yangın, B. (2012). Effects of teachıng metacognıtıon strategıes to lıstenıng omprehensıon skılls and attıtude toward lıstenıng. Adiyaman University Journal of Social Sciences, (11), 733-771.
• Kim, S.H. (2013). Metacognitive knowledge in second language writing. [Unpublished master's thesis] Michigan State University.
• Klassen, R.M., & Tze, V.M. (2014). Teachers’ self-efficacy, personality, and teaching effectiveness: A meta analysis. Educational Research Review, 12, 59 76. https://doi.org/10.1016/j.edurev.2014.06.001
• Kline, P. (1994). An easy guide to factor analysis. Routledge. https://doi.org/10.4324/9781315788135
• Kline, R.B. (2015). Principles and practice of structural equation modeling. Guilford publications.
• Kurita, T. (2012). Issues in second language listening comprehension and the pedagogical implications. Accents Asia, 5(1), 30-44.
• Küçükahmet, L. (Ed.) (2001). Dinleme Becerisi [Listening skills]. Nobel Publication Distribution.
• Liu, J. (2010). Language learning strategies and its training model. International Education Studies, 3(3), 100-104.
• MacCallum, R.C., Widaman, K.F., Preacher, K.J., & Hong, S. (2001). Sample size in factor analysis: The role of model error. Multivariate behavioral research, 36(4), 611-637. https://doi.org/10.1207/S15327906MBR3604_06
• Malureanu, F., & Enachi-Vasluianu, L. (2016). The ımportance of elements of actıve lıstenıng ın dıdactıc communıcatıon: a student’s perspectıve. In CBU International Conference Proceedings., 4, 332-335. https://doi.org/10.12955/cbup.v4.776
• Manchon, R.M., Murphy, L., & de Larios, J.R. (2007). Lexical retrieval processes and strategies in second language writing: A synthesis of empirical research. International Journal of English Studies, 7(2), 149-174.
• Marsh, H.W., Hau, K.T., Artelt, C., Baumert, J., & Peschar, J.L. (2006). OECD's brief self-report measure of educational psychology's most useful affective constructs: Cross-cultural, psychometric comparisons across 25 countries. International Journal of Testing, 6(4), 311-360. https://doi.org/10.1207/s15327574ijt0604_1
• McDonald, R.P., & Ho, M.H.R. (2002). Principles and practice in reporting structural equation analyses. Psychological methods, 7(1), 64.
• Melanlıoğlu, D. (2011). Impact of the metacognitive strategy instruction on secondary school students listening skill [Unpublished Phd dissertation]. Gazi University.
• Nix, J.M.L. (2016). Measuring latent listening strategies: Development and validation of the EFL listening strategy inventory. System, 57, 79 97. https://doi.org/10.1016/j.system.2016.02.001
• Nunnally, J.C. (1978). An overview of psychological measurement. Clinical diagnosis of mental disorders: A handbook. 97-146.
• O’Malley, M.J., Chamot, A.U., Walker, C., Russo, R.P. & Kupper, L. (1987). The role of learning strategies in second language acquisition: A selected literature review. ARI. U.S. Army Research Institute for the Behavioral and Social Sciences, Technical Report 744.
• Okur, A., & Azizoğlu, N.İ. (2016). Metacognitive Listening Strategies Instrument: Validity and Reliability Study. Mehmet Akif Ersoy University Journal of Education Faculty, 40, 113-124.
• O'Neil Jr, H.F., & Abedi, J. (1996). Reliability and validity of a state metacognitive inventory: Potential for alternative assessment. The journal of educational research, 89(4), 234-245. https://doi.org/10.1080/00220671.1996.9941208
• Osborne, J.W. (2015). What is rotating in exploratory factor analysis? Practical Assessment, Research, and Evaluation, 20(1), 1-7. https://doi.org/10.7275/hb2g-m060
• Raykov, T., & Marcoulides, G.A. (2006). On multilevel model reliability estimation from the perspective of structural equation modeling. Structural Equation Modeling, 13(1), 130-141. https://doi.org/10.1207/s15328007sem1301_7
• Rost, M. (2001). Listening. Cambridge University Press, 7 13. https://doi.org/10.1017/cbo9780511667206.002
• Rost, M. (2013). Teaching and researching: Listening. Routledge.
• Rubin, J. (2001). Language learner self-management. Journal of Asian Pacific Communication, 11(1), 25–37. https://doi.org/10.1075/japc.11.1.05rub
• Sarıkaya, B. (2021). Investigation of 7th grade elementary school students' metacognitive awareness levels and vocabulary learning strategies they use in English lessons. [Unpublished Phd dissertation]. İnönü University.
• Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74.
• Schraw, G. (1998). Promoting General Metacognitive Awareness. Instructional Science. 26(1 2), 113–125. https://doi.org/10.1023/A:1003044231033
• Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational psychology review, 7, 351-371. http://dx.doi.org/10.1007/s10648-017-9413-7
• Schreiber, J.B., Nora, A., Stage, F.K., Barlow, E.A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323-338. https://doi.org/10.3200/JOER.99.6.323-338
• Stevens, J. (2002). Applied multivariate statistics for the social sciences (Vol. 4). Lawrence Erlbaum Associates.
• Taylor, S. (1999). Better learning through better thinking: Developing students' metacognitive abilities. Journal of College Reading and Learning, 30(1), 34 45. https://doi.org/10.1080/10790195.1999.10850084
• Thomas, G., Anderson, D., & Nashon, S. (2008). Development of an instrument designed to investigate elements of science students’ metacognition, self‐efficacy and learning processes: The SEMLI‐S. International Journal of Science Education, 30(13), 1701-1724.
• Topaç, E. (2019). The effect of metacognitive listening strategy instruction on EFL learnerslistening comprehension and awareness levels and the role of ted talks as a listening resource on studentsperceptions [Unpublished master's thesis]. Yeditepe University.
• Tosun, A., & Irak, M. (2008). Adaptation, validity, and reliability of the metacognition questionnaire-30 for the turkish population, and its relationship to anxiety and obsessive-compulsive symptoms. Turkish Journal of Psychiatry, 19(1), 67-80.
• Vandergrift, L. (2003). Orchestrating strategy use: Toward a model of the skilled second language listener. Language Learning, 53(4), 463-496. https://doi.org/10.1111/1467-9922.00232
• Vandergrift, L., Goh, C.C.M., Mareschal, C.J., & Tafaghodtari, M.H. (2006). The metacognitive awareness listening questionnaire: Development and validation. Language Learning, 56(3), 431–462. https://doi.org/10.1111/j.1467-9922.2006.00373.x
• Wenden, A.L. (1998). Metacognitive knowledge and language learning. Applied Linguistics, 19(4), 515-537. https://doi.org/10.1093/applin/19.4.515
• Weston, R., & Gore Jr, P.A. (2006). A brief guide to structural equation modeling. The Counseling Psychologist, 34(5), 719-751. https://doi.org/10.1177/0011000006286345
• Wey, S. (1998). The effects of goal orientations, metacognition, self-efficacy and effort on writing achievement. University of Southern California.
• Wolfersberger, M. (2003). L1 to L2 writing process and strategy transfer: a look at lower proficiency writers. TESL-EJ, 7(2), 1-12.
• Yaşlıoğlu, M.M. (2017). Factor analysis and validity in social sciences: Application of exploratory and confirmatory factor analyses. Istanbul Business Research, 46, 74-85.
• Yıldırım A., Şimşek, H. (2008). Sosyal bilimlerde nitel araştırma yöntemleri [Qualitative research methods in social sciences]. Seçkin Yayıncılık.
• Yurdubakan, İ. (2010) Eğitimde kullanılan ölçme araçların nitelikleri [Qualifications of measurement tools used in education]. In M. Gömleksiz & S. Erkan (Eds.). Eğitimde ölçme ve değerlendirme [Measurement and evaluation in education]. (pp. 33-66). Nobel.
• Yurdugül, H. (2005). Ölçek geliştirme çalışmalarında kapsam geçerliği için kapsam geçerlik indekslerinin kullanılması [Using content validity indices for content validity in scale development studies]. XIV. National Congress of Educational Sciences. Denizli.
• Zhang, Z., & Zhang, L.J. (2011). Developing a listening comprehension problem scale for university students’ metacognitive awareness. The Journal of Asia TEFL, 8(3), 161-189.