A Reliability Generalization Meta-Analysis of Mathematics Self Efficacy- Revised (MSES-R) Scale)*

Abstract

The Mathematics Self-Efficacy-Revised (MSES-R) scale (Kranzler & Pajares, 1997) has been widely used to measure students' mathematics self-efficacy. This study aims to determine the common reliability coefficient using reliability generalization, a type of meta-analysis. The research process was organized using PRISMA guidelines. The average consistency reliability was calculated as 0.901 with a 95% CI [0.864; 0.939] using a random effect model. The studies showed heterogeneity [Q (6) = 219.788; p < .01; I2 = 96.99%]. The meta-analysis predicts that certain moderator variables significantly affect the scale's reliability. The reliability generalization analysis of the MSES-R scale reveals the different properties of the MSES-R inventory using a mixed effect model. It was found that sample type and department context significantly affect the Cronbach alpha coefficient of the MSES-R scale (p < 0.01), while the study location variable does not significantly affect the common coefficient (p > 0.01).

Keywords

• Cronbach’s alpha
• mathematics self-efficacy
• meta-analysis
• reliability generalization

Gözden Geçirilmiş Matematik Öz Yeterlik (MSES-R) Ölçeğinin Güvenirlik Genellemesi Meta-Analizi

Abstract

Matematik Öz-Yeterlik Revize Ölçeği (MSES-R), Kranzler ve Pajares (1997) tarafından geliştirilen ve öğrencilerin matematik öz-yeterlik düzeylerini ölçmek amacıyla yaygın şekilde kullanılan bir ölçme aracıdır. Bu çalışma, güvenirlik genellemesi olarak bilinen bir meta-analiz türü aracılığıyla, MSES-R ölçeğinin ortak güvenirlik katsayısını belirlemeyi amaçlamaktadır. Araştırma süreci, sistematik derleme ve meta-analiz çalışmalarına yönelik PRISMA yönergeleri doğrultusunda yapılandırılmıştır. Meta-analiz sonuçlarına göre, rastgele etkiler modeli kullanılarak hesaplanan ortalama iç tutarlılık katsayısı %95 güven aralığında [0.864; 0.939] olmak üzere 0.901’dir. Analize dahil edilen çalışmalar arasında anlamlı düzeyde heterojenlik saptanmıştır [Q(6) = 219.788; p < .01; I² = %96.99]. Ayrıca, analiz kapsamında incelenen bazı moderatör değişkenlerin ölçeğin güvenilirliğini anlamlı düzeyde etkilediği görülmüştür. Örneklem türü ve çalışmanın yapıldığı bölüm bağlamı, MSES-R ölçeğine ilişkin Cronbach alfa katsayısını anlamlı biçimde etkilemektedir (p < .01). Öte yandan, çalışma yeri değişkeninin ortak güvenirlik katsayısı üzerinde anlamlı bir etkisinin bulunmadığı belirlenmiştir (p > .01). Bu bulgular, karma etki modeliyle yürütülen güvenilirlik genelleme analizinin, MSES-R ölçeğinin bağlamsal özelliklerine duyarlılığını ortaya koyduğunu göstermektedir.

Keywords

• Cronbach alfa
• matematik öz-yeterliği
• meta-analiz
• güvenilirlik genellemesi

References

1. Ayotola, A., & Adedeji, T. (2009). The relationship between mathematics self-efficacy and achievement in mathematics. Procedia - Social and Behavioral Sciences, 1(1), 953–957. https://doi.org/10.1016/j.sbspro.2009.01.16
2. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
3. Bandura, A. (1994). Self-efficacy. In V. S. Ramachaudran (Ed.), Encyclopedia of human behavior (Vol. 4, pp. 71–81). Academic Press.
4. Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
5. Betz, N. E., & Hackett, G. (1983). Mathematics Self-Efficacy Scale [Dataset]. In PsycTESTS Dataset. American Psychological Association (APA). https://doi.org/10.1037/t01563-000
6. *Blake, M. E., Masten, W. G., Henley, T. B., & Ball, S. E. (2015). Sources and influence of mathematics self-efficacy in Jamaican college students. Caribbean Journal of Psychology, 7(1), 24-42.
7. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2019). Introduction to meta-analysis (2nd ed.). Wiley.
8. *Briley, J. S. (2012). The relationships among mathematics teaching efficacy, mathematics self-efficacy, and mathematical beliefs for elementary pre-service teachers. Issues in the Undergraduate Mathematics Preparation of School Teachers: The Journal, 5, 1–13.

9. Capraro, M. M., Capraro, R. M., & Henson, R. K. (2001). Measurement error of scores on the mathematics anxiety rating scale across studies. Educational and Psychological Measurement, 61(3), 373–386. https://doi.org/10.1177/00131640121971266
10. Capraro, R. M., & Capraro, M. M. (2002). Myers-Briggs type indicator score reliability across studies: A meta-analytic reliability generalization study. Educational and Psychological Measurement, 62(4), 590–602. https://doi.org/10.1177/0013164402062004004
11. Caruso, J. C., Witkiewitz, K., Belcourt-Dittloff, A., & Gottlieb, J. D. (2001). Reliability of scores from the Eysenck personality questionnaire: A reliability generalization study. Educational and Psychological Measurement, 61(4), 675–689. https://doi.org/10.1177/00131640121971437
12. *Cebollero, M. P., Fora, N. M., Permanyer, L. F., & Olmos, J. G. (2012). La evaluación de la ansiedad ante las matemáticas y su efecto como interferidor del rendimiento académico en la asignatura de estadística en psicología. Revista del Congrés Internacional de Docència Universitària i Innovació (CIDUI), 1(1).
13. Cousin, S. L., & Henson, R. K. (2000, January 27–29). What is reliability generalization and why is important? [Paper presentation]. Annual Meeting of the Southwest Educational Research Association, Dallas, TX.
14. Crouch, M. (2016). Reliability generalization: Exploring score reliability variance with Ryff's scale of psychological well-being [Unpublished master’s thesis]. Brock University.
15. DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7(3), 177–188. https://doi.org/10.1016/0197-2456(86)90046-2
16. *Dobarro, R. V. (2007). Solução de problemas e tipos de mente matemática: Relações com as atitudes e crenças de auto-eficácia [Problem solving and types of mathematical mind: Relationships with attitudes and self-efficacy beliefs] [Unpublished doctoral dissertation]. Universidade Estadual de Campinas.
17. Eurostat. (n.d.). International Standard Classification of Education (ISCED). Retrieved March 8, 2024, from https://ec.europa.eu/eurostat/statistics-explained/index.php?title=International_Standard_Classification_of_Education_(ISCED)
18. *Finney, S. J., & Schraw, G. (2003). Self-efficacy beliefs in college statistics courses. Contemporary Educational Psychology, 28(2), 161–186. https://doi.org/10.1016/s0361-476x(02)00015-2
19. * Guàrdia-Olmos, J., Peró-Cebollero, M., Mancho-Fora, N., & Farràs-Permanye, L. (2012). Adaptació d’una bateria per a l’avaluació de prerequisits per a l’aprenentatge d’estadística en psicologia [Adaptation of a battery for the assessment of prerequisites for learning statistics in psychology]. REIRE: Revista d'Innovació i Recerca en Educació, 5(1), 42–58.
20. Hackett, G., & Betz, N. E. (1989). An exploration of the mathematics self-efficacy/mathematics performance correspondence. Journal for Research in Mathematics Education, 20(3), 261–273. https://doi.org/10.5951/jresematheduc.20.3.0261
21. Henson, R. K., & Hwang, D.-Y. (2002). Variability and prediction of measurement error in Kolb’s learning style inventory scores: A reliability generalization study. Educational and Psychological Measurement, 62(4), 712–727. https://doi.org/10.1177/0013164402062004011
22. Henson, R. K., & Thompson, B. (2002). Characterizing measurement error in scores across studies: Some recommendations for conducting “reliability generalization” studies. Measurement and Evaluation in Counseling and Development, 35(2), 113–127. https://doi.org/10.1080/07481756.2002.12069054
23. Henson, R. K., Kogan, L. R., & Vacha-Haase, T. (2001). A reliability generalization study of the Teacher Efficacy Scale and related instruments. Educational and Psychological Measurement, 61(3), 404–420. https://doi.org/10.1177/00131640121971284
24. Hess, T. J., McNab, A. L., & Basoglu, K. S. (2014). Reliability generalization of perceived ease of use, perceived usefulness, and behavioral intentions. MIS Quarterly, 38, 1-28. https://doi.org/10.25300/MISQ/2014/38.1.01
25. Holland, D. F., Kraha, A., Zientek, L. R., Nimon, K., Fulmore, J. A., Johnson, U. Y., ... & Henson, R. K. (2018). Reliability generalization of the motivated strategies for learning questionnaire: a meta-analytic view of reliability estimates. Sage Open, 8(3). https://doi.org/10.1177/2158244018802334
26. Kilgus, S. P., Eklund, K. R., Maggin, D. M., Taylor, C. N., & Allen, A. N. (2018). The student risk screening scale: A reliability and validity generalization meta-analysis. Journal of Emotional and Behavioral Disorders, 26(3), 143-155. https://doi.org/10.1177/1063426617710207
27. Kranzler, J. H., & Pajares, F. (1997). An exploratory factor analysis of the Mathematics Self-Efficacy Scale revised (MSES-R). Measurement and Evaluation in Counseling and Development, 29(4), 215–228. https://doi.org/10.1080/07481756.1997.12068906
28. Lane, G. G., White, A. E., & Henson, R. K. (2002). Expanding reliability generalization methods with KR-21 estimates: An RG study of the Coopersmith self-esteem inventory. Educational and Psychological Measurement, 62(4), 685–711. https://doi.org/10.1177/0013164402062004010
29. Leach, L. F., Henson, R. K., Odom, L. R., & Cagle, L. S. (2006). A reliability generalization study of the self-description questionnaire. Educational and Psychological Measurement, 66(2), 285–304. https://doi.org/10.1177/0013164405284030
30. Mahapoonyanont, N., Krahomwong, R., Kochakornjarupong, D., & Rachasong, W. (2010). Critical thinking abilities assessment tools: Reliability generalization. Procedia - Social and Behavioral Sciences, 2(2), 434–438. https://doi.org/10.1016/j.sbspro.2010.03.038
31. McLeod, D. B. (1992). Research on affect in mathematics education: A reconceptualization. In D. A. Grouws (Ed.), Handbook of Research on Mathematics Teaching and Learning (pp. 575-596). Macmillan.
32. Mji, A., & Alkhateeb, H. M. (2005). Combining reliability coefficients: Toward reliability generalization of the conceptions of mathematics questionnaire. Psychological Reports, 96(3), 627–634. https://doi.org/10.2466/pr0.96.3.627-634
33. Moore, N. M. (2005). Constructivism using group work and the impact on self-efficacy, intrinsic motivation, and group work skills on middle-school mathematics students [Doctoral dissertation]. Capella University.
34. Nilsson, J. E., Schmidt, C. K., & Meek, W. D. (2002). Reliability generalization: An examination of the career decision-making self-efficacy scale. Educational and Psychological Measurement, 62(4), 647–658. https://doi.org/10.1177/0013164402062004007
35. Onwuegbuzie, A. J., & Daniel, L. G. (2000, November 17). Reliability generalization: The importance of considering sample specificity, confident intervals, and subgroup differences. [Paper presentation]. Annual Meeting of the Mid-South Educational Research Association, Bowling Green, KY, United States.
36. Pajares, M. F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 62(3), 307–332. https://doi.org/10.3102/00346543062003307
37. Pajares, F. (1996). Self-efficacy beliefs and mathematical problem-solving of gifted students. Contemporary Educational Psychology, 21(4), 325–344. https://doi.org/10.1006/ceps.1996.0025
38. Pajares, F., & Graham, L. (1999). Self-efficacy, motivation constructs, and mathematics performance of entering middle school students. Contemporary Educational Psychology, 24(2), 124–139. https://doi.org/10.1006/ceps.1998.0991
39. Pajares, F., & Kranzler, J. (1995). Self-efficacy beliefs and general mental ability in mathematical problem-solving. Contemporary Educational Psychology, 20(4), 426–443. https://doi.org/10.1006/ceps.1995.1029
40. Pajares, F., & Miller, M. D. (1995). Mathematics self-efficacy and mathematics performances: The need for specificity of assessment. Journal of Counseling Psychology, 42(2), 190–198. https://doi.org/10.1037/0022-0167.42.2.190
41. Ross, M. E., Blackburn, M., & Forbes, S. (2005). Reliability generalization of the patterns of adaptive learning survey goal orientation scales. Educational and Psychological Measurement, 65(3), 451–464. https://doi.org/10.1177/0013164404272496
42. Schunk, D. H. (2009). Learning theories: An educational perspective (5th ed.). Pearson Education.
43. Taylor, R. (2012). Review of the motivated strategies for learning questionnaire (MSLQ) using reliability generalization techniques to assess scale reliability [Doctoral dissertation]. Auburn University.
44. Usher, E. L. (2009). Sources of middle school students’ self-efficacy in mathematics: A qualitative investigation. American Educational Research Journal, 46(1), 275–314. https://doi.org/10.3102/0002831208324517
45. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58(1), 6-20. https://doi.org/10.1177/0013164498058001002
46. Vacha-Haase, T., & Thompson, B. (2011). Score reliability: A retrospective look back at 12 years of reliability generalization studies. Measurement and Evaluation in Counseling and Development, 44(3), 159–168. https://doi.org/10.1177/0748175611409845
47. Vacha-Haase, T., Kogan, L. R., Tani, C. R., & Woodall, R. A. (2001). Reliability generalization: Exploring variation of reliability coefficients of MMPI clinical scales scores. Educational and Psychological Measurement, 61(1), 45-59. https://doi.org/10.1177/00131640121971059
48. Viechtbauer, W. (2005). Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 30(3), 261–293. https://doi.org/10.3102/10769986030003261
49. West African Examinations Council. (2005–2007). Chief examiners’ report. Yaba, Lagos: WAEC.
50. Williams, A. M., & Young, J. (2021). Reliability generalization meta-analyses in mathematics education research: A research synthesis. International Journal of Education in Mathematics, Science and Technology, 9(4), 741–759. https://doi.org/10.46328/ijemst.1434