A Reliability Generalization Meta-Analysis of the Mathematics Self-Efficacy

Abstract

Mathematical self-efficacy is an individual's belief in their ability to solve mathematical problems, understand mathematical concepts, and successfully complete mathematical tasks. This study uses reliability data reported in various studies that employed the “Mathematics Self-Efficacy Scale” (MSES) developed by Umay (2001) to determine individuals' levels of self-efficacy in mathematics. It aims to estimate the average (combined) reliability coefficient using this data. This study was conducted to estimate the average (combined) reliability coefficient using the reliability data reported in different studies that used this scale and to examine the reasons for the variation in the reliability coefficients reported in these studies. In this study, which employed reliability generalization, one of the meta-analysis methods, analyses were conducted by combining the transformed coefficient values obtained via the Bonett transformation from 32 Cronbach’s alpha coefficients derived from 31 independent studies with a total sample size of 10,072, using a random effects model. Publication bias analyses have shown that there were no statistically significant publication bias findings in this study. The average Cronbach's alpha value of the MSES was estimated to be .829 (95% CI [.812 - .844]), and this estimated value was found to be statistically significant (p <.01). Analyses conducted on heterogeneity determined that there was a high level of heterogeneity among the Cronbach's alpha coefficients obtained from the studies (I² = %88,98). Moderator analyses have shown that potential variables such as the sample group, publication status, research design, region, year of publication, sample size, percentage of men, and mean scale score do not significantly explain this observed heterogeneity. The substantial heterogeneity identified (I² = 88.72%) constitutes the central finding of this study, demonstrating that MSES reliability is not transportable across studies or populations. The 95% prediction interval (PI [.726, .893]) makes this practically concrete: a researcher using the MSES in a new study should expect an alpha value anywhere in this range, not necessarily near the pooled mean of .829. These findings carry direct implications for reliability reporting norms: journal editors, peer reviewers, and researchers using the MSES or similar instruments should treat sample-specific reliability estimation as a non-negotiable methodological standard rather than an optional practice.

Keywords

• Mathematics education
• Perceived self-efficacy in mathematics
• Meta-analysis
• Reliability generalization.

Ethical Statement

Since the study did not involve human practices, there was no need to obtain an ethics committee.

References

1. *Adal, A. A., & Yavuz, İ. (2017). The relationship between mathematics self efficacy and mathematics anxiety levels of middle school students. International Journal of Field Education, 3(1), 20-41. https://izlik.org/JA53SP42GY
2. *Akay, H., & Boz, N. (2010). The effect of problem posing oriented analyses-ıı course on the attitudes toward mathematics and mathematics self-efficacy of elementary prospective mathematics teachers. Australian Journal of Teacher Education, 35(1). https://doi.org/10.14221/ajte.2010v35n1.6
3. *Akay, H., & Boz, N. (2011). Examining the relationships among prospective primary school teachers’* attitude towards mathematics, mathematics self-effıcacy beliefs, teacher self-effıcacy beliefs. The Journal of Turkish Educational Sciences, 9(2), 281-312. https://izlik.org/JA79HK57BJ
4. *Ayan, A. (2014). The relationship between mathematics self-efficacy, motivations, anxieties and the attitudes for secondary school students [Master Thesis]. Balıkesir University, Institute of Science.
5. *Aydoğan-Yenmez, A., Koşum, O., & Gökçe, S. (2024). How does GeoGebra affect academic achievement and self-efficacy perception in exponential and logarithmic functions? HAYEF: Journal of Education, 21(2), 127-138.
6. *Ayyıldız Altınbaş, A., Solak, S., & Ertekin, E. (2025). The effect of multiple representations-based and problem-based instruction in linear algebra on pre-service teachers' dimensions of understanding and self-efficacy perceptions. Journal of Mathematics Teacher Education. Advance online publication. https://doi.org/10.1007/s10857-025-09696-0
8. Badenes-Ribera, L., Duro-García, C., López-Ibáñez, C., Martí-Vilar, M., & Sánchez-Meca, J. (2023). The adult prosocialness behavior scale: A reliability generalization meta-analysis. International Journal of Behavioral Development, 47(1), 59-71. https://doi.org/10.1177/01650254221128280

9. Balduzzi, S., Rücker, G., & Schwarzer, G. (2019). How to perform a meta-analysis with R: A practical tutorial. Evidence-Based Mental Health, 22(4), 153–160. https://doi.org/10.1136/ebmental-2019-300117
10. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191
11. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.
12. Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117–148. https://doi.org/10.1207/s15326985ep2802_3
13. Barnes, L. L., Harp, D., & Jung, W. S. (2002). Reliability generalization of scores on the Spielberger state-trait anxiety inventory. Educational and psychological measurement, 62(4), 603–618. https://doi.org/10.1177/0013164402062004005
14. Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50(4), 1088–1101. https://doi.org/10.2307/2533446
15. Bonett, D. G. (2002). Sample size requirements for testing and estimating coefficient alpha. Journal of Educational and Behavioral Statistics, 27(4), 335-340. https://doi.org/10.3102/10769986027004335
16. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley.
17. Botella, J., Suero, M., & Gambara, H. (2010). Psychometric inferences from a meta-analysis of reliability and internal consistency coefficients. Psychological Methods, 15(4), 386–397. https://doi.org/10.1037/a0019626
18. *Canbulut, S. (2023). Investigation of the relationship between the attitudes towards mathematics of primary school fourth grade ınformation and perceptions of mathematics self-efficacy [Master Thesis]. Gazi University, Institute of Educational Sciences.
19. *Cere, İ., İnci, E. N., Güngör, G., Gjoni, E., & Işıksal Bostan, M. (2025). Investigating high school students’mathematics attitude, self-efficacy, and anxiety regarding gender and grade level. Bartın University Journal of Faculty of Education, 14(3), 669-684. https://doi.org/10.14686/buefad.1424194
20. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284
21. *Cırıtcı, H. (2023). Examination of secondary school students' perceptions of mathematics course self-efficiency in Azerbaijan and Turkiye [Master Thesis]. Ankara University, Institute of Educational Sciences.
22. *Çalışkan, G. (2022). Investıgatıon of the relationship between vocational high school students' perceptions of mathematics self-effıcıency and success ın vocational mathematics [Master Thesis]. Sakarya University, Institute of Educational Sciences.
23. Çelik, F. (2023). Reliability generalization meta-analysis of the turkish version of fear of missing out scale abstract. Gumushane University e-journal of Faculty of Communication, 11(2), 848-877. https://doi.org/10.19145/e-gifder.1283655
24. *Çelik, H., C., Özdemir, F. & Bindak, R. (2024). The effect of problem posing attitude on problem solving attitude: The mediating role of mathematical self-efficacy. International Journal of Educational Spectrum 6(2), 264-281. https://doi.org/10.47806/ijesacademic.1464387
25. *Çimenci-Ateş, F. (2016). Assessment of the effects of math anxiety, attitude and self-efficacy beliefs on the 8th grade students' graph reading and interpretations [Master Thesis]. Afyon Kocatepe University, Institute of Social Sciences.
26. Dilek, H., & Gelbal, S. (2023). Reliability generalization meta analysis of the teachers’ sense of efficacy scale. Recep Tayyip Erdoğan Üniversitesi Eğitim Fakültesi Dergisi (REFAD), 3(1), 1-27. https://izlik.org/JA33HU57UP
27. Ding, Y., Klapp, A., & Yang Hansen, K. (2024). The importance of mathematics self-concept and self-efficacy for mathematics achievement: A comparison between public and independent schools in Sweden. Educational Psychology, 44(8), 872-892. https://doi.org/10.1080/01443410.2024.2410217
28. *Doruk, M., Özdemir, F., & Kaplan, A. (2015). The relationship between prospective mathematics teachers' conceptions on constructing mathematical proof and their self-efficacy beliefs towards mathematics. Kastamonu Education Journal, 23(2), 861-874. https://izlik.org/JA57DB43HF
29. Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341x.2000.00455.x
30. Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ (Clinical research ed.), 315(7109), 629–634.
31. Eser, M.T., Yurtçu, M., & Aksu G. (2020). Meta-analysis applications with R programming language and Jamovi. Pegem Academy Publishing.
32. Field, A. P. (2003). The problems in using fixed-effects models of meta-analysis on real-world data. Understanding Statistics: Statistical Issues in Psychology, Education, and the Social Sciences, 2(2), 105-124. https://doi.org/10.1207/S15328031US0202_02
33. Field, A. P., & Gillett, R. (2010). How to do a meta‐analysis. British Journal of Mathematical and Statistical Psychology, 63(3), 665-694. https://doi.org/10.1348/000711010X502733
34. *Görgün, S. (2020). Investigation of middle school students' math self efficacy perceptions and math problem posing attitudes [Master Thesis]. Muğla Sıtkı Koçman University, Institute of Educational Sciences.
35. Hackett, G., & Betz, N. E. (1989). An exploration of the mathematics self-efficacy/mathematics performance correspondence. Journal for Research in Mathematics Education, 20(3), 261–273. https://doi.org/10.2307/749515
36. Henson, R.K., & Thompson, B. (2002). Characterizing measurement error in scores across studies: some recommendations for conducting “reliability generalization” studies. Measurement and Evaluation in Counseling and Development, 35(2), 113–27. https://doi.org/10.1080/07481756.2002.12069054
37. Herset, M., Ghami, M. E., & Bjerke, A. H. (2023). The effect of level-marked mathematics tasks on students’ self-efficacy: An experimental study. Frontiers in Psychology, 14, 1116386. https://doi.org/10.3389/fpsyg.2023.1116386
38. *Isiksal, M. (2005). Pre-service teachers' performance in their university coursework and mathematical self-efficacy beliefs: What is the role of gender and year in program?. The Mathematics Educator, 15(2). https://doi.org/10.63301/tme.v15i2.1887
39. *Işıksal, M., & Çakıroğlu, E. (2006). Pre-service mathematics teachers’ efficacy beliefs toward mathematics and mathematics teaching. Hacettepe University Journal of Education, 31(31), 74-84. https://izlik.org/JA29JA44PR
40. *Kurtuluş, A., & Öztürk, B. (2017). The analysis of the effect of metacognitive awareness and mathematics self-efficacy perceptions on mathematics achievement of middle school students. Dicle University Journol of Ziya Gökalp Faculty of Education, 31, 762-778. https://doi.org/10.14582/DUZGEF.1840
41. Lee, H., Jang, Y., & Jeong, Y. (2023). Supportive Care Needs Survey: A reliability generalization meta-analysis. Palliative and Supportive Care, 21(4), 714–726. https://doi.org/10.1017/S1478951522001791
42. López-Ibáñez, C., López-Nicolás, R., Blázquez-Rincón, D. M., & Sánchez-Meca, J. (2024). Reliability generalization meta-analysis: Comparing different statistical methods. Current Psychology, 43(20), 18275-18293. https://doi.org/10.1007/s12144-023-05604-y
43. Mullen, B., Muellerleile, P., & Bryant, B. (2001). Cumulative meta-analysis: A consideration of ındicators of sufficiency and stability. Personality and Social Psychology Bulletin, 27(11), 1450-1462. https://doi.org/10.1177/01461672012711006
44. Nunnally, C. J., & Bernstein, I. H. (1994). Psychological theory (3rd ed.). McGraw-Hill.
45. Özdemir, F., & İlhan, A. (2025). Reliability generalization meta-analysis of the self-efficacy scale for geometry. Hacettepe University Journal of Education, 40(3), 174-186. https://doi.org/10.16986/HUJE.2025.543
46. *Özkal, N. (2020). Relationships between teachers’ motivational support, self-efficacy perception and academic achievement: high school mathematics course sample. Akdeniz Journal of Education, 3(2), 88-97. https://izlik.org/JA83HT39FZ
47. *Özmen, A., & Mumcu, H. Y. (2020). Investigation of experimental abstraction processes for slope concept according to students’ attitude, anxiety, motivation and self efficacy perceptions towards mathematics. Inonu University Journal of the Faculty of Education, 21(2), 785-800. https://doi.org/10.17679/inuefd.657175
48. *Özsoy, N., Kopar, T. N., & Onay, N. (2022, August). The effect of creative drama on self-efficacy perception in mathematics. In Conference book of SOCRATES 5th International Conference on Education, Management, Economy and Social Sciences (pp. 25–35).
49. *Özüdoğru, M. (2013). Prediction of 9th grade students' mathematics achievement [Master Thesis]. Ege University, Institute of Social Sciences.
50. Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543-578.
51. *Pişkin, M., & Durmuş, S. (2010). Prospective primary school teachers’self-efficacy beliefs about mathematics. Education Sciences, 5(3), 1190-1196. https://izlik.org/JA23TZ64PP
52. R Core Team. (2020). R: A language and environment for statistical computing [Computer software]. https://www.R-project.org/
53. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
54. Sánchez-Meca, J., López-López, J. A., & López-Pina, J. A. (2013). Some recommended statistical analytic practices when reliability generalization studies are conducted. British Journal of Mathematical and Statistical Psychology, 66(3), 402–425. https://doi.org/10.1111/j.2044-8317.2012.02057.x
55. Sánchez-Meca, J., Marín-Martínez, F., López-López, J. A., Núñez-Núñez, R. M., Rubio-Aparicio, M., López-García, J. J., López-Pina, J. A., Blázquez-Rincón, D. M., López-Ibáñez, C., & López-Nicolás, R. (2021). Improving the reporting quality of reliability generalization meta-analyses: The REGEMA checklist. Research synthesis methods, 12(4), 516–536. https://doi.org/10.1002/jrsm.1487
56. Sandoval-Lentisco, A., López-Nicolás, R., López-López, J. A., & Sánchez-Meca, J. (2023). Florida obsessive-compulsive inventory and children's florida obsessive compulsive inventory: A reliability generalization meta-analysis. Journal of clinical psychology, 79(1), 28–42. https://doi.org/10.1002/jclp.23416
57. Schmid, C.H., Stijnen, T., & White, I. R. (2021). General themes in meta-analysis. In C.H. Schmid, T. Stijnen, & I. R. White (Eds.), Handbook of meta-analysis (19-26). CRC Press.
58. Siegle, D., & McCoach, D. B. (2007). Increasing student mathematics self-efficacy through teacher training. Journal of advanced Academics, 18(2), 278-312. https://doi.org/10.4219/jaa-2007-353
59. Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0
60. Street, K.E.S., Malmberg, LE. & Schukajlow, S. Students’ mathematics self-efficacy: a scoping review. ZDM Mathematics Education, 56, 265–280 (2024). https://doi.org/10.1007/s11858-024-01548-0
61. *Şallı, F. (2012). A research of classroom teacher candidate's mathematic self efficacy with teaching mathematic efficacy [Master Thesis]. İstanbul University, Institute of Social Sciences.
62. Şen, S. (2022). A reliability generalization meta-analysis of runco ideational behavior scale. Creativity Research Journal, 34(2), 178–194. https://doi.org/10.1080/10400419.2021.1960719
63. Şen, S., & Yıldırım, İ. (2023). Meta-analysis applications with CMA (Expanded 2nd edition). Anı Publishing.
64. *Şengül, S. (2011). Effects of concept cartoons on mathematics self-efficacy of 7th grade students. Educational Sciences: Theory & Practice, 11(4), 2305–2313.
65. *Taşdemir, C. (2012). Evaluating the mathematics self- efficacay levels of high school senior students in terms of some variables (the case of Bitlis). The Black Sea Journal of Science, 3(1), 39-50. https://izlik.org/JA24XZ99SJ
66. Taşkın, T. (2021). A content analysis related to studies about concept cartoons in scienceeducation. A Content Analysis Related to Studies About Concept Cartoons in ScienceEducation, 18(1), 622-651. https://doi.org/10.33711/yyuefd.919438
67. Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60(2), 174-195. https://doi.org/10.1177/0013164400602002
68. *Tol, H.Y. (2018). The effects on the students by using the scenario based learning method of the historical developments of the mathematics subjects [Master Thesis]. Necmettin Erbakan University, Institute of Educational Sciences.
69. *Tuğral, Z., & Güvenç, H. (2016). The effects of cooperative learning on high school students math achievement, self-efficacy and engagement & disaffection. Kırşehir Ahi Evran University Faculty of Education Journal, 17(2), 259-274. https://izlik.org/JA92EW62KS
70. *Tüzün, M. & Cihangir, A. (2020). Analysis of the relation between mathematical thinking stages and mathematics self efficacy of secondary school students. Journal of Ahmet Keleşoğlu Education Faculty, 2(2), 210-228. https://doi.org/10.38151/akef.2020.5
71. Umay, A. (2001). The effect of the primary school mathematics teaching program on the mathematics self-efficacy of students. Journal of Qafqaz University, 8, 37-44.
72. Ural, A., Umay, A., & Argün, Z. (2008). The effect of students teams-achievement divisions method based instruction on mathematics academic achievement and self-efficacy. Hacettepe University Journal of Education, 35(35), 307-318.
73. *Usta, E. (2021). Examination of middle school students test anxiation, attitudes toward mathematics and self-efficiency beliefs [Master Thesis]. Sakarya University, Institute of Educational Sciences.
74. *Üçüncü, B. (2025). The relationship between high school students' social emotional learning skills levels and mathematics self-efficacy perceptions [Master Thesis]. Düzce University, Graduate Education Institute.
75. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58(1), 6-20. https://doi.org/10.1177/0013164498058001002
76. Vacha-Haase, T., & Thompson, B. (2011). Score reliability: A retrospective look back at 12 years of reliability generalization studies. Measurement and Evaluation in Counseling and Development, 44(3), 159-168. https://doi.org/10.1177/0748175611409845
77. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36, 1-48. https://doi.org/10.18637/jss.v036.i03
78. Wang, C. H., Harrison, J., Cardullo, V., & Xi, L. (2017). Exploring the relationship among international students' English self-efficacy, using English to learn self-efficacy, and academic self-efficacy. Journal of International Students, 8(1), 233-250. https://doi.org/10.32674/jis.v8i1.163
79. Wang, Y. L., Liang, J. C., Lin, C. Y., & Tsai, C. C. (2017). Identifying Taiwanese junior-high school students' mathematics learning profiles and their roles in mathematics learning self-efficacy and academic performance. Learning and Individual Differences, 54, 92-101. http://dx.doi.org/10.1016/j.lindif.2017.01.008
80. Yang, Y., Maeda, Y., & Gentry, M. (2024). The relationship between mathematics self-efficacy and mathematics achievement: multilevel analysis with NAEP 2019. Large-Scale Assessments in Education, 12(1). https://doi.org/10.1186/s40536-024-00204-z
81. *Yerlikaya, G. (2020). Investigation of mathematics teachers' 'efficacy on mathematics and self-efficacy in mathematics teaching [Master Thesis]. Sakarya University, Institute of Educational Sciences.
82. Yörük, S., & Şen, S. (2022). A reliability generalization meta-analysis of the creative achievement questionnaire. Creativity Research Journal, 35(4), 714-729. https://doi.org/10.1080/10400419.2022.2148073