Reliability Generalization Meta-Analysis of the Self-Efficacy Scale for Geometry

Abstract

This study was conducted to estimate the average reliability coefficient of the Geometry Self-Efficacy Scale (GSES) developed by Cantürk-Günhan and Başer (2007), which is frequently used in the literature, based on the reliability findings in different studies and to examine the sources of variation of the reliability coefficients reported in the studies. In the study in which reliability generalization meta-analysis design was used, 24 studies that met the criteria were included in the study. The transformed coefficient values obtained by using Bonett transformation of 26 Cronbach's alpha coefficients obtained from these studies were combined and analyzed under the random effects model. As a result of the analysis, the mean Cronbach's alpha value of the GSES was estimated as .878 (95% CI: [.85 - .90]) and this value was found to be statistically significant (p<.01). In addition, as a result of the heterogeneity analysis, it was determined that there was a high level of heterogeneity between the studies in terms of Cronbach's alpha. Moderator analyses were conducted to determine the possible reasons for the heterogeneity and the findings of the analysis revealed that Cronbach's alpha estimates were not affected by the moderator variables considered in this study. In addition, no evidence of publication bias was found in the study. As a result, the findings of this study suggest that the GSES is a scale with good reliability values on average and may help researchers in future studies to decide to use this scale to measure students' self-efficacy perceptions towards geometry.

Keywords

• Self-efficacy in geometry
• reliability generalization
• mathematics education
• meta-analysis

References

1. [Studies included in the research are marked with an asterisk (*)]
2. Afgani, M. W., Suryadi, D., & Dahlan, J. A. (2019). The enhancement of pre-service mathematics teachers' mathematical understanding ability through ACE teaching cyclic. Journal of Technology and Science Education, 9(2), 153-167. https://doi.org/10.3926/jotse.441
3. *Aksu, A. D. (2013). Predicting the geometry knowledge of pre-service elementary teachers. Cumhuriyet International Journal of Education, 2(3), 1-16.
4. *Anıkaydın, Ö. (2017). Students oriented geometry self- efficacy beliefs, attitudes and geometry investigation of relationship between levels of geometric thinking. (Thesis Number. 472218) [Master Thesis], Adnan Menderes University, Institute of Educational Sciences.
5. *Atasoy, E. (2019). Elementary mathematics teacher candidates’ geometric thinking levels and their self-efficacy in geometry. Acta Didactica Napocensia, 12(2), 161-170. https://doi.org/10.24193/adn.12.2.12
6. Badenes-Ribera, L., Duro-García, C., López-Ibáñez, C., Martí-Vilar, M., & Sánchez-Meca, J. (2023). The adult prosocialness behavior scale: A reliability generalization meta-analysis. International Journal of Behavioral Development, 47(1), 59-71. https://doi.org/10.1177/01650254221128280
7. Balduzzi, S., Rücker, G., & Schwarzer, G. (2019). How to perform a meta-analysis with R: A practical tutorial. Evidence-Based Mental Health, 22(4), 153–160. https://doi.org/10.1136/ebmental-2019-300117
8. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.

9. Bandura, A. (1997). Self-efficacy: The exercise of control. W.H. Freeman and Company.
10. Baş, F. F., & Katrancı, Y. (2020). A Validity and Reliability Study of A Self-Efficacy Scale Related to Geometry. Academic Social Studies, 4(14), 19-29. https://doi.org/10.31455/asya.732681
11. Begg, C. B., & Mazumdar, M. (1994). Operating Characteristics of a Rank Correlation Test for Publication Bias. Biometrics, 50(4), 1088–1101. https://doi.org/10.2307/2533446
12. Bonett D. G. (2002). Sample size requirements for testing and estimating coefficient alpha. Journal of Educational and Behavioral Statistics, 27(4), 335-340. https://doi.org/10.3102/10769986027004335
13. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley.
14. *Bostancı, Ü. (2019). The investigation of the relationship between eighth-grade students' geometry self-efficacy perceptions and geometric reasoning skills (Thesis Number. 538851) [Master Thesis], Kırşehir University, Institute of Educational Sciences.
15. Botella, J., Suero, M., & Gambara, H. (2010). Psychometric inferences from a meta-analysis of reliability and internal consistency coefficients. Psychological Methods, 15(4), 386–397. https://doi.org/10.1037/a0019626
16. Cantürk-Günhan, B., & Başer, N. (2007). The development of self-efficacy scale toward geometry. Hacettepe University Journal of Education, 33(33), 68-76.
17. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284
18. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart and Winston.
19. *Çadırlı, G. (2017). An analysis of the secondary school students' geometry self-efficacy beliefs and their geometric thinking skills (Thesis Number. 472256) [Master Thesis], Kahramanmaraş Sütçü İmam University, Institute of Educational Sciences.
20. Çelik, F. (2023). Gelişmeleri kaçırma korkusu ölçeği Türkçe versiyonunun güvenirlik genelleme meta analizi. Gümüşhane Üniversitesi İletişim Fakültesi Elektronik Dergisi, 11(2), 848-877. https://doi.org/10.19145/e-gifder.1283655
21. *Çontay, E. G. (2012). Geometrik cisimlerin yüzey alanları ve hacimleri konusunda yazma etkinliklerinin 8. sınıf öğrencilerinin başarılarına ve geometriye yönelik öz-yeterliklerine etkisi (Thesis Number. 330564) [Master Thesis] Pamukkale University, Institute of Educational Sciences.
22. *Çontay, E. G., & Duatepe-Paksu, A. (2018). The effect of journal writing on achievement and geometry self-efficacy of 8th grade students. Necatibey Faculty of Education Electronic Journal of Science and Mathematics Education, 12(2), 167-198. https://doi.org/10.17522/balikesirnef.506429
23. *Demir, Ö. (2019). The effect of using concrete materials in teaching of geometric objects on students' achievement, attitudes and self-efficacy (Thesis Number. 594695) [Master Thesis] Bartın University, Institute of Educational Sciences.
24. Deringöl, Y. (2020). Middle school student’s perceptions of their self-efficacy in visual mathematics and geometry: a study of sixth to eighth grade pupils in Istanbul province, Turkey. Education 3-13, 48(8), 1012-1023. https://doi.org/10.1080/03004279.2019.1709527
25. *Dursun, Ö. (2010). The relationships among preservice teachers' spatial visualization ability, geometry self-efficacy, and spatial anxiety (Thesis Number. 277931) [Master Thesis], Middle East Technical University, Institute of Educational Sciences.
26. Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455-463. https://doi.org/10.1111/j.0006-341x.2000.00455.x
27. Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629
28. Erkek, Ö., & Işıksal-Bostan, M. (2015). The role of spatial anxiety, geometry self-efficacy and gender in predicting geometry achievement. Elementary Education Online, 14(1), 164-80. https://doi.org/10.17051/io.2015.18256
29. Eser, M. T., & Doğan, N. (2023). Life satisfaction scale: A meta-analytic reliability generalization study in turkey sample. Turkish Psychological Counseling and Guidance Journal, 13(69), 224-239. https://doi.org/10.17066/tpdrd.1223320mn
30. Eser, M. T., Yurtçu, M., & Aksu G. (2020). R Programlama Dili ve Jamovi ile Meta Analiz Uygulamaları [Meta-analysis applications with R programming language and Jamovi]. Pegem Akademi.
31. Feldt, L. S., & Charter, R. A. (2006). Averaging internal consistency reliability coefficients. Educational and Psychological Measurement, 66(2), 215-227. https://doi.org/10.1177/0013164404273947
32. Field, A. P. (2003). The problems in using fixed-effects models of meta-analysis on real-world data. Understanding Statistics: Statistical Issues in Psychology, Education, and the Social Sciences, 2(2), 105-124. https://doi.org/10.1207/S15328031US0202_02
33. Field, A. P., & Gillett, R. (2010). How to do a meta‐analysis. British Journal of Mathematical and Statistical Psychology, 63(3), 665-694. https://doi.org/10.1348/000711010X502733
34. *Geçici, M. E. (2018). An investigation of eighth grade students' skills at geometry problem posing (Thesis Number. 512080) [Master Thesis], Dicle University, Institute of Educational Sciences.
35. Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557
36. Irwing, P., Booth, T., & Hughes, D. J. (Eds), (2018). The wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development. Wiley Blackwell. https://doi.org/10.1002/9781118489772
37. Işıksal, M., & Aşkar, P. (2003). The scales of perceived mathematics and computer self-efficacy for elementary students. Hacettepe University Journal of Education, 25(1), 109-118.
38. *Kaba, Y., Boğazlıyan, D., & Daymaz, B. (2016). Middle School Students’ Attitudes and Self-efficacy Towards Geometry. The Journal of Academic Social Science Studies. The Journal of Academic Social Science Studies, 52(1), 335-350. https://doi.org/10.9761/JASSS3727
39. *Kaba, Y., Özdişçi, S., & Soylu, Ş. (2017). The effect of jigsaw-i technique on 7th grade middle school students’ attitude and self efficacy toward geometry. Electronic Turkish Studies, 12(28), 473-488. https://doi.org/10.7827/TurkishStudies.12088
40. *Kandil, S. (2016). An investigation of the effect of inquiry-based instruction enriched with origami activities on the 7th grade students' reflection symmetry achievement, attitudes towards geometry and self-efficacy in geometry (Thesis Number. 439109) [Master Thesis], Middle East Technical University, Institute of Educational Sciences.
41. *Korkmaz-Serbest, D. (2023). Investigation of self-efficacy towards geometry and geometric thinking levels of secondary school students according to some variables (Thesis Number. 825790) [Master Thesis], Gazi University, Institute of Educational Sciences.
42. *Korkmaz, D., Yenilmez, K., & Korkmaz, D. (2013). Relationship between 6th, 7th and 8th grade students’ self-efficacy towards geometry and their geometric thinking levels. Necatibey Faculty of Education Electronic Journal of Science and Mathematics Education, 7(2), 268-283.
43. *Kurt, Ö. (2019). Investigation of the effects of mathematical modeling problems on the attitude of fifth grade students towards academic achievement, geometry self-effi̇cacy and mathematics (Tez No. 557400) [Master Thesis], Fırat University, Institute of Educational Sciences.
44. Lee, H., Jang, Y., & Jeong, Y. (2023). Supportive Care Needs Survey: A reliability generalization meta-analysis. Palliative and Supportive Care, 21(4), 714–726. https://doi.org/10.1017/S1478951522001791
45. Ma, X., & Kishor, N. (1997). Assessing the relationship between attitude toward mathematics and achievement in mathematics: A meta-analysis. Journal for Research in Mathematics Education, 1997(1), 26-47. https://doi.org/10.2307/749662
46. Miles, M. B., & Huberman, A. M. (1994). Data management and analysis methods. Sage Publications.
47. Mullen, B., Muellerleile, P., & Bryant, B. (2001). Cumulative meta-analysis: A consideration of ındicators of sufficiency and stability. Personality and Social Psychology Bulletin, 27(11), 1450-1462. https://doi.org/10.1177/01461672012711006
48. Mumcu, H. Y. (2019). Investigation of Primary School Mathematics Teacher Candidates’ Mathematical Reasoning Self-Efficacy Beliefs: A Scale Development and Implementation Study. Ahi Evran University Journal of Kırşehir Education Faculty, 20(3), 1239-1280.
49. Nunnally, C. J., & Bernstein, I. H. (1994). Psychological theory (3rd ed.). McGraw-Hill.
50. Onwuegbuzie, A. J., & Daniel, L. G. (2002). A framework for reporting and interpreting internal consistency reliability estimates. Measurement and evaluation in counseling and development, 35(2), 89-103. https://doi.org/10.1080/07481756.2002.12069052
51. *Orhan, N. (2013). An investigation of private middle school students? common errors in the domain of area and perimeter and the relationship between their geometry self-efficacy beliefs and basic procedural and conceptual knowledge of area and perimeter (Thesis Number. 345125) [Master Thesis], Middle East Technical University, Institute of Educational Sciences.
52. *Özdemir, F. (2023). The investigation of the effects of ACE instructional cycle on 7th grade students' knowledge construction processes in the subdomain of polygons within the framework of APOS Theory. (Thesis Number. 809664) [Master Thesis], İnönü University, Institute of Educational Sciences.
53. *Özdişci, S. (2019). Geometriye yönelik tutum ölçeği geliştirilmesi ve değerlendirilmesi (Thesis Number. 544500) [Master Thesis], Kocaeli University, Institute of Educational Sciences.
54. Özkan, E., & Yıldırım, S. (2013). The Relationships between Geometry Achievement, Geometry Self-efficacy, Parents’ Education Level and Gender. Ankara University Journal of Faculty of Educational Science, 46(2), 249-261. https://doi.org/10.1501/Egifak_0000001304
55. *Özkeleş-Çağlayan, S. (2010). The potential of predictive of the academic achievement points of geometry using the 9th grade students geometry self-efficacy and geometry attitude (Thesis Number. 263663) [Master Thesis], Yıldız Teknik University, Institute of Educational Sciences.
56. R Core Team. (2020). R: A language and environment for statistical computing [Computer software]. https://www.R-project.org/
57. Reig-Aleixandre, N., Esparza-Reig, J., Martí-Vilar, M., Merino-Soto, C., & Livia, J. (2023). Measurement of prosocial tendencies: meta-analysis of the generalization of the reliability of the instrument. Healthcare, 11(4), 1-23. https://doi.org/10.3390/healthcare11040560
58. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
59. Sánchez-Meca, J., López-López, J. A., & López-Pina, J. A. (2013). Some recommended statistical analytic practices when reliability generalization studies are conducted. British Journal of Mathematical and Statistical Psychology, 66(3), 402–425. https://doi.org/10.1111/j.2044-8317.2012.02057.x
60. Sánchez-Meca, J., Marín-Martínez, F., López-López, J. A., Núñez-Núñez, R. M., Rubio-Aparicio, M., López-García, J. J., López-Pina, J. A., Blázquez-Rincón, D. M., López-Ibáñez, C., & López-Nicolás, R. (2021). Improving the reporting quality of reliability generalization meta-analyses: The REGEMA checklist. Research Synthesis Methods, 12(4), 516–536. https://doi.org/10.1002/jrsm.1487
61. Sandoval-Lentisco, A., López-Nicolás, R., López-López, J. A., & Sánchez-Meca, J. (2023). Florida Obsessive-Compulsive Inventory and Children's Florida Obsessive Compulsive Inventory: A reliability generalization meta-analysis. Journal of Clinical Psychology, 79(1), 28–42. https://doi.org/10.1002/jclp.23416
62. Sawilowsky, S. S. (2000). Psychometrics versus data metrics: Comment on Vacha-Haase’s “reliability generalization” method and some EPM editorial policies. Educational and Psychological Measurement, 60(2), 157-173. https://doi.org/10.1177/00131640021970439
63. Schmid, C.H., Stijnen, T., & White, I. R. (2021). General themes in meta-analysis. In C.H. Schmid, T. Stijnen, & I. R. White (Eds.), Handbook of meta-analysis (19-26). CRC Press.
64. Schunk, D. H. (2011). Learning theories: An educational perspective (6th ed.). Pearson Education.
65. Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0
66. Streiner, D. L., & Norman, G. R. (2008). Health measurement scales: A practical guide to their development and use (4th ed.). Oxford University Press. https://doi.org/10.1093/acprof:oSo/9780199231881.001.0001
67. *Şahin, B. (2015). The effect of activity based geometry instruction on students' learning outcomes (Thesis Number. 395258) [Master Thesis], Dokuz Eylül University, Institute of Educational Sciences.
68. Şen, S. (2022). A reliability generalization meta-analysis of runco ideational behavior scale. Creativity Research Journal, 34(2), 178–194. https://doi.org/10.1080/10400419.2021.1960719
69. Şen, S., & Yıldırım, İ. (2023). Meta-analysis applications with CMA (Expanded 2nd edition). Anı Publishing.
70. *Taş, S., & Yavuz, A. (2023). Evaluation of geohepta mobile application-based instructions’ impacts on affective behaviors. Marmara University Atatürk Faculty of Education Journal of Educational Sciences, 57(57), 168-186. https://doi.org/10.15285/maruaebd.1158744
71. Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is data metrics: The test is not reliable. Educational and Psychological Measurement, 60(2), 174-195. https://doi.org/10.1177/0013164400602002
72. Usher, E. L. (2009). Sources of middle school students’ self-efficacy in mathematics: A qualitative investigation. American Educational Research Journal, 46(1), 275-314. https://doi.org/10.3102/0002831208324517
73. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58(1), 6-20. https://doi.org/10.1177/0013164498058001002
74. Vacha-Haase, T., Kogan, L. R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60(4), 509–522. https://doi.org/10.1177/00131640021970682
75. Vacha-Haase, T., & Thompson, B. (2011). Score reliability: A retrospective look back at 12 years of reliability generalization studies. Measurement and Evaluation in Counseling and Development, 44(3), 159-168. https://doi.org/10.1177/0748175611409845
76. *Yayla, Y. (2021). Sharing of real life geometry samples via a social learning environment: A case study (Thesis Number. 664140) [Master Thesis], Gazi University, Institute of Educational Sciences.
77. Yörük, S., & Şen, S. (2022). A reliability generalization meta-analysis of the creative achievement questionnaire. Creativity Research Journal, 35(4), 714-729. https://doi.org/10.1080/10400419.2022.2148073