A Reliability Generalization Meta-Analysis of the Survey of Attitudes Toward Statistics (SATS)

Abstract

Students’ attitudes toward statistics play a crucial role in shaping their learning outcomes and long-term statistical thinking. The Survey of Attitudes Toward Statistics (SATS) has emerged as the most widely adopted tool in statistics education. This study aims to conduct the first reliability generalization meta-analysis of the SATS by synthesizing internal consistency estimates reported. Following the REGEMA (Reporting Guidelines for Meta-Analyses of Reliability Generalization Studies) guidelines, a systematic literature search was conducted across seven databases. A total of 41 eligible studies were identified, yielding 56 Cronbach’s alpha coefficients. A random-effects model was used to obtain pooled reliability estimates, and publication bias was assessed via funnel plots and Egger’s regression test. A mixed-effects meta-regression was conducted using several moderators, including geographic location and educational level. The SATS components demonstrated acceptable to excellent internal consistency. The findings confirm the SATS’ robust internal consistency and support their continued use in educational research.

Keywords

• Reliability generalization
• SATS
• Attitudes toward statistics
• Cronbach’s alpha
• Meta-analysis

References

1. Wigfield, A. and Eccles, J. S., "Expectancy-value theory of achievement motivation", Contemporary Educational Psychology, 25(1), 68–81, 2000.
2. Schau, C., Stevens, J., Dauphinee, T. L. and Vecchio, A. Del., "The Development and Validation of the Survey of Attitudes toward Statistics", Educational and Psychological Measurement, 55(5), 868–875, 1995.
3. Schau, C., "Students’ attitudes: The “other” important outcome in statistics education", Proceedings of the joint statistical meetings, 2003, 3673-3681.
4. Nasser, F. M., "Structural Model of the Effects of Cognitive and Affective Factors on the Achievement of Arabic-Speaking Pre-service Teachers in Introductory Statistics", Journal of Statistics Education, 12(1), 2004.
5. Chiesi, F. and Primi, C., "Assessing statistics attitudes among college students: Psychometric properties of the Italian version of the Survey of Attitudes toward Statistics (SATS)", Learning and Individual Differences, 19(2), 309–313, 2009.
6. Sarikaya, E. E., Ok, A., Aydin, Y. C. and Schau, C., "Turkish Version of the Survey of Attitudes toward Statistics: Factorial Structure Invariance by Gender", International Journal of Higher Education, 7(2), 121–127, 2018.
7. Roberts, D. M. and Bilderback, E. W., "Reliability and Validity of a Statistics Attitude Survey", Educational and Psychological Measurement, 40(1), 235–238, 1980.
8. Wise, S. L., "The Development and Validation of a Scale Measuring Attitudes toward Statistics", Educational and Psychological Measurement, 45(2), 401–405, 1985.

9. Adamson, K. A. and Prion, S., "Reliability: Measuring Internal Consistency Using Cronbach’s α", Clinical Simulation in Nursing, 9(5), e179–e180, 2013.
10. Tavakol, M. and Dennick, R., "Making sense of Cronbach’s alpha", International Journal of Medical Education, 2, 53, 2011.
11. Zinbarg, R. E., Revelle, W., Yovel, I. and Li, W., "Cronbach’s α, Revelle’s β, and Mcdonald’s ωH: their Relations with Each Other and Two Alternative Conceptualizations of Reliability", Psychometrika, 70(1), 123–133, 2005.
12. Trizano-Hermosilla, I. and Alvarado, J. M., "Best alternatives to Cronbach’s alpha reliability in realistic conditions: Congeneric and asymmetrical measurements", Frontiers in Psychology, 7, 168007, 2016.
13. Vacha-Haase, T., "Reliability Generalization: Exploring Variance in Measurement Error Affecting Score Reliability Across Studies", Educational and Psychological Measurement, 58(1), 6–20, 1998.
14. Graham, J. M., Diebels, K. J. and Barnow, Z. B., "The Reliability of Relationship Satisfaction: A Reliability Generalization Meta-Analysis", Journal of Family Psychology, 25(1), 39–48, 2011.
15. Shields, A. L. and Caruso, J. C., "A Reliability Induction and Reliability Generalization Study of the Cage Questionnaire", Educational and Psychological Measurement, 64(2), 254–270, 2004.
16. Vassar, M. and Hale, W., "Reliability reporting practices in youth life satisfaction research", Social Indicators Research, 83(3), 487–496, 2007.
17. Sánchez-Meca, J., Marín-Martínez, F., López-López, J. A., Núñez-Núñez, R. M., Rubio-Aparicio, M., López-García, J. J., López-Pina, J. A., Blázquez-Rincón, D. M., López-Ibáñez, C. and López-Nicolás, R., "Improving the reporting quality of reliability generalization meta-analyses: The REGEMA checklist", Research Synthesis Methods, 12(4), 516–536, 2021.
18. Ouzzani, M., Hammady, H., Fedorowicz, Z. and Elmagarmid, A., "Rayyan-a web and mobile app for systematic reviews", Systematic Reviews, 5(1), 1–10, 2016.
19. Cerri, L. Q., Justo, M. C., Clemente, V., Gomes, A. A., Pereira, A. S. and Marques, D. R., "Insomnia Severity Index: A reliability generalisation meta-analysis", Journal of Sleep Research, 32(4), e13835, 2023.
20. Viechtbauer, W., "Conducting Meta-Analyses in R with the metafor Package", Journal of Statistical Software, 36(3), 1–48, 2010.
21. Nolan, M. M., Beran, T. and Hecker, K. G., "Surveys assessing students' attitudes toward statistics: A systematic review of validity and reliability", Statistics Education Research Journal, 11(2), 103–123, 2012.
22. Ratanaolarn, T., "The Development of a Structural Equation Model of Graduate Students’ Statistics Achievement", International Journal of Behavioral Science, 11(2), 153–168, 2016.
23. Rosli, M. K. and Maat, S. M., "Attitude towards Statistics and Performance among Post-Graduate Students", AIP Conference Proceedings, 2017, 1–6.
24. Wahba, F., Ajlouni, A. O. and Abumosa, M. A., "The impact of ChatGPT-based learning statistics on undergraduates’ statistical reasoning and attitudes toward statistics", Eurasia Journal of Mathematics, Science and Technology Education, 20(7), 2024.
25. Ayebo, A., Bright, J. and Ballam, C., "Examining the Factor Structure of the Survey of Attitudes Towards Statistics among Undergraduate Health Science Students", International Electronic Journal of Mathematics Education, 15(1), 2020.
26. Zhang, Y., Shang, L., Wang, R., Zhao, Q., Li, C., Xu, Y. and Su, H., "Attitudes toward statistics in medical postgraduates: measuring, evaluating and monitoring", BMC Medical Education, 12, 117, 2012.
27. Gómez, E. E., "Actitudes de alumnos de posgrado hacia la estadística aplicada a la investigación", Encuentro, 85(3), 27–38, 2010.
28. Guàrdia-Olmos, J., Peró-Cebollero, M., Mancho-Fora, N. and Farràs-Permanyer, L., "Adaptation of a battery for the assessment of prerequisits for the Statistics learning in Psicology", Revista d’Innovació i Recerca en Educació, 5(2), 42–58, 2012.
29. Chiesi, F. and Primi, C., "Cognitive and non-cognitive factors related to students’ statistics achievement", Statistics Education Research Journal, 9(1), 6–26, 2010.
30. Lipka, O. and Hess, I., "Attitudes Toward Statistics Studies Among Students with Learning Disabilities", Numeracy: Advancing Education in Quantitative Literacy, 9(2), 1–17, 2016.
31. Hood, M., Creed, P. A. and Neumann, D. L., "Using the expectancy value model of motivation to understand the relationship between student attitudes and achievement in statistics", Statistics Education Research Journal, 11(2), 72–85, 2012.
32. Abbiati, N. N., Carmen Fabrizio, M., Virginia López, M., Pérez, A., Cristina Plencovich, M. and Cueto, G., "Attitudes related to students’ performance in statistics in university programs in argentina", Statistics Education Research Journal, 20(2), 1–11, 2021.
33. Zimprich, D., "Attitudes toward statistics among Swiss psychology students", Swiss Journal of Psychology, 71(3), 149–155, 2012.
34. Mills, J. D., "Students’ attitudes toward statistics: implications for the future", College Student Journal, 38(3), 349–361, 2004.
35. Sin, Y. C. and Rosli, R., "Validity and reliability of the survey of attitudes toward statistics (sats) instrument", Jurnal Pendidikan Malaysia, 45, 17–24, 2020.
36. Lavidas, K., Papadakis, S., Manesis, D., Grigoriadou, A. S. and Gialamas, V., "The Effects of Social Desirability on Students’ Self-Reports in Two Social Contexts: Lectures vs. Lectures and Lab Classes", Information, 13(10), 491, 2022.
37. Lavidas, K., Barkatsas, T., Manesis, D. and Gialamas, V., "A Structural Equation Model Investigating the Impact of Tertiary Students’ Attitudes toward Statistics, Perceived Competence at Mathematics, and Engagement on Statistics Performance", Statistics Education Research Journal, 19(2), 27–41, 2020.
38. Araoz-Melgarejo, V. A., Espinoza, B. M., Quiñones-Laveriano, D. M. and De La Cruz-Vargas, J. A., "Basic knowledge and attitudes towards biostatistics in sixth- and seventh-year medical students", Turkish Journal of Computer and Mathematics Education, 12(14), 1286–1302, 2021.
39. Stanisavljevic, D., Trajkovic, G., Marinkovic, J., Bukumiric, Z., Cirkovic, A. and Milic, N., "Assessing Attitudes towards Statistics among Medical Students: Psychometric Properties of the Serbian Version of the Survey of Attitudes Towards Statistics (SATS )", PLoS One, 9(11), 2014.
40. Hannigan, A., Hegarty, A. C. and McGrath, D., "Attitudes towards statistics of graduate entry medical students: the role of prior learning experiences", BMC Medical Education, 14, 70, 2014.
41. Walker, E. R. and Brakke, K. E., "Undergraduate psychology students’ efficacy and attitudes across introductory and advanced statistics courses", Scholarship of Teaching and Learning in Psychology, 3(2), 132–140, 2017.
42. Tempelaar, D. T., Van Der Loeff, S. S. and Gijselaers, W. H., "A structural equation model analyzing the relationship of students’ attitudes toward statistics, prior reasoning abilities, and course performance", Statistics Education Research Journal, 6(2), 78–102, 2007.
43. Akour, M. M., "Rasch rating scale analysis of the survey of attitudes toward statistics", Eurasia Journal of Mathematics, Science and Technology Education, 18(12), 2022.
44. Jatnika, R. and Abidin, F. A., "Model Factors that Affect Mastery of Statistics on Psychology Students at University of Padjadjaran", Journal of Physics: Conference Series, 1179(1), 2019.
45. Khoshnoodifar, M., Ashouri, A. and Taheri, M., "Effectiveness of gamification in enhancing learning and attitudes: a study of statistics education for health school students", Journal of Advances in Medical Education & Professionalism, 11(4), 230–239, 2023.
46. Counsell, A., Rovetti, J. and Buchanan, E. M., "Psychometric evaluation of the students’ attitudes toward statistics and technology scale (sastsc)", Statistics Education Research Journal, 21(3), 1–18, 2022.
47. Counsell, A. and Cribbie, R. A., "Students’ attitudes toward learning statistics with R", Psychology Teaching Review, 26(2), 36–56, 2020.
48. van Appel, V. and Durandt, R., "Dissimilarities in Attitudes between Students in Service and Mainstream Courses towards Statistics: An Analysis Conducted in a Developing Country", Eurasia Journal of Mathematics, Science and Technology Education, 14(8), 2018.
49. Persson, I., Kraus, K., Hansson, L. and Wallentin, F. A., "Confirming the structure of the survey of attitudes toward statistics (sats-36) by swedish students", Statistics Education Research Journal, 18(1), 83–93, 2019.
50. Coetzee, S. and Der Merwe, P., "Industrial psychology students’ attitudes towards statistics", SAJIP: South African Journal of Industrial Psychology, 36(1), 1–8, 2010.
51. Schau, C. and Emmioğlu, E., "Do introductory statistics courses in the united states improve students’ attitudes?", Statistics Education Research Journal, 11(2), 86–94, 2012.
52. Bond, M. E., Perkins, S. N. and Ramirez, C., "Students’ perceptions of statistics: An exploration of attitudes, conceptulizations, and content knowledge of statistics", Statistics Education Research Journal, 11(2), 6–25, 2012.
53. Colaco, C., Darking, S., Mar, R. A. and Martin, J., "Using online tutorials to introduce psychology undergrads to R: Student experiences, knowledge, and attitudes", Scholarship of Teaching and Learning in Psychology, 2023.
54. Wang, P., Palocsay, S. W., Shi, J. and White, M. M., "Examining Undergraduate Students’ Attitudes toward Business Statistics in the United States and China", Decision Sciences Journal of Innovative Education, 16(3), 197–216, 2018.
55. Kyaw Soe, H., Khobragade, S., Lwin, H., Nu Htay, M., Than, N., Phyu, K. and Abas, A., "Learning statistics: Interprofessional survey of attitudes toward statistics using SATS-36", Dentistry & Medical Research, 9(2), 121–125, 2021.
56. Opstad, L., "Attitudes towards Statistics among Business Students: Do Gender, Mathematical Skills and Personal Traits Matter?", Sustainability, 12(15), 6104, 2020.
57. Carillo, K. D. A., Galy, N., Guthrie, C. and Vanhems, A., "How to turn managers into data-driven decision makers: Measuring attitudes towards business analytics", Business Process Management Journal, 25(3), 553–578, 2019.
58. Curran, E. M., Carlson, K. and Celotta, D. L. T., "Changing attitudes and facilitating understanding in the undergraduate statistics classroom: A collaborative learning approach", Journal of the Scholarship of Teaching and Learning, 13(2), 49–71, 2013.
59. Xu, C. and Schau, C., "Measuring Statistics Attitudes at the Student and Instructor Levels: A Multilevel Construct Validity Study of the Survey of Attitudes Toward Statistics", Journal of Psychoeducational Assessment, 39(3), 315–331, 2021.
60. Nichols, R. L. and Wilder, L. K., "The Effectiveness of Inquiry-Based vs. Didactic Teaching Methods on Student Performance in Undergraduate Statistics", ProQuest Dissertations and Theses, 10286086, 141, 2017.
61. Rodríguez-Santero, J. and Gil-Flores, J., "Attitudes towards statistics of education Sciences students. Psychometric properties of the Spanish version of the Survey of Attitudes Towards Statistics (SATS-36)", Revista Electrónica de Investigación y Evaluación Educativa, 25(1), 2019.
62. Slootmaeckers, K., Kerremans, B. and Adriaensen, J., "Too Afraid to Learn: Attitudes towards Statistics as a Barrier to Learning Statistics and to Acquiring Quantitative Skills", Politics, 34(2), 191–200, 2014.