Investigation of Measurement Invariance According to Home Resources: TIMSS 2015 Mathematical Affective Characteristics Questionnaire

Abstract

This study aimed to examine the measurement invariance of the mathematical affective characteristics model obtained from TIMSS 2015 4th grade Turkey administration according to home resources. For this purpose, firstly, the factor structure of the mathematical affective characteristics questionnaire was examined by explanatory factor analysis and Velicer’s maximum average partial (MAP) test. It was revealed that the questionnaire had three factors. Then the structure was validated by confirmatory factor analysis. In the next stage, multi-group confirmatory factor analysis was employed with a purpose to examine whether the model displayed measurement invariance across the variables of home resources such as internet connection, heating system, cooling system, and dishwasher. The results showed that the strict measurement invariance of the mathematical affective characteristics model was achieved among the subgroups of each of the internet connection, heating system, cooling system, and dishwasher variables. Accordingly, means, variance, covariances, and item residual variances in the subgroups were found to be similar. According to the results of the study, the comparison of the mathematical affective characteristics model based on the home resources is found to be significant and comparisons made show that possible differences arise from the relevant home resource.

Keywords

• Measurement invariance
• Socio-economic status
• Home resources
• Affective characteristics

References

1. Acar Güvendir, M. (2017). Determination of the relationship between the students mathematical literacy and home and school educational resources in program for international student assessment - (PISA 2012). Mersin University Journal of the Faculty of Education, 13(1). https://doi.org/10.17860/mersinefd.305762
2. Alivernini, F. (2011). Measurement invariance of a reading literacy scale in the Italian context: A psychometric analysis. Procedia Social and Behavioral Sciences, 15, 436-441. https://doi.org/10.1016/j.sbspro.2011.03.117
3. Azina, I. N., & Halimah, A. (2012). Student factors and mathematics achievement: Evidence from TIMSS 2007. Eurasia Journal of Mathematics, Science and Technology Education, 8(4), 249-255. https://doi.org/10.12973/eurasia.2012.843a
4. Başusta, N. B., & Gelbal, S. (2015). Examination of measurement invariance at groups’ comparisons: A study on PISA student questionnaire, Hacettepe University Journal of Education, 30(4), 80-90.
5. Bouhlila, D. S. (2014). The impact of socioeconomic status on students’ achievement in the Middle East and North Africa: An essay using the TIMSS 2007 database. International Perspectives on Education and Society, 24, 199-226 https://doi.org/10.1108/S1479-367920140000024017
6. Bofah, E. A. T., & Hannula, M. S. (2017). Home resources as a measure of socio-economic status in Ghana. Large-scale Assessments in Education, 5(1), 1 15. https://doi.org/10.1186/s40536-017-0039-5
7. Brannick, M. T. (1995). Critical comments on applying covariance structure modeling. Journal of Organizational Behavior, 16(3), 201-213. https://doi.org/10.1002/job.4030160303
8. Büyüköztürk, Ş. (2009) Sosyal Bilimler İçin Veri Analizi El Kitabı, Pegem Akademi

9. Caponera, E., & Losito, B. (2016). Context factors and student achievement in the IEA studies: Evidence from TIMSS. Large scale Assessments in Education, 4(1),12. https://doi.org/10.1186/s40536-016-0030-6
10. Cheung, G. W., & Lau, R. S. (2012). A direct comparison approach for testing measurement invariance. Organizational Research Methods, 15(2), 167 198. https://doi.org/10.1177/1094428111421987
11. Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness of fit indices for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5
12. Ercikan, K., & Koh, K. (2005). Examining the construct comparability of the English and French version of TIMSS. International Journal of Testing, 5, 23 35. https://doi.org/10.1207/s15327574ijt0501_3
13. Ertürk, Z., & Erdinç-Akan, O. (2018). The investigation of the variables effecting TIMSS 2015 mathematics achievement with SEM, Journal of Theoretical Educational Science, 2, 14-34.
14. Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44, 78 94. https://doi.org/10.1097/01.mlr.0000245454.12228.8f
15. Gustafsson, J. E., Nilsen, T., & Hansen, K. Y. (2018). School characteristics moderating the relation between student socio-economic status and mathematics achievement in grade 8. Evidence from 50 countries in TIMSS 2011. Studies in Educational Evaluation, 57, 16-30. https://doi.org/10.1016/j.stueduc.2016.09.004
16. Hansson, Å., & Gustafsson, J.-E. (2013). Measurement invariance of socioeconomic status across migrational background. Scandinavian Journal of Educational Research, 57(2), 148–166. https://doi.org/10.1080/00313831.2011.625570
17. Hirschfeld, G., & Von Brachel, R. (2014). Improving multiple-group confirmatory factor analysis in R–A tutorial in measurement invariance with continuous and ordinal indicators. Practical Assessment, Research, and Evaluation, 19(1), 7. https://doi.org/10.7275/qazy-2946
18. Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117-144. https://doi.org/10.1080/03610739208253916
19. Hu, L., & Bentler, M., P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
20. International Association for the Evaluation of Educational Achievement, [IEA], (2019). Chapter 3 TIMSS 2019 context questionnaire framework, https://timss2019.org/wp-content/uploads/frameworks/T19-Assessment-Frameworks-Chapter-3.pdf
21. Jöreskog, K. G., & Sörbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. IL: Scientific Software International, Inc.
22. Kelloway, E. K. (1995). Structural equation modelling in perspective. Journal of Organizational Behavior, 16(3), 215-224. https://doi.org/10.1002/job.4030160304
23. Kıbrıslıoğlu, N. (2015). The investigation of measurement invariance PISA 2012 mathematics learning model according to culture and gender: Turkey - China (Shangai) - Indonesia, [Graduate Thesis, Hacettepe University].
24. Kline, R. B., (2011). Principles and Practices of Structural Equation Modelling. The Guilford Press.
25. Marsh, H. W., Hau, K. T., Artelt, C., Boument, J., & Peschar, J. (2006). OECD’s brief selfreport measure of educational psychology’s most useful affective constructs: cross-cultural, psychometric comparisons across 25 countries. International Journal of Testing, 6 (4), 311-360. https://doi.org/10.1207/s15327574ijt0604_1
26. McGaw, B., & Jöreskog, K. G. (1971). Factorial invariance of ability measures in groups differing in intelligence and socioeconomic status. British Journal of Mathematical and Statistical Psychology, 24(2), 154 168. https://doi.org/10.1111/j.2044-8317.1971.tb00463.x
27. Ministry of National Education (2016). TIMSS 2015 ulusal matematik ve fen ön raporu 4. ve 8. Sınıflar [TIMSS 2015 national mathematics and sciences preliminary report 4th and 8th grades]. https://odsgm.meb.gov.tr/meb_iys_dosyalar/2017_06/23161945_timss_2015_on_raporu.pdf
28. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543. https://doi.org/10.1007/BF02294825
29. Millsap, R. E., & Olivera-Aguilar, M. (2012). Investigating measurement invariance using confirmatory factor analysis. In R. H. Hoyle, (Ed.) Handbook of structural equation modeling, (pp. 380-392), Guilford.
30. Millsap, R. E. (2011). Statistical approaches to measurement invariance, Routledge.
31. O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, &Computers, 32, 396-402. https://doi.org/10.3758/BF03200807
32. Ölçüoğlu, R., & Çetin, S. (2016). The investigation of the variables that affecting eight grade students’ TIMSS 2011 math achievement according to regions, Journal of Measurement and Evaluation in Education and Psychology 7(1), 202 220. https://doi.org/10.21031/epod.34424
33. Polat, M. (2019). The investigation of measurement invariance of TIMSS-2015 mathematics and science affective characteristics models according to culture, gender and statistical region, [Graduate Thesis, Hacettepe University].
34. Schmith, N., & Kuljanin, G. (2008). Measurement invariance: review of practice and implication. Human Resources Management Review, 18, 210 222. https://doi.org/10.1016/j.hrmr.2008.03.003
35. Segeritz, M., & Pant, H. A. (2013). Do they feel the same way about math? Testing measurement invariance of the PISA students’ approaches to learning instrument across immigrant groups within Germany. Educational and Psychological Measurement, 73(4), 601-630 https://doi.org/10.1177/0013164413481802
36. Shen, C. (2005). How American middle schools differ from schools of five Asian countries: Based on cross-national data from TIMSS 1999. Educational Research and Evaluation, 11(2), 179-199. https://doi.org/10.1080/13803610500110810
37. Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417 453. https://doi.org/10.3102/00346543075003417
38. Tabachnick, B. G., & Fidell, L.S. (2007). Using Multivariate Statistics, (5. ed.) Pearson Education.
39. Tavşancıl, E. (2005) Tutumların Ölçülmesi ve SPSS ile Veri Analizi [Measuring Attitudes and Data Analysis with SPSS], Nobel Yayınları
40. Teo, T. (2010). Gender differences in the intention to use technology: A measurement invariance analysis. British Journal of Educational Technology, 41(6), 120-124. https://doi.org/10.1111/j.1467-8535.2009.01023.x
41. Uyar. Ş., & Doğan, N. (2014). An investigation of measurement invariance of learning strategies model across different groups in PISA Turkey sample, International Journal of Turkish Educational Studies, 2, 30-43.
42. Vanderberg, R. J., & Lance, C. E., (2000). A review and synthesis of the measurement invariance literature: Suggestions practices, and recommendations for organizational research. Organizational Research Methods, 3(4), 4 70. https://doi.org/10.1177/109442810031002
43. Walzebug, A. (2014). Is there a language-based social disadvantage in solving mathematical items?. Learning, Culture and Social Interaction, 3(2), 159 169. https://doi.org/10.1016/j.lcsi.2014.03.002
44. White, K. R. (1982). The relation between socioeconomic status and academic achievement, Psychological Bulletin, 91(3), 461–481. https://doi.org/10.1037/0033-2909.91.3.461
45. Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research & Evaluation, 12(3). https://doi.org/10.7275/mhqa-cd89
46. Yıldırım, S. (2019). Predicting mathematics achievement: The role of socioeconomic status, parental involvement, and self-confidence, Education and Science, 44(198), 99-113. https://doi.org/10.15390/EB.2019.7868