Investigation of Measurement Invariance According to Home Resources: TIMSS 2015 Mathematical Affective Characteristics Questionnaire

Year 2021, Volume: 8 Issue: 3, 633 - 648, 05.09.2021

Derya Çakıcı Eser

https://doi.org/10.21449/ijate.817168

Cited By: 2

Abstract

This study aimed to examine the measurement invariance of the mathematical affective characteristics model obtained from TIMSS 2015 4th grade Turkey administration according to home resources. For this purpose, firstly, the factor structure of the mathematical affective characteristics questionnaire was examined by explanatory factor analysis and Velicer’s maximum average partial (MAP) test. It was revealed that the questionnaire had three factors. Then the structure was validated by confirmatory factor analysis. In the next stage, multi-group confirmatory factor analysis was employed with a purpose to examine whether the model displayed measurement invariance across the variables of home resources such as internet connection, heating system, cooling system, and dishwasher. The results showed that the strict measurement invariance of the mathematical affective characteristics model was achieved among the subgroups of each of the internet connection, heating system, cooling system, and dishwasher variables. Accordingly, means, variance, covariances, and item residual variances in the subgroups were found to be similar. According to the results of the study, the comparison of the mathematical affective characteristics model based on the home resources is found to be significant and comparisons made show that possible differences arise from the relevant home resource.

Keywords

Measurement invariance, Socio-economic status, Home resources, Affective characteristics

References

• Acar Güvendir, M. (2017). Determination of the relationship between the students mathematical literacy and home and school educational resources in program for international student assessment - (PISA 2012). Mersin University Journal of the Faculty of Education, 13(1). https://doi.org/10.17860/mersinefd.305762
• Alivernini, F. (2011). Measurement invariance of a reading literacy scale in the Italian context: A psychometric analysis. Procedia Social and Behavioral Sciences, 15, 436-441. https://doi.org/10.1016/j.sbspro.2011.03.117
• Azina, I. N., & Halimah, A. (2012). Student factors and mathematics achievement: Evidence from TIMSS 2007. Eurasia Journal of Mathematics, Science and Technology Education, 8(4), 249-255. https://doi.org/10.12973/eurasia.2012.843a
• Başusta, N. B., & Gelbal, S. (2015). Examination of measurement invariance at groups’ comparisons: A study on PISA student questionnaire, Hacettepe University Journal of Education, 30(4), 80-90.
• Bouhlila, D. S. (2014). The impact of socioeconomic status on students’ achievement in the Middle East and North Africa: An essay using the TIMSS 2007 database. International Perspectives on Education and Society, 24, 199-226 https://doi.org/10.1108/S1479-367920140000024017
• Bofah, E. A. T., & Hannula, M. S. (2017). Home resources as a measure of socio-economic status in Ghana. Large-scale Assessments in Education, 5(1), 1 15. https://doi.org/10.1186/s40536-017-0039-5
• Brannick, M. T. (1995). Critical comments on applying covariance structure modeling. Journal of Organizational Behavior, 16(3), 201-213. https://doi.org/10.1002/job.4030160303
• Büyüköztürk, Ş. (2009) Sosyal Bilimler İçin Veri Analizi El Kitabı, Pegem Akademi
• Caponera, E., & Losito, B. (2016). Context factors and student achievement in the IEA studies: Evidence from TIMSS. Large scale Assessments in Education, 4(1),12. https://doi.org/10.1186/s40536-016-0030-6
• Cheung, G. W., & Lau, R. S. (2012). A direct comparison approach for testing measurement invariance. Organizational Research Methods, 15(2), 167 198. https://doi.org/10.1177/1094428111421987
• Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness of fit indices for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5
• Ercikan, K., & Koh, K. (2005). Examining the construct comparability of the English and French version of TIMSS. International Journal of Testing, 5, 23 35. https://doi.org/10.1207/s15327574ijt0501_3
• Ertürk, Z., & Erdinç-Akan, O. (2018). The investigation of the variables effecting TIMSS 2015 mathematics achievement with SEM, Journal of Theoretical Educational Science, 2, 14-34.
• Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44, 78 94. https://doi.org/10.1097/01.mlr.0000245454.12228.8f
• Gustafsson, J. E., Nilsen, T., & Hansen, K. Y. (2018). School characteristics moderating the relation between student socio-economic status and mathematics achievement in grade 8. Evidence from 50 countries in TIMSS 2011. Studies in Educational Evaluation, 57, 16-30. https://doi.org/10.1016/j.stueduc.2016.09.004
• Hansson, Å., & Gustafsson, J.-E. (2013). Measurement invariance of socioeconomic status across migrational background. Scandinavian Journal of Educational Research, 57(2), 148–166. https://doi.org/10.1080/00313831.2011.625570
• Hirschfeld, G., & Von Brachel, R. (2014). Improving multiple-group confirmatory factor analysis in R–A tutorial in measurement invariance with continuous and ordinal indicators. Practical Assessment, Research, and Evaluation, 19(1), 7. https://doi.org/10.7275/qazy-2946
• Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117-144. https://doi.org/10.1080/03610739208253916
• Hu, L., & Bentler, M., P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
• International Association for the Evaluation of Educational Achievement, [IEA], (2019). Chapter 3 TIMSS 2019 context questionnaire framework, https://timss2019.org/wp-content/uploads/frameworks/T19-Assessment-Frameworks-Chapter-3.pdf
• Jöreskog, K. G., & Sörbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. IL: Scientific Software International, Inc.
• Kelloway, E. K. (1995). Structural equation modelling in perspective. Journal of Organizational Behavior, 16(3), 215-224. https://doi.org/10.1002/job.4030160304
• Kıbrıslıoğlu, N. (2015). The investigation of measurement invariance PISA 2012 mathematics learning model according to culture and gender: Turkey - China (Shangai) - Indonesia, [Graduate Thesis, Hacettepe University].
• Kline, R. B., (2011). Principles and Practices of Structural Equation Modelling. The Guilford Press.
• Marsh, H. W., Hau, K. T., Artelt, C., Boument, J., & Peschar, J. (2006). OECD’s brief selfreport measure of educational psychology’s most useful affective constructs: cross-cultural, psychometric comparisons across 25 countries. International Journal of Testing, 6 (4), 311-360. https://doi.org/10.1207/s15327574ijt0604_1
• McGaw, B., & Jöreskog, K. G. (1971). Factorial invariance of ability measures in groups differing in intelligence and socioeconomic status. British Journal of Mathematical and Statistical Psychology, 24(2), 154 168. https://doi.org/10.1111/j.2044-8317.1971.tb00463.x
• Ministry of National Education (2016). TIMSS 2015 ulusal matematik ve fen ön raporu 4. ve 8. Sınıflar [TIMSS 2015 national mathematics and sciences preliminary report 4th and 8th grades]. https://odsgm.meb.gov.tr/meb_iys_dosyalar/2017_06/23161945_timss_2015_on_raporu.pdf
• Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543. https://doi.org/10.1007/BF02294825
• Millsap, R. E., & Olivera-Aguilar, M. (2012). Investigating measurement invariance using confirmatory factor analysis. In R. H. Hoyle, (Ed.) Handbook of structural equation modeling, (pp. 380-392), Guilford.
• Millsap, R. E. (2011). Statistical approaches to measurement invariance, Routledge.
• O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, &Computers, 32, 396-402. https://doi.org/10.3758/BF03200807
• Ölçüoğlu, R., & Çetin, S. (2016). The investigation of the variables that affecting eight grade students’ TIMSS 2011 math achievement according to regions, Journal of Measurement and Evaluation in Education and Psychology 7(1), 202 220. https://doi.org/10.21031/epod.34424
• Polat, M. (2019). The investigation of measurement invariance of TIMSS-2015 mathematics and science affective characteristics models according to culture, gender and statistical region, [Graduate Thesis, Hacettepe University].
• Schmith, N., & Kuljanin, G. (2008). Measurement invariance: review of practice and implication. Human Resources Management Review, 18, 210 222. https://doi.org/10.1016/j.hrmr.2008.03.003
• Segeritz, M., & Pant, H. A. (2013). Do they feel the same way about math? Testing measurement invariance of the PISA students’ approaches to learning instrument across immigrant groups within Germany. Educational and Psychological Measurement, 73(4), 601-630 https://doi.org/10.1177/0013164413481802
• Shen, C. (2005). How American middle schools differ from schools of five Asian countries: Based on cross-national data from TIMSS 1999. Educational Research and Evaluation, 11(2), 179-199. https://doi.org/10.1080/13803610500110810
• Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417 453. https://doi.org/10.3102/00346543075003417
• Tabachnick, B. G., & Fidell, L.S. (2007). Using Multivariate Statistics, (5. ed.) Pearson Education.
• Tavşancıl, E. (2005) Tutumların Ölçülmesi ve SPSS ile Veri Analizi [Measuring Attitudes and Data Analysis with SPSS], Nobel Yayınları
• Teo, T. (2010). Gender differences in the intention to use technology: A measurement invariance analysis. British Journal of Educational Technology, 41(6), 120-124. https://doi.org/10.1111/j.1467-8535.2009.01023.x
• Uyar. Ş., & Doğan, N. (2014). An investigation of measurement invariance of learning strategies model across different groups in PISA Turkey sample, International Journal of Turkish Educational Studies, 2, 30-43.
• Vanderberg, R. J., & Lance, C. E., (2000). A review and synthesis of the measurement invariance literature: Suggestions practices, and recommendations for organizational research. Organizational Research Methods, 3(4), 4 70. https://doi.org/10.1177/109442810031002
• Walzebug, A. (2014). Is there a language-based social disadvantage in solving mathematical items?. Learning, Culture and Social Interaction, 3(2), 159 169. https://doi.org/10.1016/j.lcsi.2014.03.002
• White, K. R. (1982). The relation between socioeconomic status and academic achievement, Psychological Bulletin, 91(3), 461–481. https://doi.org/10.1037/0033-2909.91.3.461
• Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research & Evaluation, 12(3). https://doi.org/10.7275/mhqa-cd89
• Yıldırım, S. (2019). Predicting mathematics achievement: The role of socioeconomic status, parental involvement, and self-confidence, Education and Science, 44(198), 99-113. https://doi.org/10.15390/EB.2019.7868