Investigation of Measurement Invariance of Turkish Subtest on ABIDE 2016 in Relation to Characteristics of Teachers: Sub-sampling Method

Year 2023, Volume: 14 Issue: 2, 154 - 170, 30.06.2023

Süleyman Ülkü , Burcu Atar

https://doi.org/10.21031/epod.1084985

Abstract

The Ministry of National Education carried out the ABIDE (Monitoring and Evaluation of Academic Skills) in 2016 in order to test the knowledge and skills of 8th-grade students. Since the ABIDE 2016 study was implemented for the first time in our country, it is very important to prove measurement invariance for the validity of the results. Within the scope of this research, the measurement invariance of the success of the students in the Turkish test according to the education level and professional experience of the teachers was examined. In the research, data were obtained from the Ministry of National Education, Directorate-General of Measurement, Evaluation, and Examination Services. Responses of students to the multiple-choice items in the ABIDE 2016 Turkish test and teacher questionnaire data were used in the study. All the data were used in the investigation of measurement invariance according to professional experience. Investigation of measurement invariance according to education level was carried out both by using and not using the method of sub-sampling. Factor 10 and Mplus 7 programs were used in the analysis of the data. At the end of the study, the Turkish achievement model provided all levels of measurement invariance among the student groups formed according to the professional experience and education level of the teachers.

Keywords

Measurement invariance, ABIDE 2016, sub-sampling method

References

• AERA, APA & NCME. (2014). The standards for educational and psychological testing. American Educational Research Association.
• Akyüz, G. (2006). Türkiye ve Avrupa Birliği ülkelerinde öğretmen ve sınıf niteliklerinin matematik başarısına etkisinin incelenmesi. İlköğretim Online. 5(2), 61-74.
• Başusta, N. B. ve Gelbal, S. (2015). Gruplar arası karşılaştırmalarda ölçme değişmezliğinin test edilmesi: PISA öğrenci anketi örneği. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi. 30(4), 80-90. https://doi.org/10.24106/kefdergi.2570
• Bialosiewicz, S., Murphy, K. & Berry, T. (2013, June). An introduction to measurement invariance testing: resource packet for participants. http://comm.eval.org/HigherLogic/System/DownloadDocumentFile.ashx?DocumentFileKey=63758fed-a490-43f2-8862-2de0217a08b8
• Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley- Interscience Publication.
• Byrne Barbara M. (2004). Testing for Multigroup Invariance Using AMOS Graphics: A Road Less Traveled, Structural Equation Modeling: A Multidisciplinary Journal, 11(2), 272-300. https://doi.org/10.1207/s15328007sem1102_8
• Chen, F. F. (2007). Sensitivity of goodness of fit indices to lack of measurement invariance. Structural Equation Modeling, 14, 464–504. https://doi.org/10.1080/10705510701301834
• Cheung, G. W. & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5
• Çokluk, Ö., Şekercioğlu, G. ve Büyüköztürk, Ş. (2010). Sosyal bilimler için çok değişkenli istatistik SPSS ve Lisrel uygulamaları. Pegem Akademi.
• Dragan, D. & Topolsek, D. (2014, Haziran). Introduction to structural equation modeling: review, methodology and practical applications. The International Conference on Logistics & Sustainable Transport, Celje.
• Fraenkel, J. R., & Wallen, N.E. (2006). How to design and evaluate research in education. McGraw-Hill.
• Gall, J. P., Gall, M. D. & Borg, W. R. (1999). Applying educational research: A practical guide. Longman Publishing Group.
• Greenwald, R., Hedges, L. V., & Laine, R. D. (1996). The effect of school resources on student achievement. Review of Educational Research, 66(3), 361–396. https://doi.org/10.2307/1170528
• Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups?: Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44(11), 78-94. https://doi.org/10.1097/01.mlr.0000245454.12228.8f
• Hoyle, R.H. (2012). Model specification in structural equation modeling. In R. H. Hoyle (Ed), Handbook of Structural Equation Modeling 126-144. The Guilford Press.
• Jöreskog, K. G. & Sörbom, D. (1993). Lisrel 8: Structural equation modeling with the simplis command language. Scientific Software International, Inc.
• Kankaras, M., Vermunt, J. K., & Moors, G. (2011). Measurement Equivalence of Ordinal Items: A Comparison of Factor Analytic, Item Response Theory, and Latent Class Approaches. Sociological Methods & Research, 40(2), 279–310. https://doi.org/10.1177/0049124111405301
• Kline, R. B., (2011). Principles and practices of structural equation modelling. The Guilford Press.
• Millî Eğitim Bakanlığı. (2017). ABIDE 2016 ulusal raporu. https://odsgm.meb.gov.tr/meb_iys_dosyalar/2017_11/30114819_iY-web-v6.pdf
• Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Pyschometrika, 58(4), 525-543. https://doi.org/10.1007/BF02294825
• Millsap, R. E., & Olivera-Aguilar, M. (2012). Investigating measurement invariance using confirmatory factor analysis. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (pp. 380–392). The Guilford Press.
• Schmith, N. & Kuljanin, G. (2008). Measurement invariance: review of practice and implication. Human resources management review, 18(4), 210-222. https://doi.org/10.1016/j.hrmr.2008.03.003
• Tabachnick, B. G. & Fidell, L. S. (2013). Using multivariate statistics (5th Edition). Pearson Education.
• Van de Schoot, R., Lugtig, P., & Hox, J. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486-492. https://doi.org/10.1080/17405629.2012.686740
• Vandenberg, R. J. & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. https://doi.org/10.1177/109442810031002
• Widaman, K. F. & Reise, S. P., (1997). Exploring the measurement invariance of psychological instruments: Applications in substance use domain. The science of prevention: Methodological advances from alcohol and substance abuse research, 281-324. https://doi.org/10.1037/10222-009
• Wu, D. A., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assesment, Research & Evaluation, 12(3), 1-26. https://doi.org/10.7275/mhqa-cd89
• Yoon, M. & Mark H. C. Lai (2018). Testing factorial invariance with unbalanced samples. Structural Equation Modeling: A Multidisciplinary Journal, 25(2), 201-213. https://doi.org/10.1080/10705511.2017.1387859