Differential Bundle Functioning of National Examinations Council Mathematics Test Items: An Exploratory Structural Equation Modelling Approach

Year 2023, Volume: 14 Issue: 1, 1 - 18, 25.03.2023

Oluwaseyi Opesemowo , Musa Adekunle Ayanwale , Titilope Opesemowo , Eyitayo Afolabi

https://doi.org/10.21031/epod.1142713

Cited By: 3

Abstract

A differential bundle function (DBF) is a situation in which examinees who are of the same ability but are from different groups are required to answer groups of items differently. DBF undermines the validity of the instrument if inadequately considered. The study examines the dimensionality of the 2017 NECO Mathematics items, determines the effect of DBF on 2017 Mathematics items concerning sex, and investigates the effect of DBF on 2017 Mathematics items concerning school ownership. This study explores Exploratory Structural Equation Modelling (ESEM), which permits the cross-loading of items that are not allowed in other models. The ex-post facto research design was adopted using secondary data, while six bundles were generated via the specification table. The population for the study comprised all 1,034,629 Senior School three students. A total of 815,104 students were selected using the simple random technique. The instrument for the study was 2017 NECO Mathematics paper three with a Cronbach's alpha of 0.82, and data were analysed using Mplus 7.4. The results revealed that the 2017 NECO Mathematics is multidimensional and items in the bundles possess construct validity as they functioned differentially to examinees' sex and school type. We recommend ESEM has a better approach to examining DBF on 2017 NECO Mathematics test items.

Keywords

Differential bundle functioning, exploratory structural equation modelling, National Examinations Council

References

• Abina, D. B. (2014). Influence of teacher characteristics, availability and utilization of instructional materials on students’ performance in mathematics [Unpublished doctoral dissertation]. University of Ibadan.
• Adegoke, B. A. (2011). Effect of direct teacher influence on dependent-prone students’ learning outcomes in secondary school mathematics. Electronic Journal of Research in Educational Psychology, 9, 283-308.
• Adeyegbe, S. O., & Oke, M. G. (2002). Science, technology and mathematics (STM) for sustainable development: The role of public examining bodies. Proceedings of STAN annual conference (pp. 144-147). Science Teachers Association of Nigeria.
• Adeyemo, D. A. (2005). Parental involvement interest in schooling and school environment as predictors of academic self-efficacy among senior secondary school students in Oyo State [Unpublished doctoral dissertation]. University of Ibadan.
• Adeyemo, E. O., & Opesemowo, O. A. (2020). Differential test let functioning (DTLF) in senior school certificate mathematics examination using multilevel measurement modelling. Sumerianz Journal of Education, Linguistics and Literature, 3(11), 249-253. https://doi.org/10.47752/sjell.311.249.253
• Akale, M. A. G. (1997). The relationship between attitude and achievement among mathematics students in senior secondary school. Journal of Science and Movement Education, 2, 77-85.
• Akinsola, M. K. (1994). Comparative effects of mastery learning and enhanced mastery learning strategies on students’ achievement and self-concept mathematics [Unpublished doctoral dissertation]. University of Ibadan.
• Aremu, O. A., & Sokan, B. O. (2003). A multi-causal evaluation of academic performance of Nigerian learners: Issues and implication for national development. Department of Guidance and counseling, University of Ibadan, Ibadan.
• Asikhia, O. A. (2010). Students and teachers’ perception of the causes of poor academic performance in Ogun state secondary schools: Implications for counseling for national development. European Journal of Social sciences, 13(2), 28-36.
• Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397-438. https://doi.org/10.1080/10705510903008204
• Awopeju, O. A., & Afolabi, E. R. I. (2016). Comparative analysis of classical test theory and item response theory based item parameter estimates of senior school certificate mathematics examination. European Scientific Journal, 12, 263-284.
• Ayanwale, M. A. (2019). Efficacy of item response theory in the validation and score ranking of dichotomous and polytomous response mathematics achievement tests in Osun State, Nigeria [Unpublished doctoral dissertation]. University of Ibadan.
• Ayanwale, M. A. (2022). Performance of exploratory structural equation model (ESEM) in detecting differential item functioning. EUREKA: Social and Humanities, 1, 58–73. http://doi.org/10.21303/2504-5571.2022.002254
• Black, S. (2001). Building blocks: How schools are designed and constructed affects how students learn. American School Board Journal, 188(10), 44-47.
• Boughton, K. A., Gierl, M. J., & Khaliq, S. N. (2000). Differential bundle functioning on mathematics and science achievement tests: A small step toward understanding differential performance. Annual meeting of the Canadian Society for Studies in Education (CSSE), Edmonton, Alberta, Canada.
• Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford publications.
• Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Sage.
• Byrne, B. M. (1998). Structural equation modelling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Lawrence Erlbaum Associates.
• Creswell, J. W. (2003). Research design: Qualitative, quantitative and mixed methods approaches (2nd ed.). Sage.
• Douglas, J. A., Roussos, L. A., & Stout, W. (1996). Item-bundle DIF hypothesis testing: Identifying suspect bundles and assessing their differential functioning. Journal of Educational Measurement, 33(4), 465-484. https://doi.org/10.1111/j.1745-3984.1996.tb00502.x
• Finch, W. H. (2012). The MIMIC model as a tool for differential bundle functioning detection. Applied Psychological Measurement, 36(1), 40-59. https://doi.org/10.1177/0146621611432863
• Furlow, C. F., Raiford, R. T., & Gagné, P. (2009). The impact of multidimensionality on the detection of differential bundle functioning using simultaneous item bias test. Applied Psychological Measurement, 33(6), 441-464. https://doi.org/10.1177/0146621609331959
• Gierl, M. J., Tan, X., & Wang, C. (2005). Identifying content and cognitive dimensions on the SAT® (Report No. 2005-11). College Board.
• Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
• Jennrich, R. I., & Sampson, P. F. (1966). Rotation for simple loadings. Psychometrika, 31(3), 313-323. https://doi.org/10.1007/BF02289465
• Jöreskog, K. (1969). A general approach to confirmatory factor analysis. Psychometrika, 34, 183-202.
• Latifi, S., Bulut, O., Gierl, M., Christie, T., & Jeeva, S. (2016). Differential performance on national exams: Evaluating item and bundle functioning methods using English, Mathematics, and Science Assessments. SAGE Open, 6(2), 1-14. https://doi.org/10.1177/2158244016653791
• Lee, S., Bulut, O., & Suh, Y. (2016). Multidimensional extension of multiple indicators multiple causes models to detect DIF. Educational and Psychological Measurement, 77(4), 545-569. https://doi.org/10.1177/0013164416651116
• MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130-149. https://doi.org/10.1037/1082-989X.1.2.130
• Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., & Craven, R. G. (2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set-ESEM: Optimal balance between goodness of fit and parsimony. Multivariate Behavioural Research, 55(1), 102-119. https://doi.org/10.1080/00273171.2019.1602503
• Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22(3), 471-491. https://doi.org/10.1037/a0019227
• Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10(1), 85-110. https://doi.org/10.1146/annurev-clinpsy-032813-153700
• Marsh, H. W., Muthén, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students' evaluations of university teaching. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 439-476. https://doi.org/10.1080/10705510903008220
• McCarty, F. A., Oshima, T. C., & Raju, N. S. (2007). Identifying possible sources of differential functioning using differential bundle functioning with polytomously scored data. Applied Measurement in Education, 20(2), 205-225. https://doi.org/10.1080/08957340701301660
• Min, S., & He, L. (2020). Test fairness: Examining differential functioning of the reading comprehension section of the GSEEE in China. Studies in Educational Evaluation, 64, 100811. https://doi.org/10.1016/j.stueduc.2019.100811
• Montoya, A. K., & Jeon, M. (2019). MIMIC models for uniform and nonuniform DIF as moderated mediation models. Applied Psychological Measurement, 44(2), 118-136. https://doi.org/10.1177/0146621619835496
• Morin, A. J. S., & Maïano, C. (2011). Cross-validation of the short form of the physical self-inventory (PSI-S) using exploratory structural equation modeling (ESEM). Psychology of Sport and Exercise, 12(5), 540-554. https://doi.org/10.1016/j.psychsport.2011.04.003
• Mucherah, W., Finch, W. H., & Keaikitse, S. (2012). Differential bundle functioning analysis of the self-description questionnaire self-concept scale for Kenyan female and male students using the MIMIC model. International Journal of Testing, 12(1), 78-99. https://doi.org/10.1080/15305058.2011.620724
• Muthén, L., & Muthén, B. (2012). Mplus user's guide (7th ed.). Muthén and Muthén.
• Ojimba, D. P. (2012). Strategies for teaching and sustaining mathematics as an indispensable tool for technological development in Nigeria. Journal of Mathematical Sciences, 3, 23-35.
• Ong, Y. M., Williams, J., & Lamprianou, I. (2015). Exploring crossing differential item functioning by gender in mathematics assessment. International Journal of Testing, 15(4), 337-355. https://doi.org/10.1080/15305058.2015.1057639
• Oshima, T. C., Raju, N. S., Flowers, C. P., & Slinde, J. A. (1998). Differential bundle functioning using the DFIT framework: Procedures for identifying possible sources of differential functioning. Applied Measurement in Education, 11(4), 353-369. https://doi.org/10.1207/s15324818ame1104_4
• Perry, J. L., Nicholls, A. R., Clough, P. J., & Crust, L. (2015). Assessing model fit: Caveats and recommendations for confirmatory factor analysis and exploratory structural equation modeling. Measurement in Physical Education and Exercise Science, 19(1), 12-21. https://doi.org/10.1080/1091367X.2014.952370
• Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29(4), 347–363. http://doi.org/10.1177/0734282911406661
• Schmitt, T. A. (2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29(4), 304–321. http://doi.org/10.1177/0734282911406653
• Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58(2), 159-194. https://doi.org/10.1007/BF02294572
• Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52(4), 589-617. https://doi.org/10.1007/BF02294821
• Tata, U. S., Abba, A., & Abdullahi, M. S. (2014). The causes of poor performance in mathematics among public senior secondary school students in Azare Metropolis of Bauchi State, Nigeria. IOSR Journal of Research & Method in Education, 4, 32-40.
• Tate, R. (2002). Test dimensionality. In G. Tindal & T. M. Haladyna (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 181-211). Lawrence Erlbaum.
• Tsigilis, N., Gregoriadis, A., Grammatikopoulos, V., & Zachopoulou, E. (2018). Applying exploratory structural equation modeling to examine the student-teacher relationship scale in representative Greek sample. Frontiers in Psychology, 9, 733. https://www.frontiersin.org/article/10.3389/fpsyg.2018.00733
• Umameh, M. A. (2011). A survey of factors responsible for students’ poor performance in mathematics in Senior Secondary School Certificate Examination (SSCE) in Idah Local Government Area of Kogi State, Nigeria [Unpublished BSc(ED) thesis]. University of Benin.
• Uwadie, I. (2012). Federal government, teachers and parents battle students’ under-performance. Vanguard Newspaper. Retrieved September 23, 2022.
• Wainer, H., & Thissen, D. (1993). Combining multiple-choice and constructed-response test scores: Toward a Marxist theory of test construction. Applied Measurement in Education, 6(2), 103-118. https://doi.org/10.1207/s15324818ame0602_1
• Wainer, H., Wang, X.-B., & Thissen, D. (1994). How well can we compare scores on test forms that are constructed by examinees choice? Journal of Educational Measurement, 31(3), 183-199. https://doi.org/10.1111/j.1745-3984.1994.tb00442.x
• Walker, C. M., Zhang, B., Banks, K., & Cappaert, K. (2011). Establishing effect size guidelines for interpreting the results of differential bundling functioning analyses using SIBTEST. Educational and Psychological Measurement, 72(3), 415-434. https://doi.org/10.1177/0013164411422250
• Wang, J., & Wang, X. (2012). Structural equation modeling with Mplus methods and applications. Wiley/Higher Education Press.