Differential Bundle Functioning of National Examinations Council Mathematics Test Items: An Exploratory Structural Equation Modelling Approach

Abstract

A differential bundle function (DBF) is a situation in which examinees who are of the same ability but are from different groups are required to answer groups of items differently. DBF undermines the validity of the instrument if inadequately considered. The study examines the dimensionality of the 2017 NECO Mathematics items, determines the effect of DBF on 2017 Mathematics items concerning sex, and investigates the effect of DBF on 2017 Mathematics items concerning school ownership. This study explores Exploratory Structural Equation Modelling (ESEM), which permits the cross-loading of items that are not allowed in other models. The ex-post facto research design was adopted using secondary data, while six bundles were generated via the specification table. The population for the study comprised all 1,034,629 Senior School three students. A total of 815,104 students were selected using the simple random technique. The instrument for the study was 2017 NECO Mathematics paper three with a Cronbach's alpha of 0.82, and data were analysed using Mplus 7.4. The results revealed that the 2017 NECO Mathematics is multidimensional and items in the bundles possess construct validity as they functioned differentially to examinees' sex and school type. We recommend ESEM has a better approach to examining DBF on 2017 NECO Mathematics test items.

Keywords

• Differential bundle functioning
• exploratory structural equation modelling
• National Examinations Council

References

1. Abina, D. B. (2014). Influence of teacher characteristics, availability and utilization of instructional materials on students’ performance in mathematics [Unpublished doctoral dissertation]. University of Ibadan.
2. Adegoke, B. A. (2011). Effect of direct teacher influence on dependent-prone students’ learning outcomes in secondary school mathematics. Electronic Journal of Research in Educational Psychology, 9, 283-308.
3. Adeyegbe, S. O., & Oke, M. G. (2002). Science, technology and mathematics (STM) for sustainable development: The role of public examining bodies. Proceedings of STAN annual conference (pp. 144-147). Science Teachers Association of Nigeria.
4. Adeyemo, D. A. (2005). Parental involvement interest in schooling and school environment as predictors of academic self-efficacy among senior secondary school students in Oyo State [Unpublished doctoral dissertation]. University of Ibadan.
5. Adeyemo, E. O., & Opesemowo, O. A. (2020). Differential test let functioning (DTLF) in senior school certificate mathematics examination using multilevel measurement modelling. Sumerianz Journal of Education, Linguistics and Literature, 3(11), 249-253. https://doi.org/10.47752/sjell.311.249.253
6. Akale, M. A. G. (1997). The relationship between attitude and achievement among mathematics students in senior secondary school. Journal of Science and Movement Education, 2, 77-85.
7. Akinsola, M. K. (1994). Comparative effects of mastery learning and enhanced mastery learning strategies on students’ achievement and self-concept mathematics [Unpublished doctoral dissertation]. University of Ibadan.
8. Aremu, O. A., & Sokan, B. O. (2003). A multi-causal evaluation of academic performance of Nigerian learners: Issues and implication for national development. Department of Guidance and counseling, University of Ibadan, Ibadan.

9. Asikhia, O. A. (2010). Students and teachers’ perception of the causes of poor academic performance in Ogun state secondary schools: Implications for counseling for national development. European Journal of Social sciences, 13(2), 28-36.
10. Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397-438. https://doi.org/10.1080/10705510903008204
11. Awopeju, O. A., & Afolabi, E. R. I. (2016). Comparative analysis of classical test theory and item response theory based item parameter estimates of senior school certificate mathematics examination. European Scientific Journal, 12, 263-284.
12. Ayanwale, M. A. (2019). Efficacy of item response theory in the validation and score ranking of dichotomous and polytomous response mathematics achievement tests in Osun State, Nigeria [Unpublished doctoral dissertation]. University of Ibadan.
13. Ayanwale, M. A. (2022). Performance of exploratory structural equation model (ESEM) in detecting differential item functioning. EUREKA: Social and Humanities, 1, 58–73. http://doi.org/10.21303/2504-5571.2022.002254
14. Black, S. (2001). Building blocks: How schools are designed and constructed affects how students learn. American School Board Journal, 188(10), 44-47.
15. Boughton, K. A., Gierl, M. J., & Khaliq, S. N. (2000). Differential bundle functioning on mathematics and science achievement tests: A small step toward understanding differential performance. Annual meeting of the Canadian Society for Studies in Education (CSSE), Edmonton, Alberta, Canada.
16. Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford publications.
17. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Sage.
18. Byrne, B. M. (1998). Structural equation modelling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Lawrence Erlbaum Associates.
19. Creswell, J. W. (2003). Research design: Qualitative, quantitative and mixed methods approaches (2nd ed.). Sage.
20. Douglas, J. A., Roussos, L. A., & Stout, W. (1996). Item-bundle DIF hypothesis testing: Identifying suspect bundles and assessing their differential functioning. Journal of Educational Measurement, 33(4), 465-484. https://doi.org/10.1111/j.1745-3984.1996.tb00502.x
21. Finch, W. H. (2012). The MIMIC model as a tool for differential bundle functioning detection. Applied Psychological Measurement, 36(1), 40-59. https://doi.org/10.1177/0146621611432863
22. Furlow, C. F., Raiford, R. T., & Gagné, P. (2009). The impact of multidimensionality on the detection of differential bundle functioning using simultaneous item bias test. Applied Psychological Measurement, 33(6), 441-464. https://doi.org/10.1177/0146621609331959
23. Gierl, M. J., Tan, X., & Wang, C. (2005). Identifying content and cognitive dimensions on the SAT® (Report No. 2005-11). College Board.
24. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
25. Jennrich, R. I., & Sampson, P. F. (1966). Rotation for simple loadings. Psychometrika, 31(3), 313-323. https://doi.org/10.1007/BF02289465
26. Jöreskog, K. (1969). A general approach to confirmatory factor analysis. Psychometrika, 34, 183-202.
27. Latifi, S., Bulut, O., Gierl, M., Christie, T., & Jeeva, S. (2016). Differential performance on national exams: Evaluating item and bundle functioning methods using English, Mathematics, and Science Assessments. SAGE Open, 6(2), 1-14. https://doi.org/10.1177/2158244016653791
28. Lee, S., Bulut, O., & Suh, Y. (2016). Multidimensional extension of multiple indicators multiple causes models to detect DIF. Educational and Psychological Measurement, 77(4), 545-569. https://doi.org/10.1177/0013164416651116
29. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130-149. https://doi.org/10.1037/1082-989X.1.2.130
30. Marsh, H. W., Guo, J., Dicke, T., Parker, P. D., & Craven, R. G. (2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set-ESEM: Optimal balance between goodness of fit and parsimony. Multivariate Behavioural Research, 55(1), 102-119. https://doi.org/10.1080/00273171.2019.1602503
31. Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22(3), 471-491. https://doi.org/10.1037/a0019227
32. Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10(1), 85-110. https://doi.org/10.1146/annurev-clinpsy-032813-153700
33. Marsh, H. W., Muthén, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students' evaluations of university teaching. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 439-476. https://doi.org/10.1080/10705510903008220
34. McCarty, F. A., Oshima, T. C., & Raju, N. S. (2007). Identifying possible sources of differential functioning using differential bundle functioning with polytomously scored data. Applied Measurement in Education, 20(2), 205-225. https://doi.org/10.1080/08957340701301660
35. Min, S., & He, L. (2020). Test fairness: Examining differential functioning of the reading comprehension section of the GSEEE in China. Studies in Educational Evaluation, 64, 100811. https://doi.org/10.1016/j.stueduc.2019.100811
36. Montoya, A. K., & Jeon, M. (2019). MIMIC models for uniform and nonuniform DIF as moderated mediation models. Applied Psychological Measurement, 44(2), 118-136. https://doi.org/10.1177/0146621619835496
37. Morin, A. J. S., & Maïano, C. (2011). Cross-validation of the short form of the physical self-inventory (PSI-S) using exploratory structural equation modeling (ESEM). Psychology of Sport and Exercise, 12(5), 540-554. https://doi.org/10.1016/j.psychsport.2011.04.003
38. Mucherah, W., Finch, W. H., & Keaikitse, S. (2012). Differential bundle functioning analysis of the self-description questionnaire self-concept scale for Kenyan female and male students using the MIMIC model. International Journal of Testing, 12(1), 78-99. https://doi.org/10.1080/15305058.2011.620724
39. Muthén, L., & Muthén, B. (2012). Mplus user's guide (7th ed.). Muthén and Muthén.
40. Ojimba, D. P. (2012). Strategies for teaching and sustaining mathematics as an indispensable tool for technological development in Nigeria. Journal of Mathematical Sciences, 3, 23-35.
41. Ong, Y. M., Williams, J., & Lamprianou, I. (2015). Exploring crossing differential item functioning by gender in mathematics assessment. International Journal of Testing, 15(4), 337-355. https://doi.org/10.1080/15305058.2015.1057639
42. Oshima, T. C., Raju, N. S., Flowers, C. P., & Slinde, J. A. (1998). Differential bundle functioning using the DFIT framework: Procedures for identifying possible sources of differential functioning. Applied Measurement in Education, 11(4), 353-369. https://doi.org/10.1207/s15324818ame1104_4
43. Perry, J. L., Nicholls, A. R., Clough, P. J., & Crust, L. (2015). Assessing model fit: Caveats and recommendations for confirmatory factor analysis and exploratory structural equation modeling. Measurement in Physical Education and Exercise Science, 19(1), 12-21. https://doi.org/10.1080/1091367X.2014.952370
44. Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29(4), 347–363. http://doi.org/10.1177/0734282911406661
45. Schmitt, T. A. (2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29(4), 304–321. http://doi.org/10.1177/0734282911406653
46. Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58(2), 159-194. https://doi.org/10.1007/BF02294572
47. Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52(4), 589-617. https://doi.org/10.1007/BF02294821
48. Tata, U. S., Abba, A., & Abdullahi, M. S. (2014). The causes of poor performance in mathematics among public senior secondary school students in Azare Metropolis of Bauchi State, Nigeria. IOSR Journal of Research & Method in Education, 4, 32-40.
49. Tate, R. (2002). Test dimensionality. In G. Tindal & T. M. Haladyna (Eds.), Large-scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 181-211). Lawrence Erlbaum.
50. Tsigilis, N., Gregoriadis, A., Grammatikopoulos, V., & Zachopoulou, E. (2018). Applying exploratory structural equation modeling to examine the student-teacher relationship scale in representative Greek sample. Frontiers in Psychology, 9, 733. https://www.frontiersin.org/article/10.3389/fpsyg.2018.00733
51. Umameh, M. A. (2011). A survey of factors responsible for students’ poor performance in mathematics in Senior Secondary School Certificate Examination (SSCE) in Idah Local Government Area of Kogi State, Nigeria [Unpublished BSc(ED) thesis]. University of Benin.
52. Uwadie, I. (2012). Federal government, teachers and parents battle students’ under-performance. Vanguard Newspaper. Retrieved September 23, 2022.
53. Wainer, H., & Thissen, D. (1993). Combining multiple-choice and constructed-response test scores: Toward a Marxist theory of test construction. Applied Measurement in Education, 6(2), 103-118. https://doi.org/10.1207/s15324818ame0602_1
54. Wainer, H., Wang, X.-B., & Thissen, D. (1994). How well can we compare scores on test forms that are constructed by examinees choice? Journal of Educational Measurement, 31(3), 183-199. https://doi.org/10.1111/j.1745-3984.1994.tb00442.x
55. Walker, C. M., Zhang, B., Banks, K., & Cappaert, K. (2011). Establishing effect size guidelines for interpreting the results of differential bundling functioning analyses using SIBTEST. Educational and Psychological Measurement, 72(3), 415-434. https://doi.org/10.1177/0013164411422250
56. Wang, J., & Wang, X. (2012). Structural equation modeling with Mplus methods and applications. Wiley/Higher Education Press.