Investigating Group Invariance of Equating Results

Year 2017, Volume: 16 Issue: 1, 0 - 0, 29.12.2016

Neşe Gübeş , Hülya Kalecioğlu

https://doi.org/10.17051/io.2017.34481

Abstract

In this study, it was investigated that Item Response Theory (IRT) equating results whether or not group invariant. Group invariance of equating functions means that equating is same for everyone in the population. The raw scores which were taken from 9^th grade 2009 ÖBBS D form of Social Sciences were equated to 2009 ÖBBS B form of Social Sciences with IRT true-and observed- score equating methods. Equating study was conducted with using equivalent groups design. The subgroups were generated with regard to examinees’ self-perceived competence in geography and history lessons. The results indicated that under the both IRT true-score and observed-score equating methods, equating results were group sensitive.

References

• Bandura, A. (1986). Social foundation of thought and action: A social cognitive theory. Prentice Hall, NJ: Englewood Cliffs.
• Braun, H. I., & Holland, P. W. (1982). Observed score equating: A mathematical analysis of some ETS equating
• procedures. In P. W. Holland & D. B. Rubin (Eds.). Test equating (pp.9-49). New York: Academic Press.
• Brennan, R. L. (2008). A discussion of population invariance. Applied Psychological Measurement, 32 (1), 102-114.
• Cook, L. L., & Eignor, D. R. (1991). An NCME instructional module on IRT equating methods. Educational
• Measurement: Issues and Practice, 10, 191-200.
• Cook, L. L., & Petersen, N. S. (1987). Problems related to the use of conventional and item response theory
• equating methods in less than optimal circumstances. Applied Psychological Measurement, 11, 225-244.
• Crocker, L., & Algina, J. (1986). Introduction to classical & modern test theory. Philadelphia: Harcourt Brace
• Jovanovich College Publishers.
• Dorans, N. J. (Ed.). (2003). Population invariance of score linking: Theory and applications to Advanced
• Placement program examinations (ETS Research Report RR-03-27). Princeton, NJ: Educational Testing Service.
• Dorans, N. J. (2004). Equating, concordance, and expectation. Applied Psychological Measurement, 28(4), 227
• Dorans, N. J., Feigenbaum, M. D. (1994). Equating issues engendered by changes to the SAT and PSAT/NMSQT.
• In I. M. Lawrance, N. J. Dorans, M. D. Feigenbaum, N. J. Feryok, A. P. Schmitt, & N. K. Wright (Eds.), Technical issues related to the introduction of the new SAT and PSAT/NMSQT. ETS Research memorandum. No. RM-94-10. Princeton, NJ: Educational Testing Service.
• Dorans, N. J., & Holland, P. W. (2000). Population invariance and the equitability of tests: Basic theory and the
• linear case. Journal of Educational Measurement, 37, 281-306.
• Dorans, N.J., Holland, P. W., Thayer, D.T., & Tateneni, K. (2002, April). Invariance of score linking across gender
• groups for three Advanced Placement Program exams. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, La.
• Dorans, N., Liu, J., & Hammond, S. (2008). Anchor test type and population invariance: An exploration across
• subpopulations and test administrations. Applied Psychological Measurement, 32, 81-97.
• Eğitim, Araştırma ve Geliştirme Daire Başkanlığı. (2010). Ortaöğretim ÖBBS raporu 2009. Retrieved from
• http://egitek.meb.gov.tr/dosyalar/obbs/OBBS_2009.pdf
• Hanson, B., & Zeng, L. (n.d.). PIE: A computer program for IRT equating. (Windows Console Version, Revised by
• Cui, May 20, 2004) [Manual]. Unpublished manuscript, College of Education, University of Iowa, Iowa City, Iowa
• Harris, D. J., & Kolen, M. J. (1986). Effect of examinee group on equating relationships. Applied Psychological Measurement, 10(1), 35-43.
• Holland, P. W., & Dorans, N. J. (2006). Linking and Equating. In R. L. Brennan (Ed.) Educational measurement
• (4th ed., pp. 187-220). Westport, CT: Praeger Publishers.
• Huggins, A. C., & Penfield, R. D. (2012). An NCME instructional module on population invariance in linking and equating. Educational Measurement: Issues and Practice, 31(1), 27-40.
• Kolen, M. J. (1988). An NCME instructional module on traditional equating methodology. Educational Measurement: Issues and Practice, 7, 29-36.
• Kolen, M. J. (2004). Population invariance in equating and linking: Concept and history. Journal of Educational Measurement, 41(1), 3-14.
• Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practices (2nd ed.). New York: Springer.
• Liu, M., & Holland, P. W. (2008). Exploring population sensitivity of linking functions across three law school admission test administrations. Applied Psychological Measurement, 32, 27-44.
• Petersen, N. S. (2008). A discussion of population invariance of equating. Applied Psychological Measurement,
• , 98-101.
• von Davier, A. A. (2007). Potential solutions to practical equating issues. In N. J. Dorans, M. Pommerich & P. W. Holland (Eds.). Linking and Aligning Scores and Scales (pp.89-105). New York: Springer.
• von Davier, A. A., & Wilson, C. (2008). Assumption of item response theory true-score equating across two subgroups of examinees and two test formats. Applied Psychological Measurement, 32, 11-26.
• Yang, W.-L., & Gao, R. (2008). Invariance of score linkings across gender groups for forms of a testlet-based College Level Examination Program Examination. Applied Psychological Measurement, 32, 45-61.
• Yi, Q., Harris, D., & Gao, X. (2008). Invariance of equating functions across different subgroups of examinees taking a science achievement test. Applied Psychological Measurement, 32, 62-80.
• Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (2003). BILOGMG 3.0 for Windows: Multiple-group IRT analysis and test maintenance for binary items [Computer software]. Lincolnwood, IL: Scientific Software International, Inc.