Comparative Study of Classical Test Theory and Item Response Theory Using Diagnostic Quantitative Economics Skill Test Item Analysis Results

Year 2018, Volume: 3 Issue: 1, 57 - 75, 26.05.2018

Lydia İjeoma Eleje Frederick Ekene Onah Chidiebere Christopher Abanobi

Abstract

In
this study a comparison was made on DQEST item parameters and test statistics
results estimated using
Classical Test Theory (CTT) approach and Item Response Theory (IRT) three
parameter logistic model (3PLM) to find out the similarities and differences in
the two frameworks.
517 randomly selected senior secondary three (SS3) economics students comprised
the sample. Three research questions guided the study. Responses obtained from
SS3 economics students in 50 multiple choice items of Diagnostic Quantitative
Economics Skill Test (DQEST) were used for the analysis. DQEST items certified
the unidimensionality, local independence and model-data fit assumptions. Then
results from CTT and IRT analyses were compared. In terms of very difficult
item and item that discriminate poorly, CTT were found not to be comparable
with the 3PLM the most appropriate model for DQEST data. The calculated
reliability value for CTT was found to be low when compared to that generated
by 3PLM. Therefore,
it could be concluded that there was disparity between CTT approach and 3
parameter IRT model in terms of item parameters and test statistics. Thus IRT model with the best data
fit should be employed for an enhanced test validity and reliability.

Keywords

Item response theory, classical test theory, item parameters, local independence, unidimensionality

References

• Adedoyin, C. (2010). Investigating the invariance of persons’ parameter estimates based on classical test and item response theories. An International Journal on Education Science, 2(2), 107-113. Retrieved from http:// krepublishers.com/...Journals/...2...2...Adedoyin.../IJES-2-2-107-10-033-Adedoyin-O...
• Adedoyin, O.O., & Mokobi, T. (2013). Using IRT psychometric analysis in examining the quality of junior certificate mathematics multiple choice examination test items. International Journal of Asian Social Science, 3(4), 992-1011. Retrieved from http://www.aessweb.com/journal etail.php?id=5007
• Ani, E.N. (2014). Application of item response theory in the development and validation of multiple choice test in economics. (Master’s thesis). University of Nigeria, Nsukka.
• Bichi, A.A., Embong, R., Mamat, M., & Maiwada, D. A. (2015). Comparison of classical test theory and item response theory: a review of empirical studies. Australian Journal of Basic and Applied Sciences, 9(7), 549-556. Doi:10.13140/RG.2.1.1561.
• Cappelleri, J.C., Lundy, J.J., & Hays, R. D. (2014). Overview of classical test theory and item response theory for quantitative assessment of items in developing patient-reported outcome measures. Doi: 10.1016/j.clinthera.2014.04.006
• Esomonu, N.P.M. & Eleje, L.I. (2017). Diagnostic quantitative economics skill test for secondary schools: Development and validation using item response theory. Journal of Education and Practice, 8(22), 110-125. Retrieved from www.iiste.org
• Fan, X. (1998). Item response theory and classical test theory: an empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58(3), 357. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/0013164498058003001
• Guler, N., Uynik, G.K., & Teker, G.T. (2014). Comparison of classical test theory and item response theory in terms of item parameters. European Journal of Research on Education, 2(1), 1-6. Retrieved from http://iassr.org/journal
• Hambleton, R.K. and Swaminathan, H. (1985). Item response theory. Principles and application. Retrieved from www.springer.com/gp/book/9780898380651
• Hambleton, R.K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA:Sage Publications
• Hays, R. D., Bjorner, J. B., Revicki, D. A., Spritzer, K. L., & Cella, D. (2009). Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Quality of Life Research, (18), 873–80.
• Kolawole, E.B. (2010). Principles of test construction and administration (Revised Edition). Lagos: Bolabay Publications.
• Kyung, T. H. (2013). Windows software that generates IRT parameters and item responses: research and evaluation program methods (REMP). University of Massachusetts Amherst. Retrieved from https://www.umass.edu/remp/software/simcata/wingen/homeF.html
• Lord, F.M. (1980). Application of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
• Magis, D. (2007). Influence, information and item response theory in discrete data analysis. Retrieved from http://bictel.ulg.ac.be/ETDdb/collection/available/ULgetd-06122007-100147/.
• Mead, A.D., & Meade, A.W. (2010). Item selection using CTT and IRT with unrepresentative samples. Paper presented at the twenty-fifth annual meeting of the Society for Industrial and Organizational Psychology in Atlanta, GA. Retrieved from https://www.researchgate.net/...Classical_Test_Theory...Item.../Comparison-of-Classica...
• Ojerinde, D. (2013). Classical test theory (CTT) vs item response theory (IRT): an evaluation of the comparability of item analysis results. Retrieved from https://ui.edu.ng/sites/.../PROF%20OJERINDE'S%20LECTURE%20(Autosaved).pdf
• Ojerinde, D., & Ifewulu, B. C. (2012). Item unidimensionality using 2010 unified tertiary matriculation examination mathematics pre-test. A paper presented at the 2012 international conference of IAEA, Kazastan. Retrieved from https://ui.edu.ng/sites/.../PROF%20OJERINDE'S%20LECTURE%20(Autosaved).pdf
• Rijn, R.W.V., Sinharay, S., Haberman, S.J., & Johnson, M.S. (2016). Assessment of fit of item response theory models used in large-scale educational survey assessments. DOI: 10.1186/s40536-016-0025-3
• Stage, C. (1998). A comparison between item analysis based on item response theory and classical test theory. A study of the SweSAT subtest ERC. (Educational Measurement). Umeå University, Department of Educational Measurement. Retrieved from www.edusci.umu.se/digitalAssets/60/60608_enr3098sec.pdf
• Thorpe, G. L., & Favia, A. (2012). Data analysis using item response theory methodology: an introduction to selected programs and applications. Retrieved from http://digitalcommons.lidrary.umaine.edu/psy_facpub/20