COMPARISON OF TEST AND ITEM PARAMETERS UNDER GRADED RESPONSE MODEL (IRT) AND CLASSICAL TEST THEORY

Abstract

The primary objective of the present research was to investigate the test and item parameters of Constructivist Learnin Envirement Scale (CLES) under graded response model (GRM) and classical test theory (CTT) and submit the best model which was fitted to real data. The results from a traditionally reliability analysis and a eigenvalue plot indicated that CLES scale satisfied the unidimensionality assumption and reliable. The mean response of CLES scale items was 3,39 which reflected the fact that the items tended to have fairly moderate difficulty values overall. Regarding the item discrimination, there were 10 items with low discrimination values for CTT because their item dicsrimination values are less than 0,60 and .there was 3 items with a very low level for GRM analysis. Item discrimination parameters for each item ranged between 0,68 and 1,81. It should be noticed that item discriminations can be considered as generally adequate and efective at distinquishing among respondents based on their estimated trait levels. The discriminations from the graded response model and classical test theory analysis correlated highly (r=.96) with each other. As a result, it can be colcluded that item and test parameters were highly correlated and researchers could use two methods if their assumpions were met and limitations were considerd. 

Keywords

• Graded Response Model
• Item Response Theory
• Classical test Theory
• Item Parameter Estimation

AŞAMALI TEPKİ MODELİ VE KLASİK TEST KURAMI ALTINDA ELDE EDİLEN TEST VE MADDE PARAMETRELERİNİN KARŞILAŞTIRILMASI

Abstract

Bu çalışmanın temel amacı klasik test kuramı (KTK) ve aşamalı tepki modeli (ATM) altında Yapılandırmacı Öğrenme Ortamları Ölçeğinin kestirilen madde ve test parametrelerinin karşılaştırılarak, araştırmacılara hangi kuram altında kestirilen parametrelerin daha keskin ve güvenilir olduğunu sunmaktır. Geleneksel güvenirlik analizi ve özdeğer grafiği sonuçları veri grubunun tek boyutlu olduğunu ve ölçekten elde edilen sonuçların güvenilir olduğunu göstermektedir. Yapılandırmacı öğrenme ortamları ölçeği ortaokul formunun madde ortalaması 3,39 olarak hesaplanmış ve bu da orta düzeyde madde güçlüğüne karşılık gelmektedir. Madde ayırıcılık parametreleri incelendiğinde KTK altında 10 maddenin düşük ayırıcılığa sahip olduğu, ATM altında ise 3 maddenin düşük ayırıcılığa sahip olduğu ve ayırıcılık değerlerinin 0,61 ile 1,81 arasında değiştiği ortaya konmuştur. Her iki kuram altında madde ayırıcılık parametreleri arasında r=0,96 düzeyinde korelasyon bulunmuştur. Sonuç olarak her iki kuram altında kestirilen madde ve test parametrelerinin yüksek düzeyde ilişkili olduğu, model varsayımların karşılandığı ve sınırlılıkların göz önüne alındığı durumlarda her iki kuramında kullanılabileceği sonucuna ulaşılmıştır.

Keywords

• Aşamalı tepki modeli
• madde tepki kuramı
• klasik test kuramı
• Madde parametre kestirimi

References

1. Arkün, S. & Aşkar, P. (2010). The development of a scale on assessing constructivist learning environments. HU Journal of Education, 39, 32-43
2. Baker, F.B. (1985). The basics of item response theory. Portsmouth, NH:Heineman
3. Bobcock, B.G.E.(2009).Estimating a Noncompensatory IRT Model Using a modified Metropolis algorithm. Unpublished Doctoral Dissertation.The University of Minesota.
4. Bock, R. D.,Thissen, D. & Zimowski, M.F. (1997). IRT estimation of domain scores. Journal of Educational Measurement, 34, 197-211
5. Chow, P. & Winzer, M.M. (1992). Reliability and validity of a scale measuring attitudes toward mainstreaming. Educational and Psychological Measurement, 52, 223-228.
6. De Ayala, R.J., Dodd, B.G. & Koch, W.R. (1989). Acomparison of the graded response and partial credit models for assessing writing ability. Paper Presented at the Anuual Meeting of the National Council on Measurement in Education, San Francisco, CA.
7. Demars, C. (2010). Item response theory. Understanding statistics measurement. Oxford University Press.
8. Embretson, S.E. & Reise, S.P. (2000). Item Response Theory for Psychologists. Mahwah, NJ:Erlbaum.

9. Gray-Little, B., Williams, V.S.L. & Hancock, T.D. (1997). An item response theory analysis of the Rosenberg Self Esteem Scale. Personality and Social Psychology Bulletin, 23, 443-451.
10. Hambleton, R.K. & Swaminathan, H. (1985). Item Response Theory. Kluwer-Nijhoff Publishing. Boston-USA.
11. Hambleton, R.K. & Swaminathan, H. (1989). Item Response Theory. Principles And Applications. Kluwer-Nijhoff Publishing. Boston-USA.
12. Hambleton, R.K., Swaminathan, H.& Rogers, H.J. (1991). Fundamentals of Item Response Theory. Sage Publications, London.
13. LaHuis, D.M., Clark, P. & O’Brien, E. (2011). An examination of item response theory item fit indices fort he graded response model. Organizational Reseach Methods, 14(1), 10-23.
14. Lane, S., Stone, C.A., Ankenmann, R.D. & Liu, M. (1995). Examination of the assumptions and properties of the graded item response model: An example using a mathematics perfromance assessment. Applied Measurement in Education, 8(4), 313-340.
15. Lee, K.H. (1995). Application of the graded response model to the revised Tennessee self-concept scale: Unidimensionality, parameter invariance, and differential item functioning. Unpublished Doctoral Dissertation. University of Southern California.
16. Keith, P.R. (1983). Application of a graded response model to the assessment of job satisfaction. Unpublished Doctoral Dissertation. University of Illinois.
17. Koch, W.R. (1983). Likert scaling using the graded response latent trait model. Applied Psychological Measurement, 7(1), 15-32.
18. Madera, E.K. (2003). Application of the graded response model to the assessment of student attitudes. Unpublished Doctoral Dissertation. University of Toronto.
19. Marie, L.A. (1997). The application of item response theory to employee attitude survey. Unpublished Doctoral Dissertation. University of Connecticut.
20. Mielenz, T.J., Edwards, M.C. & Callahan, L.F. (2010). Item response theory analysis of two questionnaire measures of arthritis-related self efficacy beliefs from community based US samples. Hindawi Publishing Corporation Arthritis.
21. Muraki, E. (1990). Fitting a polytomous item response model to Likert type data. Applied Psychological Measurement, 14(1), 59-71.
22. Ostini, R. & Nering, M.L. (2006). Polytomous Item Response Theory Models. Sage Publications, Inc. California, USA.
23. Reise, S.P. & Yu, J. (1990). Parameter recovery in the graded response model using MULTİLOG. Journal of Educational Measurement, 27 (2), 133-144.
24. Roberts, J.S. & Laughlin, J.E. (1996). The graded unfolding model: A unidimensional item response model for unfolding graded responses. Research Report, Educational Testing Service, Princeton.
25. Rubio, V.J., Aguado, D., Hontangas, P.M. & Hernandez, J.M. (2007). Psychometric properties of an emotional adjustment measure. European Journal of Psychological Assessment, 23 (1), 39-46.
26. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric monograph No. 17). Richmond, VA: Psychometric society. Retrieved from http://www.psychometrika.org/journal/online/MN17.pdf
27. Seungho Yang, M. A. (2007). A Comparison of Unidimensional and Multidimensional RaschModels Using Parametrer Estimates and Fit İndices When Assumption of Unidimensionality is Violated. Unpublished Doctoral Dissertation. The Ohio State University
28. Sijstma, K. &Hemker, B.T. (2000). A taxonomyof IRT models for ordering persons anditems using simple sum scores. Journal of Educational and Behavioral Statistics, 25 (4), 391-415.
29. Sijstma, K. & Junker, B.W. (2006). Item response theory: Past performance, present developments and future expectatitons. Behaviormetrika, 33 (1), 75-102.
30. Stone, C.A., Akenmann, R.D. & Liu, M. (1995). Examination of the assumptions and properties of the graded item response model: An example using a mathematics perfromance assessment. Applied Measurement in Education, 8(4), 313-340.
31. Sukirno, A. & Sununta, S. (2010). The comparison of graded response model and classical test theory in human resource research: A model fitness test. Research and Practice in Human Resource Management, 18(2), 77-86.
32. Tezbaşaran, A. & Kelecioğlu, H. (2004). Madde-ölçek korelasyonlarına, alt-üst grup ortalamalarına ve aşamalı tepki modeline göre geliştirilen sigaraya ilişkin tutum ölçeğinin madde ve ölçek özelliklerinin incelenmesi. XIII. Ulusal Eğitim Bilimleri Kurultayı, İnönü Üniversitesi, Malatya.
33. Thissen, D. (1988). MULTILOG (Computer program). Mooresville, IN: Scientific Software.
34. Uttaro, T. & Lehman, A. (1999). Graded response modeling of the Quality of Life Interview. Evaluation and Program Planning, 22(1999), 41-52
35. Wu, M. & Adams, R. (2006). Modelling mathematics problem solving item responses using a multidimensional IRT model. Mathematics Education Research Journal, 18(2), 93-113.