Abstract

Bu çalışmada, pozitif duygu durumlarının (mutlu, huzurlu gibi) ölçülmesinde kullanılmak üzere geliştirilen 14 maddelik Pozitif Duygu Durum Ölçeği’nde yer alan likert tipi maddelerin beşli tepki kategorileri için maddeler-içi ve maddeler-arası geçişlilik özellikleri Aşamalı Tepki Modeli, Kısmi Puanlama Modeli ve Dereceli Ölçekleme Modeli kullanılarak karşılaştırmalı olarak incelenmiştir. Her bir model için, model veri uyumu, ölçekte yer alan maddelerin tepki kategorileri, kategori eşik (threshold) değerleri ve kategori sıraları hesaplanmış, maddeler arası eşdeğerlik varsayımının ne kadar sağlandığı incelenmiştir. Model veri uyumu, model kestirimlerinden elde edilen -2log likelihood ki-kare değerleri, AIC ve BIC değerleri kullanılarak değerlendirilmiştir. Uygulama verisinde 339 gönüllü katılımcı yer almaktadır ve ölçek için hesaplanan Cronbach α güvenirlik katsayısı 0.92’dir. Analizler ölçeğin tek boyutlu bir yapıya sahip olduğunu göstermektedir. Model-veri uyumu değerlendirmesi sonucunda veriye en iyi uyum sağlayan modelin Aşamalı Tepki Modeli olduğu sonucu elde edilmiştir. Aşamalı Tepki Modeli ile parametre kestirimi yapılan maddeler için gözlenen kategori eşik parametrelerinin hem madde içlerinde beklendiği gibi neredeyse eşit aralıklar ile sıralı olduğu hem de bu aralıkların ölçeği oluşturan tüm maddeler için karşılaştırılabilir olduğu sonucu elde edilmiştir.

Keywords

• likert
• pozitif duygu durumu
• madde parametre kestirimi
• çok kategorili madde tepki kuramı

Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models

Abstract

The purpose of this study was to investigate within- and between-threshold parameter invariance for items of a fourteen-item Positive Affect Scale developed to assess positive moods (like happy, peaceful, etc.) of university students. To test whether the estimated threshold parameters were as expected (1 to 5, with increments of 1) across all the 14 items, Graded Response, Partial Credit, and Rating Scale Models were fit the response data collected from 326 students. A comparison of the model fit statistics, such as the negative 2log likelihood and chi-square values, revealed that the Graded Response Model had the best fit and that the thresholds estimates for all the items in the Positive Affective Scale were reasonably close to the expected 1 to 5 values with increments of 1. The study illustrates how polytomous response models can be used to test the psychometric quality of items with ordinal rating scales.

Keywords

• likert
• positive affect
• item parameter estimation
• polytomous ıtem response theory

References

1. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561-573. doi: 10.1007/BF02293814
2. Baker, F. B. (2001). The basis of item response theory. USA: ERIC Clearing house on Assessment and Evaluation.
3. Baker, J. G., Rounds, J. B., & Zevon, M. A. (2000). A comparison of graded response and Rasch partial credit models with subjective well-being. Journal of Educational and Behavioral Statistics, 25(3), 253-270. doi: 10.3102/10769986025003253
4. Brown, T. A. (2015). Confirmatory factor analysis for applied research. New York, NY: Guilford.
5. Caycho-Rodríguez, T., Vilca, L. W., Carbajal-León, C., White, M., Vivanco-Vidal, A., Saroli-Araníbar, D., ..., Moreta-Herrera, R. (2021). Coronavirus anxiety scale: New psychometric evidence for the Spanish version based on CFA and IRT models in a Peruvian sample. Death Studies. doi: 10.1080/07481187.2020.1865480
6. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. doi: 10.18637/jss.v048.i06
7. Chernyshenko, O. S., Stark, S., Chan, K. Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: issues and insights. Multivariate Behavioral Research, 36(4), 523-562. doi: 10.1207/S15327906MBR3604_03
8. Cho, S., Drasgow, F., & Cao, M. (2015). An investigation of emotional intelligence measures using item response theory. Psychological Assessment, 27(4), 1241-1252. doi: 10.1037/pas0000132

9. de Ayala, R. J., Dodd, B. G., & Koch, W. R. (1990, April). A comparison of the partial credit and graded response model in computerized adaptive testing. Paper presented at the AERA Annual Meeting. Boston.
10. DeMars, C. (2010). Item response theory: Understanding statistics measurement. Oxford: Oxford University Press.
11. Demirtaşlı, N., Yalçın, S., & Ayan, C. (2016). The development of irt based attitude scale towards educational measurement course. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 1(7), 133-144. doi: 10.21031/epod.43804
12. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New Jersey: LEA publishers.
13. Ferrando, P., Lorenzo, U., & Molina, G. (2001). An item response theory analysis of response stability in personality measurement. Applied Psychological Measurement, 25(1), 3-17. doi: 10.1177/01466216010251001
14. Flannery, W. P., Reise, S. P., & Widaman, K. F. (1995). An item response theory analysis of the general and academic scales of the self-description questionnaire II. Journal of Research in Personality, 29(2), 168-188. doi: 10.1006/jrpe.1995.1010
15. Glass, G. V., & Hopkins, K. D. (1984). Statistical methods in education and psychology. Englewood Cliffs, NJ: Prentice Hall.
16. Gray-Little, B., Williams, V., & Hancock, T. (1997). An item response theory analysis of the Rosenberg self - esteem scale. Personality and Social Psychology Bulletin, 23(5), 443-451. doi: 10.1177/0146167297235001
17. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. New York: Springer Science and Business Media.
18. Hattie, J. (1992). Self-concept. Hillsdale, NJ: Erlbaum.
19. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. doi: 10.1080/10705519909540118
20. Kahraman, N., Akbaş, D., & Sözer, E. (2019). Bilişsel-olmayan öğrenme durum ve süreçlerini ölçme ve değerlendirmede boylamsal yaklaşımlar: Duygu Cetveli Alan Uygulaması örneği. Bolu Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 19(1), 257-269. doi: 10.17240/aibuefd.2019.19.43815-459831
21. Kaptan, S. (1995). Bilimsel araştırma ve istatistik teknikleri (10. Basım). Ankara: Rehber Yayınevi.
22. Koch, W. R. (1983). Likert scaling using the graded response latent trait model. Applied Psychological Measurement, 7(1), 15-32. doi: 10.1177/014662168300700104
23. Köse, İ. A. (2015). Aşamalı tepki modeli ve klasik test kuramı altında elde edilen test ve madde parametrelerinin karşılaştırılması. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 15(2), 184-197. Retrieved from https://dergipark.org.tr/tr/download/article-file/17439
24. Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85-106. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.424.2811&rep=rep1&type=pdf
25. Lord, F. M. (1980). Applications of item response theory practical testing problems. Hillsdale, NJ: Erlbaum.
26. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. doi: 10.1002/j.2333-8504.1994.tb01618.x
27. Min, S., & Aryadoust, V. (2021). A systematic review of item response theory in language assessment: implications for the dimensionality of language ability. Studies in Educational Evaluation, 68. doi: 10.1016/j.stueduc.2020.100963
28. Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159-176. doi: 10.1177/014662169201600206
29. R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
30. Roskam, E. E. (1985). Current issues in item response theory. In E. E. Roskam (Ed.), Measurement and personality assessment (pp. 3-19). Amsterdam: North Holland.
31. Rubio, V. J., Aguado, D., Hontangas, P. M., & Hernandez, J. M. (2007). Psychometric properties of an emotional adjustment measure: an application of the Graded Response Model. European Journal of Psychological Assessment, 23(1), 39-46. doi: 10.1027/1015-5759.23.1.39
32. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monography No. 17). Retrieved from https://www.psychometricsociety.org/sites/main/files/file-attachments/mn17.pdf?1576606975
33. Samejima, F. (1996). Evaluation of mathematical models for ordered polychotomous responses. Behaviormetrika, 23, 17-35. doi: 10.2333/bhmk.23.17
34. Silvia, P. J. (2021). The self-reflection and insight scale: applying item response theory to craft an efficient short form. Current Psychology. doi: 10.1007/s12144-020-01299-7
35. Steiger, J. H., & Lind, J. M. (1980, May). Statistically based tests for the number of common factors. Paper presented in Psychometric Society. Iowa City.
36. Wang, W. C., Wilson, M., & Shih, C. L. (2006). Modeling randomness in judging rating scales with a random-effects rating scale model. Journal of Educational Measurement, 43(4), 335-353. doi: 10.1111/j.1745-3984.2006.00020.x
37. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063-1070. doi: 10.1037/0022-3514.54.6.1063
38. Wu, M., & Adams, R. (2006). Modelling mathematics problem solving item responses using a multidimensional IRT model. Mathematics Education Research Journal, 18(2), 93-113. doi: 10.1007/BF03217438
39. Yaşar, M., & Aybek, E. C. (2019). Üniversite öğrencileri için bir yılmazlık ölçeğinin geliştirilmesi: Madde tepki kuramı temelinde geçerlilik ve güvenilirlik çalışması. İlköğretim Online, 18(4), 1687-1699. Retrieved from https://ilkogretim-online.org/fulltext/218-1597121020.pdf?1618815938
40. Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145. doi: 10.1177/014662168400800201