Development of a “Perceived Stress Scale" Based on Classical Test Theory and Graded Response Model

Öz

The main purpose of this study is to develop a perceived stress scale based on Classical Test Theory (CTT) and Graded Response Model (GRM); to compare the parameters of the items in the scale that are tried to be developed according to both models, and to determine under which theory the measurement tool produces more reliable and valid results according to these compared item parameters. The item discrimination parameter value calculated according to CTT ranges from 0.472 to 0.735. On the other hand, item discrimination parameter values calculated under GRM vary between 1.062 (Item 15) and 2.606 (Item 2). Correlations between item thresholds were tested and the calculated correlation coefficients were; r =0.840 for β1 (p<0.01), r = 0.947 for β2 (p<0.01), r = 0.713 for β3 (p<0.05), and r = 0.559 for β4 (p<0.05) respectively. It can be assumed that these values not only support the item invariance of the items in the scale, but also show that the GRM is suitable for the data used for the scaling of the items. The reliability coefficient of the scale, in terms of internal consistency, was calculated as 0.919 according to the CTT, while the marginal reliability coefficient calculated as 0.931 according to GRM. Both reliability coefficients are quite high. In conclusion, there is a high correlation between the item parameters calculated according to both approaches, and the perceived stress scale (PSS) that is being developed can measure the desired features.

Anahtar Kelimeler

• Classical test theory
• Item response theory
• Item difficulty
• Item discrimination index
• Item invariance

Development of a “Perceived Stress Scale" Based on Classical Test Theory and Graded Response Model

Abstract

The main purpose of this study is to develop a perceived stress scale based on Classical Test Theory (CTT) and Graded Response Model (GRM); to compare the parameters of the items in the scale that are tried to be developed according to both models, and to determine under which theory the measurement tool produces more reliable and valid results according to these compared item parameters. The item discrimination parameter value calculated according to CTT ranges from 0.472 to 0.735. On the other hand, item discrimination parameter values calculated under GRM vary between 1.062 (Item 15) and 2.606 (Item 2). Correlations between item thresholds were tested and the calculated correlation coefficients were; r =0.840 for β1 (p<0.01), r = 0.947 for β2 (p<0.01), r = 0.713 for β3 (p<0.05), and r = 0.559 for β4 (p<0.05) respectively. It can be assumed that these values not only support the item invariance of the items in the scale, but also show that the GRM is suitable for the data used for the scaling of the items. The reliability coefficient of the scale, in terms of internal consistency, was calculated as 0.919 according to the CTT, while the marginal reliability coefficient calculated as 0.931 according to GRM. Both reliability coefficients are quite high. In conclusion, there is a high correlation between the item parameters calculated according to both approaches, and the perceived stress scale (PSS) that is being developed can measure the desired features.

Keywords

• Classical test theory
• Item response theory
• Item difficulty
• Item discrimination index

References

1. Akdağ, F., & Yüksel, M. (2010). İnsan kaynakları yönetimi açısından işkoliklik ve algılanan stres ilişkisinde kontrol odağının rolü. [Framing Focus of Control & Workaholism Positively with Reference to Perceived Stress]. Organizasyon ve Yönetim Bilimleri Dergisi, 2(1), 47-55.
2. Bichi, A.A., Embong, R, Mamat, M & Maiwada, D.A. (2015). Comparison of classical test theory and item response theory: A Review of empirical studies. Australian Journal of Basic and Applied Sciences, 9 (7), 549-556.
3. Bichi, A.A. & Talib, R. (2018). Item response theory: An introduction to latent trait models to test and item development. International Journal of Evaluation and Research in Education (IJERE), 7(2), 142-151.
4. Bilgen, Ö.B., & Doğan, N., (2017). Çok kategorili parametrik ve parametrik olmayan madde tepki kuramı modellerinin karşılaştırılması [Comparison of polytomous parametric and nonparametric item response theory models]. Journal of Measurement and Evaluation in Education and Psychology, 8(3), 354-372.
5. Chernyshenko, O. S., Stark, S., Chan, K. Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523-562.
6. Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. USA: Rinehart and Winston Inc.
7. Demars, C. (2010). Item response theory. Understanding statistics measurement. (Turkish translation editor: H. Kellecioğlu). Oxford University Press
8. Doğan, N. & Tezbaşaran, A. (2003). Klasik test kuramı ve örtük özellikler kuramının örneklemler bağlamında karşılaştırılması [Comparison of classical test theory and latent traits theory by Samples]. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 25(25), 58-67.

9. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
10. Erci, B. (2006) Reliability and validity of the Turkish version of perceived stress scale. Atatürk Üniv. Hemşirelik Yüksekokulu Dergisi, 9 (1), 52-67.
11. Erkuş, A., Sünbül, Ö., Sünbül, S., Yormaz, S., & Aşiret, S. (2017) Psikolojide ve Ölçme Ve Ölçek Geliştirme-II [Testing and Scale Development in Psychology]. Ankara: Pegem Akademi
12. Eskin, M., Harlak, H., Demirkıran, F., & Dereboy, Ç. (2013). Algılanan stres ölçeğinin Türkçe’ye uyarlanması: Güvenirlik ve geçerlik Analizi [The adaptation of the perceived stress scale into Turkish: A reliability and validity analysis]. New/Yeni Journal, 51 (3), 132-140
13. Fan, X. T. (1998). Item response theory and classical test theory: an empirical comparison of their item / person statistics. Educational and Psychological Measurement, 58(3), 357-381. Gelbal, S., (1994). p madde güçlük indeksi ile Rasch modelinin b parametresi ve bunlara dayalı yetenek ölçüleri üzerine bir karşılaştırma [p parameter of the Rasch model with the item difficulty index and A comparison of measures based on ability]. Hacettepe University Journal of Education Faculty, 10, 85-94.
14. Hambleton, R. K. & Swaminathan, H. (1985). Item response theory: principles and applications. Boston: Academic Puslishers Group
15. Hambleton, R.K., Swaminathan, H. & Rogers, H.J. (1991). Fundamentals of item response theory. Sage Publications, London.
16. Hambleton, R.K., & Jones, R.W.(1993) Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12 (3), 3847. Kan, A. (2006). Klasik test teorisine ve örtük özellikler teorisine göre kestirilen madde parametrelerinin karşılaştırılması üzerine ampirik bir çalışma. [An empirical study on the comparison of predicted item parameters with respect to classical and item response test theories]. Mersin Üniversitesi Eğitim Bilimleri Dergisi, 2 (2), 227-235.
17. Karaca, A., Yıldırım, N., Ankaralı, H., Çıkgöz, F., & Akkuş, D. (2015). Hemşirelik öğrencileri için algılanan stres, biyo-psiko-sosyal cevap ve stresle başetme davranışları ölçeklerinin Türkçe’ye uyarlanması [Turkish adaptation of perceived stress scale, Bio-psycho-social response, and coping behaviours of stress scales for nursing students]. Psikiyatri Hemşireliği Dergisi, 6 (1), 15-25.
18. Köksal, G., ve Kabasakal, Z. (2012) Zihinsel engelli çocukları olan ebeveynlerin yaşamlarında algıladıkları stresi yordayan faktörlerin incelenmesi. [The examination of predicting factors of perceived stress of parents with mental retarded children]. Buca Eğitim Fakültesi Dergisi, 32, 71-91.
19. Köse, A. (2015). Aşamalı tepki modeli ve klasik test kuramı altında elde edilen test ve madde parametrelerinin karşılaştırılması. [Comparison of test and item parameters under graded response model (IRT) and classical test theory]. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 15(2), 184 - 197.
20. LaHuis, D. M., & Copeland, D. A. (2009). Investigating faking using a multilevel logistic regression approach to measuring person fit. Organizational Research Methods, 12, 396-319.
21. Lautenschlager, G. J., Meade, A. W., & Kim, S. H. (2006). Cautions regarding sample characteristics when using the graded response model. Paper presented at the 21" Annual Conference of the Society for Industrial and Organizational Psychology, Dallas, Texas
22. Lord, F. N. & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison- Wesley.
23. Lord, F. M. (1980). Aplications of item response theory to practical testing problems. Hillsdale, New Jersey: Lawrence Erlbaum Associates.
24. Nartgün, Z. (2002). Aynı tutumu ölçmeye yönelik Likert tipi ölçek ile metrik ölçeğin madde ve ölçek özelliklerinin klasik test kuramı ve örtük özellikler kuramına göre incelenmesi. yayımlanmamış doktora tezi [The investigation of item and scale properties of Likert type scale and metric scale measuring the same attitude according to classisical test theory and item response theory], Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
25. Otrar, M., Ekşi, H., Dilmaç, B. & Şirin, A. (2002). Türkiye’de öğrenim gören Türk ve akraba topluluk öğrencilerinin stres kaynakları, başa çıkma tarzları ile ruh sağlığı arasındaki ilişki üzerine bir araştırma. [The sources of stress, coping, and psychological Well-Being among Turkic and relative societies' students in Turkey]. Kuram ve Uygulamada Eğitim Bilimleri, 2 (2), 473-506
26. Reckase, M.D. (2007). Multidimensional item response theory. C.R. Rao, S. Sinharay, (Ed.) Handbook of Statistics, Vol, 26: Psychometrics (pp. 607-642) Amsterdam: Elsevier
27. Reckase, M.D. (2009). Multidimensional item response theory. New York: Springer Dordrecht Heidelberg.
28. Robie, C., Zickar, M. J., & Schmit, M. J. (2001). Measurement equivalence between applicant and incumbent groups: An IRT analysis of personality scales. Human Performance, 14, 187-207.
29. Şanlı, Ö. (2017). Öğretmenlerin algılanan stres düzeylerinin çeşitli değişkenler açısından incelenmesi [Analysing the Perceived Stress Level of Teachers with Regards to Some Variables]. Electronic Journal of Social Sciences. 16(61), 85-96.
30. Savcı, M., & Aysan, F. (2014). Üniversite öğrencilerinde algılanan stres düzeyi ile stresle ile başa çıkma stratejileri arasındaki ilişki [The Relationship Between the Perceived Stress Level and the Stress Coping Strategies in University Students]. Uluslararası Türk Eğitim Bilimleri Dergisi. 3, 44-56.
31. Sijstma, K. & Junker, B.W. (2006). Item response theory: Past performance, present developments and future expectatitons. Behaviormetrika, 33(1), 75-102.
32. Sijtsma, K., & Meijer, R.R. (2007). Nonparametric item response theory and special topicd. In C.R. Rao and S. Sinharary (Eds.) Handbook of Statistics, Vol, 26: Psychometrics (pp. 719-746) Amsterdam: Elsevier
33. Stage, C. (1998a). A Comparison between item analysis based on item response theory and classical test theory. A study of the SweSAT Subtest WORD. Report, Sweden Umea University, Department of Educational Measurement. [Online]: Retrieved on 04-December-2007, at URL: http://www.umu.se/edmeas/publikationer/pdf/enr2998sec.pdf
34. Sünbül, Ö. & Erkuş, A. (2013) Madde parametrelerinin değişmezliğinin çeşitli boyutluluk özelliği gösteren yapılarda madde tepki kuramına göre incelenmesi. [Examining item parameter invariance for several dimensionality types by using unidimensional item response theory]. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 9(2), 378-398.
35. Tatar, A, Saltukoğlu, G. & Özmen, E. (2018) Madde yanıt kuramıyla öz bildirim türü stres ölçeği geliştirme çalışması – I: Madde seçimi, faktör yapısının oluşturulması ve psikometrik özelliklerinin incelenmesi. [Development of a self report stress scale using item response theory-I: Item selection, formation of factor structure and examination of its psychometric properties]. Arch Neuropsychiatry, 55, 161−170. https://doi.org/10.5152/npa.2017.18065
36. Turgut, M.F., (1992). Eğitimde Ölçme ve Değerlendirme Teknikleri. [Assessment and Evaluation in Education]. Ankara: Saydam Matbaacılık.
37. Turgut, M.F., & Baykul, Y. (1992). Ölçekleme Teknikleri. [Scaling Techniques]. ÖSYM Yayınları. Yayın No 1. Ankara
38. Yen, W. M. (1981). Using simulation results to choose a latent trait model. Applied Psychological Measurement, 5, 245-262.
39. Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three parameter logistic model. Applied Psychological Measurement, 8, 125-145.