A Review of Measurement Tools Developed and Adapted Based on the Rasch Model

Year 2023, Volume: 6 Issue: 2, 249 - 275, 27.10.2023

Emine Burcu Tunç

https://doi.org/10.53048/johass.1369336

Cited By: 1

Abstract

It is often observed that the Rasch model is frequently used in determining the psychometric properties of measurement tools because the Rasch model has many advantages in the development and adaptation of measurement instruments. The aim of this study is to evaluate the theses included in the National Thesis Center, which examine the psychometric properties of measurement tools within the framework of the Rasch model, within the scope of the requirements of the Rasch model. In line with this purpose, the model of the research is a document analysis research within the scope of qualitative research. All theses containing the word Rasch in the thesis name and index were examined, and 24 theses in which the measurement tool was developed and adapted within the scope of the Rasch model were found. In order to examine these measurement tools, a coding list was created and the data was analyzed by applying categorical analysis which is one of the content analysis methods. According to the results obtained, it was revealed that in the majority of theses, information was given about unidimensionality, but in half of the theses, no information was given about the local independence assumption. There are studies that do not specify which model is used for polytomous items, and it was observed that model comparison was not performed. It was determined that item model fit was generally tested with different approaches in the theses, and item parameters were generally included. It is among the results that the Person separation index related to reliability was not reported in all studies, and sufficient information was not provided in some studies, even though Differential Item Functioning analyses were performed. In light of these results, it is seen that there is no common systematic approach in the development or adaptation of measurement tools within the framework of the Rasch model in the studies. Therefore, it is recommended that more detailed studies explaining this systematic approach should be conducted.

Keywords

Rasch model, measurement tool, scale development, scale adaptation

References

• Acar Güvendir, M., & Özer Özkan, Y. (2015). Türkiye’deki eğitim alanında yayımlanan bilimsel dergilerde ölçek geliştirme ve uyarlama konulu makalelerin incelenmesi. Elektronik Sosyal Bilimler Dergisi, 52, 23-33. https://doi.org/10.17755/esosder.54872
• Ağır, H. (2019). Fiziksel ve sosyal katılım öznel indeksi (subjective index of physical and social outcome) anketinin Türkçe adaptasyon, geçerlik ve güvenirlik çalışması [Turkish adaptation, validity and reliability of the subjective index of physical and social outcome]. [Uzmanlık tezi, Kırıkkale Üniversitesi]. Ulusal Tez Merkezi
• Akşehirli Seyfeli, M.Y. (2023). Objektif yapılandırılmış sınav aracının klasik test kuramı, genellenebilirlik kuramı ve madde tepki kuramı ile değerlendirilmesi [Evaluation of objective structured examination tool with classical testing institution, generalizability theory and item response theory] [Yüksek lisans tezi, Erciyes Üniversitesi] Ulusal Tez Merkezi
• Akşehirli, Ö. (2022). Gebelerde doğum şekli hakkında bilgi düzeylerinin belirlenmesine yönelik test geliştirilmesi [Development of a test for determining the level of knowledge about the delivery method in pregnant women] [Doktora tezi, Ankara Üniversitesi] Ulusal Tez Merkezi
• Al-Deges, W. (2019). Pelvik taban sağlığı bilgi testi geliştirme, geçerlik ve güvenirliği development [Validity and reliability of pelvic floor health knowledge quiz] [Yüksek Lisans Tezi, Ankara Yıldırım Beyazıt Üniversitesi] Ulusal Tez Merkezi
• Alınca, G. (2018). Grup çalışmasına yönelik tutum ölçeğinin geçerlik ve güvenirliğinin incelenmesi [Validity and reliability of the attitude scale for group work] [Yüksek lisans tezi, Ege Üniversitesi] Ulusal Tez Merkezi
• Azrilah, A.A., Mohd, S.M., & Azami, Z. (2017). Asas model pengukuran rasch: pembentukan skala dan struktur pengukuran (1st ed). Penerbit Universiti Kebangsaan Malaysia.
• Barış Pekmezci, F., & Ayan C. (2020). Confusion of scale development: Investigation of self-efficacy scales. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, 48, 130-151. https://doi.org/10.9779/pauefd.529986
• Bond, T. G., & Fox, C. M. (2015). Applying the rasch model: fundamental measurement in the human sciences (3rd ed.). Mahwah, NJ: L. Erlbaum.
• Bond, T.G., & Fox, C.M. (2015). Applying the rasch model: fundamental measurement in the human sciences (3rd ed.). Mahwah, NJ: L. Erlbaum.
• Boone, W.J. (2016). Rasch analysis for instrument development: why, when, and how?. CBE-Life Sciences Education, 15(4), 1-7.
• Bowen, G.A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27-40.
• Brentari, E., & Golia, S. (2008). Measuring job satisfaction in the social services sector with the Rasch model. Journal of Applied Measurement, 9(1), 45-56. Retrieved from http://www.unibs.it/sites/default/files/ricerca/allegati/10061.pdf
• Brinthaupt, T.M., & Kang, M. (2014). Many-faceted rasch calibration: An example using the self-talk scale. Assessment, 21(2), 241-249.
• Brown, T.A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford.
• Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. SAGE Publications.
• Clauser, B.E., & Mazor, K. M. (1998). Using statistical procedures to identify differential item functioning test items. Educational Measurement: Issues and Practice, 17, 31-44.
• Corbin, J., & Strauss, A. (2015). Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage.
• Courville, T.G. (2004). An empirical comparison of item response theory and classical test theory item/person statistics (Unpublished doctoral dissertation). Texas A&M University, Texas. http://oaktrust.library.tamu.edu/bitstream/handle/1969.1/1064/etdtamu-2004B-EPSY-Courville-2.pdf?sequence=1&isAllowed=y
• Çüm, S., & Koç, N. (2013). Türkiye’de psikoloji ve eğitim bilimleri dergilerinde yayımlanan ölçek geliştirme ve uyarlama çalışmalarının incelenmesi, Eğitim Bilimleri ve Uygulama, 12(24), 115-135. Retrieved from https://www.idealonline.com.tr/IdealOnline/pdfViewer/index.xhtml?uId=5928&ioM=Paper&preview=true&isViewer=true#pagemode=bookmarks
• Çiçekçi, H.C. (2019). Tıp eğitiminde klinik öncesi eğitim dönemi ve klinik eğitim dönemi testlerinin psikometrik özelliklerinin incelenmesi: Ege Üniversitesi Tıp Fakültesi Örneği [The psychometric properties of the preclinical and clinical phase testing: A sample of the Ege University] [Yüksek lisans tezi, Ege Üniversitesi] Ulusal Tez Merkezi
• de Ayala, R. J. (2009). The theory and practice of item response theory. The Guilford Press.
• Delice, A., & Ergene, Ö. (2015). Ölçek geliştirme ve uyarlama çalışmalarının incelenmesi: Matematik eğitimi makaleleri örneği. Karaelmas Eğitim Bilimleri Dergisi, 3(1), 60-75. Retrieved from https://dergipark.org.tr/tr/pub/kebd/issue/67216/1049114
• DeMars, C. (2010). Item response theory. Oxford University Press.
• Doğan, E. M. (2009). Türkiye’deki psikolojik çalışmalarda kullanılan testlerin psikometrik özelliklerinin incelenmesi: Kültürel açıdan test uyarlama çalışmalar. [Yayımlanmamış yüksek lisans tezi], Muğla Üniversitesi Sosyal Bilimler Fakültesi, Muğla.
• Ebel, R.L. & Frisbie, D.A. (1991). Essentials of educational measurement. Englewood Cliffs, New Jersey: Prentice Hall.
• Eckes, T. (2009). Many-facet Rasch measurement. In S. Takala (Ed.) Reference supplement to the manual for relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment (Section H). Council of Europe/Language Policy Division.
• Elhan, A.H., & Atakurt, Y. (2005). Ölçeklerin değerlendirilmesinde niçin Rasch analizi kullanılmalıdır? Ankara Üniversitesi Tıp Fakültesi Mecmuası, 58(1), 47-50.
• Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Quality of Life Research. Lawrence Erlbaum Associates.
• Engelhard, G. (2013). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. Routledge.
• Erkuş, A. (2007). Ölçek geliştirme ve uyarlama çalışmalarında karşılaşılan sorunlar. Türk Psikoloji Bülteni, 13(40), 17-25. https://bursa.psikolog.org.tr/tr/yayinlar/dergiler/1031828/tpb134004.pdf
• Erol, R. & Eskici, M. (2022). Analysis of developed/adapted scales for distance education. Journal of Educational Technology & Online Learning, 5(4), 936-951.
• Esen, Y. (2013). Development of a test for assessing teachers mathematical content knowledge for teaching geometric measurement at elementary grade level [Middle East Technical University] Ulusal Tez Merkezi
• Fidan, Ç. (2021). Türkiye’de geliştirilen dindarlık ölçekleri (1989-2015): dindarlık ölçme çalışmaları üzerine değerlendirmeler. Türk Din Psikolojisi Dergisi, (4), 101-118.
• Finch, W.H., & French F.B. (2007). Detection of crossing differential item functioning: a comprasion of four methods. Educational and Psychological Measurement, 67(4), 565-582.
• Garrett, P. (2009). A Monte Carlo study ınvestigating missing data, differential item functioning and effect size (Doctoral Dissertation). Georgia State University.
• Gül Ş., & Sözbilir, M. (2015). Fen ve matematik eğitimi alanında gerçekleştirilen ölçek geliştirme araştırmalarına yönelik tematik içerik analizi. Eğitim ve Bilim, 40(178), 85-102. http://dx.doi.org/10.15390/EB.2015.4070
• Güler, G., & Ayan, C. (2020). Review of attitude scales developed in Turkey between 2002-2018 regarding the scale development process. Journal of Faculty of Educational Sciences, 53(3), 839-864. https://doi.org/10.30964/auebfd.658488
• Hagquist, C., Bruce, M., & Gustavsson, J.P. (2009). Using the Rasch model in nursing research: an introduction and illustrative example. International journal of nursing studies, 46(3), 380-393.
• Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principles and appliccations. Springer Science and Business Media, LLC.
• Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. CA: Sage.
• Hinkin, T. R. (1995). A review of scale development practices in the study in organizations. Journal of Management, 21(5), 967-988. https://doi.org/10.1177/014920639502100509
• Hubley, A.M., & Zumbo, B.D. (1996). A dialectic on validity: Where we have been and where we are going. The Journal of General Psychology, 123(3), 207-215.
• Irmak, D.E. (2021). Dikkat eksikliği hiperaktivite bozukluğu (dehb) olan çocuklarda “çocuk aktivite öz değerlendirme ölçeği’nin (cosa)” Türkçe uyarlamasının geçerlilik ve güvenilirliğinin incelenmesi [Evaluation of the validity and reliability of the Turkish adaptation of the "child activity self-assessment scale (cosa)" in children with attention deficit and hyperactivity disorder] [Yüksek Lisans Tezi, Hacettepe Üniversitesi] Ulusal Tez Merkezi
• Jodoin, M.G., & Gierl, M.J. (2001). Evaluating Type I error and power rates using an effect size measure with logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329–349.
• Keskin, E. (2023). İnme sonrası mobiliteyi değerlendiren Abıloco anketinin Türkçe versiyonu, geçerlik ve güvenilirliğinin araştırılması [Investigation of Turkish version, validity and reliability of Abıloco questionnaire evaluating mobility after stroke] [Doktora Tezi, Hacettepe Üniversitesi] Ulusal Tez Merkezi
• Kılıç, A. F., Koyuncu, İ, & Uysal, İ. (2023). Scale development based on item response theory: A systematic review. International Journal of Psychology and Educational Studies, 10(1), 209-223. https://dx.doi.org/10.52380/ijpes.2023.10.1.982
• Koch, W.R., & Dodd, B.G. (1989). An investigation of procedures for computerized adaptive testing using partial credit scoring. Applied Measurement in Education, 2(4), 335-357.
• Koç, G. (2022). İş ve sosyal uyum ölçeğinin çocuk ve ebeveyn formunun Türkçe geçerlilik ve güvenilirlik çalışması [Tıpta Uzmanlık Tezi, Sağlık Bilimleri Üniversitesi] Ulusal Tez Merkezi
• Koşar, C. (2015). Hasta aktiflik düzeyi ölçüm aracının (patient activation measure) Türkçe ‘ye uyarlanması: geçerlik ve güvenirlik çalışması [Adaptation of patient activation measure into Turkish: reliability and validity] [Yüksek Lisans Tezi, Dokuz Eylül Üniversitesi] Ulusal Tez Merkezi
• Krishnan, S., & Idris, N. (2018). Using partial credit model to improve the quality of an instrument. International Journal of Evaluation and Research in Education, 7, 4, 313-316.
• Kulak, E. (2020). Hasta aktiflik düzeyinde klinisyen desteği ölçüm aracının (clinician support for patient activation measure) Türkçe’ye uyarlanması: geçerlik ve güvenirlik çalışması [Adaptation of clinician support for patient activation measure into Turkish: Reliability and validity study] [Uzmanlık Tezi, Marmara Üniversitesi] Ulusal Tez Merkezi
• Linacre, J.M. (2002). What do infit and outfit, mean-square and standardized mean. Rasch measurement transactions, 16(2), 878.
• Linacre, J.M. (2015). A user’s guide to Winsteps® Rasch-model computer programs. Beaverton, Oregon.
• Lord, F.M. (1980). Aplications of item response theory to practical testing problems. Lawrence Erlbaum Associates.
• Maindal, H.T., Sokolowski, I., & Vedsted, P. (2009). Translation, adaptation and validation of the American short form Patient Activation Measure (PAM13) in a Danish version. BMC public health, 9, 1-9.
• Mallinson, T. (2007). Why measurement matters for measuring patient vision outcomes. Optometry and Vision Science, 84(8), 675-682.
• Marais, I. (2009). Response dependence and the measurement of change. Journal of Applied Measurement, 10(1), 17-29.
• Maxwell, J.A. (1996) Qualitative research design: An interpretive approach, Thousand Oaks, CA: Sage.
• Miles, M., & Huberman, M. A. (1994). An expanded sourcebook qualitative data analysis. Sage Publications.
• Moral, E. (2020). Besleme uygulamaları ve yapısı anketi Türkçe geçerlilik güvenilirlik çalışması [Feeding practices and structure questionnaire Turkish validity and reliability study] [Yüksek lisans tezi, Marmara Üniversitesi] Ulusal Tez Merkezi
• Morizot, J., Ainsworth, A.T., & Reise, S.P. (2007). Toward modern psychometrics. In R. W. Robins, R. C. Fraley, & R. F. Krueger (Eds.), Handbook of research methods in personalty psychology. The Guilford Press.
• Murphy, K.R., & Davidshofer, C.O. (2005). Psychological testing: principles and applications. New Jersey: Pearson Education International.
• O’Leary, Z. (2017). The essential guide to doing your research project. SAGE Publications Inc.
• Özalp Ateş, F.S. (2015). Ölçeklerde yapı geçerliliğinin değerlendirilmesinde faktör analizi ve Rasch analizi yaklaşımları [Factor analysis and Rasch analysis in the evaluation of construct validity of scales] [Yüksek lisans tezi, Ankara Üniversitesi] Ulusal Tez Merkezi
• Özdemir Deniz, P. (2021). Çocuk ve ergenlerde dısabkıds astım modülünün geçerlilik ve güvenilirliği, astım yönetiminde video eğitiminin yaşam kalitesi üzerine etkisi [Validity and reliability of the disabkids asthma module in children and adolescents, the effect of video education on quality of life in asthma management] [Uzmanlık Tezi, Aydın Adnan Menderes Üniversitesi] Ulusal Tez Merkezi
• Özdeş, N. (2018). Lise öğrencilerinin fiziksel uygunluk bilgi düzeylerinin incelenmesi [Physical fitness knowledge levels (PFKL) of high school students] [Yüksek lisans tezi, Çanakkale Onsekiz Mart Üniversitesi] Ulusal Tez Merkezi
• Öztuna, D. (2008). Kas-iskelet sistemi sorunlarının özürlülük değerlendiriminde bilgisayar uyarlamalı test yönteminin uygulanması [An application of computerized adaptive testing in the evaluation of disability in musculoskeletal disorders] [Doktora Tezi, Ankara Üniversitesi] Ulusal Tez Merkezi
• Pallant, J.F., & Tennant, A. (2007). An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale (HADS). British Journal of Clinical Psychology, 46(1), 1-18.
• Price, L.R. (2017). Psychometric methods: Theory and practice. New York, NY: The Guilford Press.
• Riazi, A., Aspden, T., & Jones, F. (2014). Stroke Self-efficacy Questionnaire: a Rasch-refined measure of confidence post stroke. Journal of rehabilitation medicine, 46(5), 406-412.
• Røe, C., Damsgård, E., Fors, T., & Anke, A. (2014). Psychometric properties of the pain stages of change questionnaire as evaluated by Rasch analysis in patients with chronic musculoskeletal pain. BMC Musculoskelet Disord, 15(1), 95. https://dx.doi.org/10.1186/1471-2474-15-95, PubMed 24646065
• Roever, C. (2005). That’s not fair! Fairness, bias, and differential item functioning in language testing. SLS Brownbag, 9(15), 1-14.
• Sakınmaz, E. (2019). Çocuklar için istismar bilgi ölçeğinin Türkçe’ye uyarlanması, geçerlik ve güvenirliği [Adaptation, validity and reliability of exclusive information survey for children] [Yüksek lisans tezi, Akdeniz Üniversitesi] Ulusal Tez Merkezi
• Slavec, A., & Drnovsek, M. (2012). A perspective on scale development in entrepreneurship research. Economic and Business Review, 14(1), 39-62. http://ojs.ebrjournal.net/ojs/index.php/ebr/article/view/69/pdf
• Soycan, M., & Babacan, E. (2019). Müziksel işitme, okuma ve yazma ile ilgili geliştirilmiş ölçme araçlarının incelenmesi: içerik analizi çalışması. Elektronik Sosyal Bilimler Dergisi, 18(69), 343-353.
• Sumintono, B. (2017) Rasch Model Measurement as Tools in Assessment for Learning. In: International Conference on Educational Innovation (ICEI 2017), 14 October 2017, Wyndham Hotel, Surabaya, Indonesia.
• Şahin, A. (2022). İnhalasyon uygulamalarına yönelik ebeveyn bilgi ölçeğinin geliştirilmesi [Development of parental knowledge scale for inhalation practices] [Doktora Tezi, Atatürk Üniversitesi] Ulusal Tez Merkezi
• Şengül Avşar, A., & Barış Pekmezci, F. (2023). Examination of motivation scales: Is the purpose academic promotion or the need to measure psychological constructs? Psycho-Educational Research Reviews, 11(3), 774-791. doi: 10.52963/PERR_Biruni_V11.N3.19
• Şenol, A. (2018). Klinik hemşirelik uygulamalarına yönelik öz düzenlemeli öğrenme ölçeğinin geçerlik güvenirliğinin incelenmesi [The investigation of the validity and reliability of the self-regulated learning scale in clinical nursing practice] [Yüksek lisans tezi, Ege Üniversitesi] Ulusal Tez Merkezi
• Tavşancıl, E., & Aslan, E. (2001). Sözel, yazılı ve diğer materyaller için içerik analizi ve uygulama örnekleri. Epsilon Yayınları.
• Tavşancıl, E., Güler, G., & Ayan, C. (2014). 2002-2012 yılları arasında Türkiye’de geliştirilen bazı tutum ölçeği geliştirme çalışmalarının ölçek geliştirme sureci acısından incelenmesi. IV. Ulusal Eğitimde ve Psikolojide Ölçme ve Değerlendirme Kongresi (Uluslararası Katılımlı) 9-13 Haziran, Hacettepe Üniversitesi, Ankara.
• Teke, C. (2017). Pozitif mental sağlık ölçeğinin Türkçe geçerlilik ve güvenirliği [Reliability and validity of the positive mental health questionnaire in a sample of Spanish University students] [Yüksek lisans tezi, İzmir Kâtip Çelebi Üniversitesi] Ulusal Tez Merkezi
• Tennant, A., & Conaghan, P. G. (2007). The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper?. Arthritis Care & Research, 57(8), 1358-1362.
• Tetik, E. (2022). Afazide aile yaşamı etki ölçeğinin Türkçe ‘ye uyarlanması, geçerlik ve güvenirlik çalışması [Adaptatıon of the family aphasia measure of life impact scale into Turkish: validity and reliability study] [Yüksek lisans tezi, Anadolu Üniversitesi] Ulusal Tez Merkezi
• Thorndike, R.L. (1982). Aplied psychometrics. Houghton Mifflin Company, Boston
• Tosun, C., & Taşkesenligil, Y. (2015). The instruments used in science education in Turkey: A descriptive content analysis. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 15(2), 364- 383. Retrieved from https://dergipark.org.tr/tr/download/article-file/17460
• Tunay, Z. Ö. (2013). Çocuklarda kullanılan 25-maddelik Cardiff görsel yeti ölçeğinin Türkçe sürümünün geçerlilik ve güvenilirliği [The reliability and validity of Turkish version of 25-item cardiff visual ability questionnaire for children] [Yüksek lisans tezi, Ankara Üniversitesi] Ulusal Tez Merkezi
• Tüzüngüç, B. (2019). Ortaöğretim öğrencilerinin sosyobilimsel muhakeme yeteneklerinin araştırılması [Investigation of socio-scientific reasoning skills of high school students] [Yüksek lisans tezi, Marmara Üniversitesi] Ulusal Tez Merkezi
• Walker, C.M., Beretvas, S.N., & Ackerman, T.A. (2001). An examination of conditioning variables used in computer adaptive testing for DIF. Applied Measurement in Education, 14, 3-16.
• Walker, E.R., Engelhard, G., & Thompson, N.J. (2012). Using Rasch measurement theory to assess three depression scales among adults with epilepsy. Seizure, 21(6), 437-443. http://dx.doi.org/10.1016/j.seizure.2012.04.009
• Wei, S., Liu, X., & Jia, Y., (2014). Using rasch measurement to validate the instrument of students’ understanding of models in science (SUMS). International Journal of Science and Mathematics,12, 1067.
• Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806-838. https://doi.org/10.1177/0011000006288127
• Wright, B.D., & Linacre, J.M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370-371. Retrieved from https://www.rasch.org/rmt/rmt83b.htm
• Wright, B.D., & Masters, G. (1982). Rating scale analysis: Rasch measurement. Chicago: MESA Press.
• Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145.
• Yılmaz, Ö. (2017). Yeterlik kazanma ölçeğinin geçerlik ve güvenirliğinin incelenmesi [İnvestigation of validity and reliability of ascent to competence scale] [Yüksek lisans tezi, Ege Üniversitesi] Ulusal Tez Merkezi
• Zieky, M. (1993). DIF statistics in test development. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337–347). Erlbaum.
• Zumbo, B. D. (1999). A Handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Ontario, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.