A Comperative Adaptation of the Crick Learning for Resilient Agency (CLARA) with Classical Test Theory and Item Response Theory

Year 2022, Volume: 9 Issue: 4, 1030 - 1061, 22.12.2022

Hasan Fehmi Özdemir , Ömer Kutlu , Shaofu Huang , Ruth Crıck

https://doi.org/10.21449/ijate.1058145

Abstract

The aim of this study is to adapt the Crick Learning for Resilient Agency (CLARA) to Turkish culture, and to examine the psychometric features of the Inventory according to both Classical Test Theory (CTT) and Item Response Theory (IRT). In this respect, it is a descriptive level survey design research. Two different study groups were formed in accordance with the purpose of the study. Lingual equivalence applications were performed on two separate groups, one of which consisted of English Language and Literature Department students and the other consisted of English Language instructors. 1054 students participated in the validity and reliability studies from 101 different undergraduate programs at Ankara University. Before testing the research questions, it was examined whether the assumptions of CTT and IRT were met. With the application data; the predicted item discrimination indices, ability levels, students’ scores forming their learning power profiles, and reliability coefficient values were found to be similar in both theories. It can be said that with CLARA-Tr, obtained by adapting CLARA, a valid and reliable tool has been provided to the Turkish literature to be used in future studies.

Keywords

Learning power, Resilient agency, Classical Test Theory, Item Response Theory, Intercultural test adaptation

References

• Arthur, James & Crick, Ruth & Samuel, Elspeth & Wilson, Kenneth & Mcgettrick, Bart. (2006). Character Education: The Formation of Virtues and Dispositions in 16-19 Year Olds with particular reference to the religious and spiritual. West Conshohocken, PA: John Templeton Foundation. Retrieved from: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.473.7540&rep=rep1&type=pdf
• Broadfoot, P. (1998). Records of Achievement and the Learning Society: a tale of two discourses. Assessment in Education, 5(3), 447 477. https://doi.org/10.1080/0969595980050307
• Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö.E., Karadeniz, Ş., & Demirel, F. (2014). Bilimsel Araştırma Yöntemleri [Scientific Research Methods]. Pegem Akademi.
• Carr, M., & Claxton, G. (2002). Tracking the development of learning dispositions. Assessment in Education, 9 (1), pp. 9-37. https://doi.org/10.1080/09695940220119148
• Claxton, G. (1999). Wise Up: The Challenge of Lifelong Learning. Bloomsbury.
• Cohen, R.J., & Swerdlik, M.E. (2010). Psychological testing and assessment. McGrawHill Companies.
• Crick, R., & Bentley, J. (2020). Becoming a resilient organisation: integrating people and practice in infrastructure services. International Journal of Sustainable Engineering, 13(6), 423-440. https://doi.org/10.1080/19397038.2020.1750738
• Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Harcourt, New York.
• Cronbach, L.J. (1960). Essentials of psychological testing. Harper Collins Publishers.
• Deakin Crick, R. (2005). Learning Power: Dynamic Assessment for Learning. The Leader: Secondary Heads Association.
• Deakin Crick, R. (2006). Learning power in practice: A guide for teachers. Paul Chapman.
• Deakin Crick, R. (2007). Learning how to learn: the dynamic assessment of learning power. Curriculum Journal, 18(2), 135-153. https://doi.org/10.1080/09585170701445947
• Deakin Crick, R. (2009a). Assessment in Schools - Dispositions. In: McGaw, B., Peterson, P., & Baker, E. (eds.) The International Encyclopedia of Education (3rd Edition). Elsevier.
• Deakin Crick, R. (2009b). Inquiry-based learning: reconciling the personal with the public in a democratic and archaeological pedagogy. Curriculum Journal, 20(1), 73-92. https://doi.org/10.1080/09585170902764021
• Deakin Crick, R., Broadfoot, P., & Claxton, G. (2004). Developing an effective lifelong learning inventory: The ELLI Project. Assessment in Education, 11(3), 248-272. https://doi.org/10.1080/0969594042000304582
• Deakin Crick, R., & McCombs, B. (2006). The Assessment of Learner Centered Principles: An English Case Study. Educational Research and Evaluation, 12(5), 423-444. https://doi.org/10.1080/13803610600697021
• Deakin Crick, R., & Salway, A. (2006). Locked Up Learning: learning power and young offenders. ViTaL Partnerships Ltd.
• Deakin Crick, R., & Wilson, K. (2005). Being a Learner: A Virtue for the 21st Century. British Journal of Educational Studies, 53(5), 359-374. https://doi.org/10.1111/j.1467-8527.2005.00300.x
• Deakin Crick, R., & Yu, G. (2008). Assessing learning dispositions: is the effective lifelong learning inventory valid and reliable as a measurement tool? Educational Research, 50(4), 387-402. https://doi.org/10.1080/00131880802499886
• Deakin Crick, R., Haigney, D., Huang, S., Coburn, T., & Goldspink, C. (2013). Learning power in the workplace: the effective lifelong learning inventory and its reliability and validity and implications for learning and development. The International Journal of Human Resource Management, 24(11), 2255 2272. https://doi.org/10.1080/09585192.2012.725075
• Deakin Crick, R., Huang, S., Shafi, A.A., & Goldspink, C. (2015). Developing resilient agency in learning: The internal structure of learning power. British Journal of Educational Studies, 63(2), 121-160. https://doi.org/10.1080/00071005.2015.1006574
• De Ayala, R.J. (2009). The theory and practice of item response theory. U.S.A.: Guilford Press.
• Diker Coşkun, Y., & Demirel, M. (2012). Üniversite Öğrencilerinin Yaşam Boyu Öğrenme Eğilimleri [Lifelong Learning Dispositions of University Students]. Hacettepe University Journal of Education, 42, 108-120.
• Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates, Inc.
• Erkuş, A. (2013). Davranış Bilimleri İçin Bilimsel Araştırma Süreci [Scientific Research Process for Behavioral Sciences]. Seçkin Yayıncılık.
• Flannery, W.P., Reise, S.P., & Widaman, F.K. (1995). An item response theory analysis of the general and academic scales of self-description questionnaire-II. Journal of Research in Personality, 29(1), 168-188.
• Gornall, S., Chambers, M.R., & Claxton, G. (2005). Building Learning Power in Action. TLO Ltd. Retrieved from: https://www.buildinglearningpower.com/wp content/uploads/2014/11/BLPIA_A4_v1l-sample.pdf
• Hambleton, R., & Swaminathan, R. (1985). Fundamentals of Item Response Theory. Sage Publishers.
• Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. Sage.
• Hambleton, R.K. & Patsula, L. (1999). Increasing the validity of adapted tests: Myths to be avoided and guidelines for improving test adaptation practices. Journal of Applied Testing Technology, 1(1), 1-12.
• Hambleton, R.K., Merenda, P.F., & Spielberger, C.D. (2005). Adapting educational and psychological tests for cross-cultural assessment. Lawrence Erlbaum.
• Harlen, W., & Deakin Crick, R. (2003a). A systematic review of the impact of summative assessment and testing on pupils' motivation for learning. Retrieved from: https://dspace.stir.ac.uk/bitstream/1893/19607/1/SysRevImpSummativeAssessment2002.pdf
• Harlen, W., & Deakin Crick, R. (2003b). Testing and Motivation for Learning. Assessment in Education, 10(2), 169-207. https://doi.org/10.1080/0969594032000121270
• Hu, L.T., & Bentler, P.M. (1999). Cut off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. http://dx.doi.org/10.1080/10705519909540118
• International Test Commission, ITC. (2018). Guidelines for translating and adapting tests (Second edition). International Journal of Testing, 18(2), 101-134.
• Jöreskog, K.G., & Sörbom, D. (1993). Lisrel 8: structural equation modeling with the simplis command language. Erlbaum Associates Publishers.
• Karakılıç, M. (2009). Beden eğitimi dersi için hazırlanan tutum ölçeğinin psikometrik kuramlar açısından incelenmesi [Examination of the attitude scale prepared for the physical education lesson in terms of psychometric theories] [Unpublished Doctoral Thesis]. Ankara University.
• Kelecioğlu, H. (2001). Örtük Özellikler Teorisindeki b ve a Parametreleri İle Klasik Test Teorisindeki p ve r İstatistikleri Arasındaki İlişki [The relationship between b ve a parameters in latent trait theory and p and r statistiscs in classical test theory]. Hacettepe University Journal of Education, 20(1), 104-110.
• Kline, R.B. (2005). Principles and practice of structural equation modeling. The Guilford Press.
• Koch, W.R. (1983). Likert scaling using the graded response latent trait model. Applied Psychological Measurement, 7(1), 15-32. https://doi.org/10.1177/014662168300700104
• Masten, A. (2007). Resilience in developing systems: progress and promise as the fourth wave rises. Developmental Psychopathology, 19(3), 921 930. https://doi.org/10.1017/S0954579407000442
• MoNE (2014). Türkiye hayat boyu öğrenme strateji belgesi ve eylem planı 2014-2018 [Türkiye Lifelong Learning Strategy Document and Action Plan 2014-2018]. Retrieved from: http://www.resmigazete.gov.tr/eskiler/2014/07/20140716-8-1.pdf
• MoNE (2018). 2023 Eğitim Vizyon Belgesi [2023 Education Vision Document]. Retrieved from: http://2023vizyonu.meb.gov.tr/
• Nartgün, Z. (2002). Aynı Tutumu Ölçmeye Yönelik Likert Tipi Ölçek ile Metrik Ölçeğin Madde ve Ölçek Özelliklerinin Klasik Test Kuramı ve Örtük Özellikler Kuramına Göre İncelenmesi [The investigation of item and scale properties of likert type scale and metric scale measuring the same attitude according to classisical test theory and item response theory] [Unpublished PhD Thesis]. Hacettepe University.
• Nunnally, J.C. (1978). Psychometric theory. 2nd Edition, McGraw-Hill, New York.
• Ostini, R., & Nering, M.L. (Eds.). (2006). Polytomous item response theory models. Sage Publication.
• Özdamar, K. (2013). Paket Programlar ile İstatistiksel Veri Analizi [Statistical Data Analysis with Package Programs] (9. Edition). Nisan Kitabevi.
• Özdemir, H.F., Toraman, Ç., & Kutlu, Ö. (2019). The use of polychoric and Pearson correlation matrices in the determination of construct validity of Likert type scales. Turkish Journal of Education, 8(1), 180-195. https://doi.org/10.19128/turje.519235
• Reise, S.P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27(2), 133 144. https://doi.org/10.1111/j.1745-3984.1990.tb00738.x
• Rutter, M. (1985). Resilience in the face of adversity. Protective factors and resistance to psychiatric disorder. British Journal of Psychiatry, 147(6), 598 611. https://doi.org/10.1192/bjp.147.6.598
• Rutter, M. (2012). Resilience as a Dynamic Concept. Development and Psychopathology, 24 335-344. https://doi.org/10.1017/S0954579412000028
• Sala, A., Punie, Y., Garkov, V., & Cabrera Giraldez, M. (2020). LifeComp: The European Framework for Personal, Social and Learning to Learn Key Competence. Luxembourg: Publications Office of the European Union. https://data.europa.eu/doi/10.2760/302967
• Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34(4), 5 17. https://www.psychometricsociety.org/sites/main/files/file-attachments/mn17.pdf
• Samejima, F. (1996). Polychotomous responses and the test score. Tennessee: The University of Tennessee.
• Savaşır, I. (1994). Ölçek Uyarlamasındaki Sorunlar ve Bazı Çözüm Yolları [Problems in Scale Adaptation and Some Solutions]. Turkish Journal of Psychology, 9(33), 27-32. https://www.psikolog.org.tr/tr/yayinlar/dergiler/1031828/tpd1300443319940000m000295.pdf
• Scott, D., & Usher, R. (2011). Researching education: Data, methods and theory in educational enquiry. Continuum.
• Sümer, N. (2000). Yapısal Eşitlik Modelleri: Temel Kavramlar ve Örnek Uygulamalar [Structural Equation Models: Basic Concepts and Applications]. Turkish Psychological Articles, 3(6), 49-73. https://www.psikolog.org.tr/tr/yayinlar/dergiler/1031828/tpy1301996120000000m000225.pdf
• Şimşek, Ö.F. (2007). Yapısal Eşitlik Modellemesine Giriş: Temel İlkeler ve LISREL Uygulamaları [Introduction to Structural Equation Modeling: Fundamentals and Applications of LISREL]. Ekinoks Yayınları.
• The Assessment Reform Group. (2010, December). Retrieved from https://www.nuffieldfoundation.org/project/the-assessment-reform-group
• Thissen, D. (1991). Multilog user's guide. Scientific Software
• Tunca, N., Alkın Şahin, S., & Aydın, Ö. (2015). Öğretmen adaylarının yaşam boyu öğrenme eğilimleri. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 11(2), 432 446. https://doi.org/10.17860/efd.92694
• Uyar, Ş., Öztürk Gübeş, N., & Kelecioğlu H. (2013). PISA 2009 Tutum Anketi Madde Puanlarının Aşamalı Tepki Modeli ile İncelenmesi [Examinig the Pisa 2009 Attitude Questionnaire Item Scores with Using Graded Response Model]. Journal of Research in Education and Teaching, 2(4), 125 134. http://www.jret.org/FileUpload/ks281142/File/15.uyar.pdf
• Uysal, M. (2015). Araştırma özyeterlik ölçeğinin psikometrik özelliklerinin klasik test kuramı ve madde tepki kuramına göre incelenmesi [An investigation of psychometric properties of research self-efficacy scale according to Classical Test Theory and Item Response Theory] [Unpublished Master Thesis]. Gazi University.
• Vieira, A.L. (2011). Preparation of the analysis. Interactive LISREL in practice (First Edition). Springer.