A Comperative Adaptation of the Crick Learning for Resilient Agency (CLARA) with Classical Test Theory and Item Response Theory

Abstract

The aim of this study is to adapt the Crick Learning for Resilient Agency (CLARA) to Turkish culture, and to examine the psychometric features of the Inventory according to both Classical Test Theory (CTT) and Item Response Theory (IRT). In this respect, it is a descriptive level survey design research. Two different study groups were formed in accordance with the purpose of the study. Lingual equivalence applications were performed on two separate groups, one of which consisted of English Language and Literature Department students and the other consisted of English Language instructors. 1054 students participated in the validity and reliability studies from 101 different undergraduate programs at Ankara University. Before testing the research questions, it was examined whether the assumptions of CTT and IRT were met. With the application data; the predicted item discrimination indices, ability levels, students’ scores forming their learning power profiles, and reliability coefficient values were found to be similar in both theories. It can be said that with CLARA-Tr, obtained by adapting CLARA, a valid and reliable tool has been provided to the Turkish literature to be used in future studies.

Keywords

• Learning power
• Resilient agency
• Classical Test Theory
• Item Response Theory
• Intercultural test adaptation

References

1. Arthur, James & Crick, Ruth & Samuel, Elspeth & Wilson, Kenneth & Mcgettrick, Bart. (2006). Character Education: The Formation of Virtues and Dispositions in 16-19 Year Olds with particular reference to the religious and spiritual. West Conshohocken, PA: John Templeton Foundation. Retrieved from: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.473.7540&rep=rep1&type=pdf
2. Broadfoot, P. (1998). Records of Achievement and the Learning Society: a tale of two discourses. Assessment in Education, 5(3), 447 477. https://doi.org/10.1080/0969595980050307
3. Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö.E., Karadeniz, Ş., & Demirel, F. (2014). Bilimsel Araştırma Yöntemleri [Scientific Research Methods]. Pegem Akademi.
4. Carr, M., & Claxton, G. (2002). Tracking the development of learning dispositions. Assessment in Education, 9 (1), pp. 9-37. https://doi.org/10.1080/09695940220119148
5. Claxton, G. (1999). Wise Up: The Challenge of Lifelong Learning. Bloomsbury.
6. Cohen, R.J., & Swerdlik, M.E. (2010). Psychological testing and assessment. McGrawHill Companies.
7. Crick, R., & Bentley, J. (2020). Becoming a resilient organisation: integrating people and practice in infrastructure services. International Journal of Sustainable Engineering, 13(6), 423-440. https://doi.org/10.1080/19397038.2020.1750738
8. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Harcourt, New York.

9. Cronbach, L.J. (1960). Essentials of psychological testing. Harper Collins Publishers.
10. Deakin Crick, R. (2005). Learning Power: Dynamic Assessment for Learning. The Leader: Secondary Heads Association.
11. Deakin Crick, R. (2006). Learning power in practice: A guide for teachers. Paul Chapman.
12. Deakin Crick, R. (2007). Learning how to learn: the dynamic assessment of learning power. Curriculum Journal, 18(2), 135-153. https://doi.org/10.1080/09585170701445947
13. Deakin Crick, R. (2009a). Assessment in Schools - Dispositions. In: McGaw, B., Peterson, P., & Baker, E. (eds.) The International Encyclopedia of Education (3rd Edition). Elsevier.
14. Deakin Crick, R. (2009b). Inquiry-based learning: reconciling the personal with the public in a democratic and archaeological pedagogy. Curriculum Journal, 20(1), 73-92. https://doi.org/10.1080/09585170902764021
15. Deakin Crick, R., Broadfoot, P., & Claxton, G. (2004). Developing an effective lifelong learning inventory: The ELLI Project. Assessment in Education, 11(3), 248-272. https://doi.org/10.1080/0969594042000304582
16. Deakin Crick, R., & McCombs, B. (2006). The Assessment of Learner Centered Principles: An English Case Study. Educational Research and Evaluation, 12(5), 423-444. https://doi.org/10.1080/13803610600697021
17. Deakin Crick, R., & Salway, A. (2006). Locked Up Learning: learning power and young offenders. ViTaL Partnerships Ltd.
18. Deakin Crick, R., & Wilson, K. (2005). Being a Learner: A Virtue for the 21st Century. British Journal of Educational Studies, 53(5), 359-374. https://doi.org/10.1111/j.1467-8527.2005.00300.x
19. Deakin Crick, R., & Yu, G. (2008). Assessing learning dispositions: is the effective lifelong learning inventory valid and reliable as a measurement tool? Educational Research, 50(4), 387-402. https://doi.org/10.1080/00131880802499886
20. Deakin Crick, R., Haigney, D., Huang, S., Coburn, T., & Goldspink, C. (2013). Learning power in the workplace: the effective lifelong learning inventory and its reliability and validity and implications for learning and development. The International Journal of Human Resource Management, 24(11), 2255 2272. https://doi.org/10.1080/09585192.2012.725075
21. Deakin Crick, R., Huang, S., Shafi, A.A., & Goldspink, C. (2015). Developing resilient agency in learning: The internal structure of learning power. British Journal of Educational Studies, 63(2), 121-160. https://doi.org/10.1080/00071005.2015.1006574
22. De Ayala, R.J. (2009). The theory and practice of item response theory. U.S.A.: Guilford Press.
23. Diker Coşkun, Y., & Demirel, M. (2012). Üniversite Öğrencilerinin Yaşam Boyu Öğrenme Eğilimleri [Lifelong Learning Dispositions of University Students]. Hacettepe University Journal of Education, 42, 108-120.
24. Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates, Inc.
25. Erkuş, A. (2013). Davranış Bilimleri İçin Bilimsel Araştırma Süreci [Scientific Research Process for Behavioral Sciences]. Seçkin Yayıncılık.
26. Flannery, W.P., Reise, S.P., & Widaman, F.K. (1995). An item response theory analysis of the general and academic scales of self-description questionnaire-II. Journal of Research in Personality, 29(1), 168-188.
27. Gornall, S., Chambers, M.R., & Claxton, G. (2005). Building Learning Power in Action. TLO Ltd. Retrieved from: https://www.buildinglearningpower.com/wp content/uploads/2014/11/BLPIA_A4_v1l-sample.pdf
28. Hambleton, R., & Swaminathan, R. (1985). Fundamentals of Item Response Theory. Sage Publishers.
29. Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. Sage.
30. Hambleton, R.K. & Patsula, L. (1999). Increasing the validity of adapted tests: Myths to be avoided and guidelines for improving test adaptation practices. Journal of Applied Testing Technology, 1(1), 1-12.
31. Hambleton, R.K., Merenda, P.F., & Spielberger, C.D. (2005). Adapting educational and psychological tests for cross-cultural assessment. Lawrence Erlbaum.
32. Harlen, W., & Deakin Crick, R. (2003a). A systematic review of the impact of summative assessment and testing on pupils' motivation for learning. Retrieved from: https://dspace.stir.ac.uk/bitstream/1893/19607/1/SysRevImpSummativeAssessment2002.pdf
33. Harlen, W., & Deakin Crick, R. (2003b). Testing and Motivation for Learning. Assessment in Education, 10(2), 169-207. https://doi.org/10.1080/0969594032000121270
34. Hu, L.T., & Bentler, P.M. (1999). Cut off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. http://dx.doi.org/10.1080/10705519909540118
35. International Test Commission, ITC. (2018). Guidelines for translating and adapting tests (Second edition). International Journal of Testing, 18(2), 101-134.
36. Jöreskog, K.G., & Sörbom, D. (1993). Lisrel 8: structural equation modeling with the simplis command language. Erlbaum Associates Publishers.
37. Karakılıç, M. (2009). Beden eğitimi dersi için hazırlanan tutum ölçeğinin psikometrik kuramlar açısından incelenmesi [Examination of the attitude scale prepared for the physical education lesson in terms of psychometric theories] [Unpublished Doctoral Thesis]. Ankara University.
38. Kelecioğlu, H. (2001). Örtük Özellikler Teorisindeki b ve a Parametreleri İle Klasik Test Teorisindeki p ve r İstatistikleri Arasındaki İlişki [The relationship between b ve a parameters in latent trait theory and p and r statistiscs in classical test theory]. Hacettepe University Journal of Education, 20(1), 104-110.
39. Kline, R.B. (2005). Principles and practice of structural equation modeling. The Guilford Press.
40. Koch, W.R. (1983). Likert scaling using the graded response latent trait model. Applied Psychological Measurement, 7(1), 15-32. https://doi.org/10.1177/014662168300700104
41. Masten, A. (2007). Resilience in developing systems: progress and promise as the fourth wave rises. Developmental Psychopathology, 19(3), 921 930. https://doi.org/10.1017/S0954579407000442
42. MoNE (2014). Türkiye hayat boyu öğrenme strateji belgesi ve eylem planı 2014-2018 [Türkiye Lifelong Learning Strategy Document and Action Plan 2014-2018]. Retrieved from: http://www.resmigazete.gov.tr/eskiler/2014/07/20140716-8-1.pdf
43. MoNE (2018). 2023 Eğitim Vizyon Belgesi [2023 Education Vision Document]. Retrieved from: http://2023vizyonu.meb.gov.tr/
44. Nartgün, Z. (2002). Aynı Tutumu Ölçmeye Yönelik Likert Tipi Ölçek ile Metrik Ölçeğin Madde ve Ölçek Özelliklerinin Klasik Test Kuramı ve Örtük Özellikler Kuramına Göre İncelenmesi [The investigation of item and scale properties of likert type scale and metric scale measuring the same attitude according to classisical test theory and item response theory] [Unpublished PhD Thesis]. Hacettepe University.
45. Nunnally, J.C. (1978). Psychometric theory. 2nd Edition, McGraw-Hill, New York.
46. Ostini, R., & Nering, M.L. (Eds.). (2006). Polytomous item response theory models. Sage Publication.
47. Özdamar, K. (2013). Paket Programlar ile İstatistiksel Veri Analizi [Statistical Data Analysis with Package Programs] (9. Edition). Nisan Kitabevi.
48. Özdemir, H.F., Toraman, Ç., & Kutlu, Ö. (2019). The use of polychoric and Pearson correlation matrices in the determination of construct validity of Likert type scales. Turkish Journal of Education, 8(1), 180-195. https://doi.org/10.19128/turje.519235
49. Reise, S.P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27(2), 133 144. https://doi.org/10.1111/j.1745-3984.1990.tb00738.x
50. Rutter, M. (1985). Resilience in the face of adversity. Protective factors and resistance to psychiatric disorder. British Journal of Psychiatry, 147(6), 598 611. https://doi.org/10.1192/bjp.147.6.598
51. Rutter, M. (2012). Resilience as a Dynamic Concept. Development and Psychopathology, 24 335-344. https://doi.org/10.1017/S0954579412000028
52. Sala, A., Punie, Y., Garkov, V., & Cabrera Giraldez, M. (2020). LifeComp: The European Framework for Personal, Social and Learning to Learn Key Competence. Luxembourg: Publications Office of the European Union. https://data.europa.eu/doi/10.2760/302967
53. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34(4), 5 17. https://www.psychometricsociety.org/sites/main/files/file-attachments/mn17.pdf
54. Samejima, F. (1996). Polychotomous responses and the test score. Tennessee: The University of Tennessee.
55. Savaşır, I. (1994). Ölçek Uyarlamasındaki Sorunlar ve Bazı Çözüm Yolları [Problems in Scale Adaptation and Some Solutions]. Turkish Journal of Psychology, 9(33), 27-32. https://www.psikolog.org.tr/tr/yayinlar/dergiler/1031828/tpd1300443319940000m000295.pdf
56. Scott, D., & Usher, R. (2011). Researching education: Data, methods and theory in educational enquiry. Continuum.
57. Sümer, N. (2000). Yapısal Eşitlik Modelleri: Temel Kavramlar ve Örnek Uygulamalar [Structural Equation Models: Basic Concepts and Applications]. Turkish Psychological Articles, 3(6), 49-73. https://www.psikolog.org.tr/tr/yayinlar/dergiler/1031828/tpy1301996120000000m000225.pdf
58. Şimşek, Ö.F. (2007). Yapısal Eşitlik Modellemesine Giriş: Temel İlkeler ve LISREL Uygulamaları [Introduction to Structural Equation Modeling: Fundamentals and Applications of LISREL]. Ekinoks Yayınları.
59. The Assessment Reform Group. (2010, December). Retrieved from https://www.nuffieldfoundation.org/project/the-assessment-reform-group
60. Thissen, D. (1991). Multilog user's guide. Scientific Software
61. Tunca, N., Alkın Şahin, S., & Aydın, Ö. (2015). Öğretmen adaylarının yaşam boyu öğrenme eğilimleri. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 11(2), 432 446. https://doi.org/10.17860/efd.92694
62. Uyar, Ş., Öztürk Gübeş, N., & Kelecioğlu H. (2013). PISA 2009 Tutum Anketi Madde Puanlarının Aşamalı Tepki Modeli ile İncelenmesi [Examinig the Pisa 2009 Attitude Questionnaire Item Scores with Using Graded Response Model]. Journal of Research in Education and Teaching, 2(4), 125 134. http://www.jret.org/FileUpload/ks281142/File/15.uyar.pdf
63. Uysal, M. (2015). Araştırma özyeterlik ölçeğinin psikometrik özelliklerinin klasik test kuramı ve madde tepki kuramına göre incelenmesi [An investigation of psychometric properties of research self-efficacy scale according to Classical Test Theory and Item Response Theory] [Unpublished Master Thesis]. Gazi University.
64. Vieira, A.L. (2011). Preparation of the analysis. Interactive LISREL in practice (First Edition). Springer.