Pamukkale eleştirel düşünme becerisi ölçeği: geçerlik ve güvenirlik çalışması

Öz

Bu çalışmanın amacı, üniversite öğrencilerinin eleştirel düşünme becerilerini ölçen kanıt temelli, geçerlik ve güvenilirliği yüksek bir ölçme aracı geliştirmektir. Pamukkale Eleştirel Düşünme Becerileri Ölçeği, çoktan seçmeli ve açık uçlu olmak üzere iki ayrı form olarak geliştirilmiştir. Çoktan seçmeli formla ilgili geçerlik güvenirlik çalışmaları klasik test kuramı ve madde-tepki kuramı olmak üzere iki farklı kuramsal çerçeve üzerine temellendirilmiştir. Klasik test kuramına göre ölçeğin çoktan seçmeli formunun yapı geçerliğini test etmek için açımlayıcı ve doğrulayıcı faktör analizi yapılmıştır. Analiz sonuçları ölçeğin tek boyutlu yapısını desteklemiştir. Ölçeğin güvenirlik çalışmaları kapsamında yapılan analizler, ölçeğin iç tutarlılık katsayısının .92 olduğunu, madde-toplam korelasyon değerlerinin .37 ile .84 arasında değiştiğini, eleştirel düşünme düzeyi düşük ve yüksek grupları anlamlı olarak ayırt edebildiğini göstermiştir. Test-tekrar test analizi sonuçları, ölçeğin ölçtüğü davranış alanına ilişkin zaman içinde kararlılık gösterdiğini destekler niteliktedir. Madde Tepki Kuramı temelli analiz sonuçları da ölçeğin, madde model uyumu varsayımlarını karşıladığını göstermiştir. Ölçeğin açık uçlu formunun değerlendirilmesinde ise araştırmacı/lar tarafından geliştirilen “Dereceli Puanlama Anahtarı” kullanılmıştır. Açık uçlu formun geçerlik ve güvenirliğiyle ilgili de bir dizi çalışma yapılmış, analiz sonuçları ölçeğin geçerliği ve güvenirliğine yönelik psikometrik destek sağlamıştır. Sonuç olarak, iki form halinde geliştirilen Pamukkale Eleştirel Düşünme Becerileri Ölçeği üniversite öğrencilerinin eleştirel düşünme becerilerini ölçmede kullanılabilecek geçerli ve güvenilir bir ölçme aracıdır. Bulgular alanyazın ışığında tartışılmış ve bazı öneriler verilmiştir.

Anahtar Kelimeler

• Critical thinking
• Test development
• University students
• Validity and reliability

Pamukkale critical thinking skill scale: a validity and reliability study

Abstract

The aim of this study is to develop a valid and reliable measurement tool that measures critical thinking skills of university students. Pamukkale Critical Thinking Skills Scale was developed as two separate forms; multiple choice and open-ended. The validity and reliability studies of the multiple-choice form were constructed on two different theoretical frameworks as classical test theory and item-response theory. According to classical test theory, exploratory and confirmatory factor analyses were performed, to item-response theory, the Generalized Partial Credit Model (GPCM) for one-dimensional and multi-category scales was tested for the construct validity of the multiple-choice form of the scale. Analysis results supported the unidimensional structure of the scale. The reliability analyzes showed that the internal consistency coefficient of the scale and the item-total correlation values were high enough. The test-retest analysis results supported that the scale shows stability over time regarding the field it measures. The results of the item-response theory-based analysis also showed that the scale met the item-model fit assumptions. In the evaluation of the open-ended form of the scale, a rubric was used. Several studies were conducted on the validity and reliability of the open-ended form, and the results of the analysis provided psychometric support for the validity and reliability. As a result, Pamukkale Critical Thinking Skills Scale, which was developed in two forms, is a valid and reliable measurement tool to measure critical thinking skills of university students. The findings were discussed in the light of the literature and some suggestions were given.

Keywords

• Critical thinking
• Test development
• University students
• Validity and reliability

References

1. Abrami, P.C., Bernard, R.M., Borokhovski, E., Wade, A., Surkes, M. ., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102-1134.
2. Anderson, L.W., & Krathwohl, D.R. (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives: Complete Edition. Longman.
3. Atay, S., Ekim, E., Gökkaya, S., & Sağım, E. (2009). Sağlık Yüksekokulu öğrencilerinin eleştirel düşünme düzeyleri. [Critical thinking tendencies of Health School students] Sağlık Bilimleri Fakültesi Hemşirelik Dergisi, 39-46.
4. Ayberk, B., & Çelik, M. (2007). Watson-Glaser Eleştirel, Akıl Yürütme Gücü Ölçeği'nin (W-GEAYGÖ) üniversite ikinci, üçüncü ve dördüncü sınıf İngilizce bölümü öğretmen adayları üzerindeki güvenlik çalışması. [Reliability study related to the power of Watson-Glaser critical thinking appraisal scale on university second, third and fourth-grade English department teacher candıdates] Çukurova Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 16 (1), 101-112
5. Aybek, E.C. (2021). CatIRT tools: A “Shiny” application for item response theory calibration and computerized adaptive testing simulation. Journal of Applied Testing Technology, 22(1), 23-27.
6. Baker, F.B. (2001). The basics of item response theory. College Park, ERIC, Clearinghouse on Assessment and Evaluation.
7. Batur, Z., & Özcan, H.Z. (2020). Eleştirel düşünme üzerine yazılan lisansüstü tezlerinin bibliyometrik analizi. [Bibliometric analysis of graduate theses written on critical thinking] Uluslararası Türkçe Edebiyat Kültür Eğitim Dergisi, 9(2), 834-854.
8. Bailey, R., & Mentz, E. (2015). IT teachers’ experience of teaching–learning strategies to promote critical thinking. Issues in Informing Science and Information Technology, 12(1), 141–152.

9. Bensley, D.A., Crowe, D.S., Bernhardt, P., Buckner, C., & Allman, A.L. (2010). Teaching and assessing critical thinking skills for argument analysis in psychology. Teaching of Psychology, 37(2), 91–96. https://doi.org/10.1080/00986281003626656
10. Bennett, D.A. (2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25(5), 464–469.
11. Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238
12. Bilican, S.D. (2021). Başarı testlerinin geliştirilmesi ve madde yazımı [Development of achievement tests and item writing] Yıldırım, Ö. ve Kartal, S.K. (Ed.), Eğitimde Ölçme ve Değerlendirme [Measurement and Evaluation in Education] (125-163) 1. Baskı, Lisans Yayıncılık.
13. Browne, N., & Freeman, K. (2000). Distinguishing features of critical thinking classrooms. Teaching in Higher Education. 5(3), 301-309. https://doi.org/10.1080/713699143
14. Boyd, E.M., & Fales, A.W. (1983). Reflective learning: Key to learning from experience. Journal of Humanistic Psychology, 23(2), 99 117. https://doi.org/10.1177/0022167883232011
15. Browne, M.W., & Cudeck, R. (1993). Alternative ways of assessing model fit. K. A. Bollen and J.S. Long (Ed.), Testing structural equation models (pp. 136-162). Sage.
16. Carpendale, J.I., & Lewis, C. (2006). How children develop social understanding, Blackwell.
17. Chalmers, R.P. (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1 29. https://doi.org/10.18637/jss.v048.i06
18. Cisneros R .M. (2009). Assessment of critical thinking in pharmacy students. American Journal of Pharmaceutical Education, 73(4), 66. https://doi.org/10.5688/aj730466
19. Cohen, R.J., Swerdlik, M.E., Smith, D.K., & Cohen, R.J. (1992). Psychological testing and assessment: An introduction to tests and measurement. Mayfield Pub. Co.
20. Comfort, L.K. (2007). Crisis management in hindsight: cognition, communication, coordination, and control. Public Administration Review, 67(1), 189–197.
21. Crockett, L. (2019). Future-focused Learning: 10 essential shifts of everyday practice. Solution Tree Press.
22. Dewey, J. (1933). How we think. DC Herman.
23. Doğan, N. (2013). Eleştirel düşünmenin ölçülmesi [Measuring the Critical Thinking]. Cito Eğitim: Kuram ve Uygulama, 22(1), 29-42.
24. Doğanay, A., Akbulut-Taş, M., & Erden, Ş. (2007). Üniversite öğrencilerinin bir güncel tartışmalı konu bağlamında eleştirel düşünme becerilerinin değerlendirilmesi. [Assessing university students’ critical thinking skills in the context of a current controversial issues]. Kuram ve Uygulamada Eğitim Yönetimi, 52(1), 511-546.
25. Dumitru, D., Bîgu, D., Elen, J., Jiang, L., Railiene, A., Penkauskiene, D., Papathanasiou, I.V., Tsaras, K., Fradelos, E.C., Ahern, A.K., McNally, C., O'Sullivan, J., Verburgh, A.P., Jarošová, E., Lorencová, H., Poce, A., Agrusti, F., Re, M.R., Puig, B., Blanco, P., Mosquera, I., Crujeiras-Pérez, B., Dominguez, C., Cruz, G., Silva, H., & Morais, M.D., Nascimento, M.M., & Payan-Carreira, R. (2018). A European review on Critical Thinking educational practices in Higher Education Institutions. http://hdl.handle.net/10197/9865
26. Duru, E., Duatepe-Paksu, A., Balkıs, M., Duru, S., & Bakay, E. (2020). Examination of 21st century competencies and skills of graduates from the perspective of sector representatives and academicians. Journal of Qualitative Research in Education, 8(4), 1059-1079. https://doi.org/10.14689/issn.2148-2624.8c.4s.1m
27. Dwyer, C.P., Hogan, M.J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12(1), 43 52. https://doi.org/10.1016/j.tsc.2013.12.004
28. Ebel, R.L. (1972). Essentials of educational measurement. Prentice-Hall.
29. Eğmir, E., & Ocak, G. (2016). Eleştirel düşünme becerisini ölçmeye yönelik bir başarı testi geliştirme. [Developing an achievement test towards evaluating critical thinking skill]. Turkish Studies, 11(19), 337-360.
30. Ennis, R. (1991). Critical thinking: A streamlined conception. Teaching Philosophy, 14(1), 15-24. http://dx.doi.org/10.5840/teachphil19911412
31. Ennis, R. H., & Millman, J. (1985). Cornell Critical Thinking Test (Level X). Critical Thinking Press & Software.
32. Facione, P.A. (1990a). Critical thinking: A statement of expert consensus for purposes of educational assessment and ınstruction. executive summary: “The Delphi Report”. The California Academic Press.
33. Facione, P.A. (1990b). The California Critical Thinking Skills Test-College Level Technical Report 1: Experimental Validation and Content Validity. The California Academic Press.
34. Facione, P.A. (1990c). The California Critical Thinking Skills Test-College Level Technical Report 2: Factors Predictive of Critical Thinking Skills. The California Academic Press.
35. Facione, N.C. (1997) Critical thinking assessment in nursing education programs An aggregate data analysis. The California Academic Press.
36. Facione, P.A. (2015). Critical thinking: What it is and why it counts. http://www.student.uwa.edu.au/__data/assets/pdf_file/0003/1922502/Critical-Thinking-What-it-is-and-why-it-counts.pdf
37. Facione, P.A., & Facione N.C. (1992). The california critical thinking dispositions inventory. The California Academic Press.
38. Fayers, P., & Machin, D. (1995). Factor analysis for assessing validity. Quality of Life Research, 4(5), 424.
39. Flores, K., Matkin, G.S., Burbach, M.E., Quinn, C.E., & Harding, H. (2012). Deficient critical thinking skills among college graduates: ımplications for leadership. Educational Philosophy and Theory, 44(2), 212 230. https://doi.org/10.1111/j.1469 5812.2010.00672.x
40. Güçlü, G., & Evcili, F. (2021). Sağlık hizmetleri meslek yüksekokulu öğrencilerinin eleştirel düşünme yetileri ve boyun eğici davranış eğilimlerinin incelenmesi. [Investigation of critical thinking qualifications and submissive behavior tendency of health services vocational school students]. Turkish Journal of Science and Health. 2(1), 31-39.
41. Haladyna, T.M. (1997). Writing test ıtem to evaluate higher order thinking. Allyn & Bacon.
42. Hambleton, R.K., & Swaminathan, H. (1985). Item response theory. Kluwer-Nijhoff Publishing.
43. Huber, C.R., & Kuncel, N.R. (2016). Does college teach critical thinking? A meta-analysis. Review of Educational Research, 86(2), 431 468. https://doi.org/10.3102/0034654315605917
44. Jacobs, S.S. (1995). Technical characteristics and some correlates of the California Critical Thinking Skills Test, Forms A and B. Research in Higher Education, 36(1), 89-108. https://doi.org/10.1007/BF0220776
45. Irani, T., Rudd, R., Gallo, M., Ricke, J., Friedel, C., & Rhoades, E. (2007) Critical thinking ınstrumentation manual. University of Florida.
46. Lipman, M. (1988) Critical thinking—What can it be? Educational Leadership, 461(1), 38–43.
47. Marais, I. (2009). Response dependence and the measurement of change. Journal of Applied Measurement, 10(1), 17-29.
48. Marin, L., & Halpern, D. (2011). Pedagogy for developing critical thinking in adolescents: Explicit instruction produces greatest gains. Thinking Skills and Creativity, 6(1), 1–13.
49. Mazer, J.P., Hunt, S.K., & Kuznekoff, J.H. (2007). Revising general education: Assessing a critical thinking ınstructional model in the basic communication course. The Journal of General Education 56(3), 173-199. https://doi.org/10.1353/jge.0.0000
50. Mpofu, N., & Maphalala, M.C. (2017). Fostering critical thinking in initial teacher education curriculums: A comprehensive literature review. Gender and Behaviour, 15(2), 9342–9351.
51. Msila, V. (2014). Critical Thinking in open and distance learning programmes: Lessons from the University of South Africa’s NPDE Programme. Journal of Social Sciences, 38(1), 33–42
52. Nalçacı, A., Meral, E., & Şahin, İ.F. (2016). Sosyal bilgiler öğretmen adaylarının eleştirel düşünme ile medya okuryazarlıkları arasındaki ilişki [Correlation between critical thinking and media literacy of social sciences pre-service teachers]. Doğu Coğrafya Dergisi, 21(36), 1-12. https://doi.org/10.17295/dcd.99051
53. Norris, S.P., & Ennis, R.H. (1990). The practitioners' guide to teaching thinking series. Evaluating Critical Thinking. Hawker Bronlow Education.
54. Özmen, K.S. (2008). İngilizce öğretmeni eğitiminde eleştirel düşünce: Bir vaka çalışması. [Critical Thinking in English teacher education: A case study]. Ekev Akademi Dergisi, 12(36), 253-266.
55. Orhan, A., & Çeviker-Ay, Ş. (2022). Developing the critical thinking skill test for high school students: A validity and reliability study. International Journal of Psychology and Educational Studies, 9(1), 132-144. https://dx.doi.org/10.52380/ijpes.2022.9.1.561
56. Parkhurst, H.B. (1999). Confusion, lack of consensus, and the definition of creativity as a construct. Journal of Creative Behavior, 33(1), 1–21.
57. Paul, R. (2005) The state of critical thinking today, New Directions for Community Colleges, 130(1), 27–38.
58. Paul, R., & Elder, L. (2001) Critical thinking: Inert information, activated ignorance, and activated knowledge, Journal of Developmental Education, 25(2), 36–37.
59. Paul, R.W., & Elder, L. (2002). Critical thinking: Tools for taking charge of your professional and personal life. Pearson Education Inc.
60. Paul, R., & Nosich, G. (1991). A proposal for the national assessment of higher-order thinking. Paper commissioned by the U.S. Department of Education Office of Educational Research and Improving National Center for Education Statistics.
61. Pascarella, E.T., & Terenzini, P.T. (1991). How college affects students: Findings and insights from twenty years of research. Jossey-Bass.
62. Pascarella, E.T., & Terenzini, P.T. (2005). How college affects students: A third decade of research. Jossey-Bass.
63. Portney, L.G., & Watkins, M.P. (2000) Foundations of clinical research: Applications to practice. 2nd Edition, Prentice Hall.
64. Puig, B., Blanco-Anaya, P., Bargiela, I.M., & Crujeiras-Pérez, B. (2019). A systematic review on critical thinking intervention studies in higher education across professional fields. Studies in Higher Education, 44(5), 860-869.
65. R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.
66. Rezaee, M., Farahian, M., & Ahmadi, A. (2012). Critical thinking in higher education: Unfulfilled expectations. BRAIN. Broad Research in Artificial Intelligence and Neuroscience, 3(2), 64-73.
67. Ruggiero, V.R. (1990) Beyond feelings. A guide to critical thinking, (3rd ed.) Mayfield Publishing.
68. Røe, C., Damsgård, E., Fors, T., & Anke, A. (2014). Psychometric properties of the pain stages of change questionnaire as evaluated by Rasch analysis in patients with chronic musculoskeletal pain. BMC Musculoskelet Disord, 15(1), 95. https://dx.doi.org/10.1186/1471-2474-15-95, PubMed 24646065
69. Sahool, S., & Mohammed, C.A. (2018). Fostering critical thinking and collaborative learning skills among medical students through a research protocol writing activity in the curriculum. Korean J Med Educ, 30(2); 109-118. https://doi.org/10.3946/kjme.2018.86
70. Schafer, J.L. (1999). Multiple imputation: a primer. Stat Methods in Med., 8(1), 3–15. https://doi.org/10.1191/096228099671525676
71. Shipman, V. (1983). New Jersey test of reasoning skills. IAPC, Test Division, Montclair State College.
72. Siegel, H. (1988). Educating for reason: Rationality, critical thinking, and education. Routledge.
73. Snyder, L.G., & Snyder, M.J. (2008). Teaching critical thinking and problem solving skills. The Journal of Research in Business Education, 50 (2), 90–99.
74. Stevens, J. (1992). Applied multivariate statistics for the social sciences. Second Edition, Lawrence Erlbaum Associates.
75. Tabachnick, B.G., & Fidell, L.S. (2013). Using multivariate statistics (6th ed.), Allyn and Bacon.
76. Tolman, E.C. (1932). Purposive behavior in animals and men. Century/Random House UK.
77. Trilling, B., & Fadel, C. (2009). 21st-century skills: Learning for life in our times. John Wiley & Sons.
78. Uzuntiryaki- Kondakçı, E., & Çapa-Aydın, Y. (2013). Predicting critical thinking skills of university students through metacognitive self-regulation skills and chemistry self-efficacy. Educational Sciences: Theory & Practice, 13(1), 666-670.
79. Van Laar, E., van Deursen, A.J.A. M., van Dijk, J.A.G.M., & de Haan, J. (2019). Determinants of 21st-century digital skills: A large-scale survey among working professionals. Computers in Human Behavior, 100, 93–104. https://doi.org/10.1016/j.chb.2019.06.017
80. Voogt, J., & Roblin, N.P. (2012). A comparative analysis of international frameworks for 21st-century competencies: implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299–321.
81. Watson, G., & Glaser, E.M. (1980). Watson-Glaser critical thinking appraisal. Psychological Corporation
82. Williams, R.L., Oliver, R., Allin, J.L., Winn, B., & Booher, C.S. (2003). Psychological critical thinking as a course predictor and outcome variable. Teaching of Psychology, 30(3), 220–223. https://doi.org/10.1207/S15328023TOP3003_04
83. Yen, W.M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30(1), 187-213.