Developing the Critical Thinking Skill Test for High School Students: A Validity and Reliability Study

Abstract

This research aimed to develop “Critical Thinking Skill Test for High School Students” to measure critical thinking (CT) skills of high school students. For the CT test prepared based on the sub-skills of inference, evaluating arguments, deduction, recognizing assumptions and interpretation which are deemed to represent CT (Watson and Glaser, 1994), content validity and face validity were achieved with expert opinions and the table of specifications. Following the item difficulty and item discrimination analyses performed to test the construct validity, 34 items were omitted from the test which was finalized with 51 items. While mean item difficulty values of the sub-tests vary between 0.51 and 0.63, mean item discrimination values range from 0.35 to 0.49. The total test has a mean item difficulty value of 0.52 and a mean item discrimination value of 0.42. Decision-making skill was used to test the criterion-related validity of the test. KR20 reliability coefficients calculated for the sub-tests ranged from 0.62 to 0.75. A KR20 reliability coefficient of the total test was 0.87. Moreover, a correlation coefficient of 0.84 was calculated with the split-half method. To test time invariance of the test, the correlation values calculated between the results of the two applications which were performed three months apart ranged from 0.57 to 0.70. The correlation coefficient for the total test is 0.70. Based on the results of validity and reliability studies, it can be said that the CT test will yield valid and reliable results in measuring CT skills of high school students.

Keywords

• Critical thinking skills
• critical thinking test
• measuring critical thinking skills
• high school students

References

1. Ay, Ş. &Akgöl, H. (2008). Critical thinking, gender, age and grade level. Journal of Theoretical Educational Science, 1(2), 65-75.
2. Aybek, B. &Çelik, M. (2007). Watson-Glaser eleştirel akıl yürütme gücü ölçeğinin (W-GEAYGÖ) üniversite ikinci, üçüncü ve dördüncü sınıf İngilizce bölümü öğretmen adayları üzerindeki güvenirlik çalışması. Ç.Ü. Sosyal Bilimler Enstitüsü Dergisi, 16(1), 101-112.
3. Aybek, B. (2006). The Effect of content and skill based critical thinking teaching on prospective teachers'disposition and level in critical thinking[Doctoral dissertation]. Çukurova University, Adana.
4. Bailin, S. (1998). Critical thinking and drama education. Research in Drama Education: The Journal of Applied Theatre and Performance, 3(2), 145-153.
5. Büyüköztürk, Ş., Çakmak, E. K., Akgün, Ö. E., Karadeniz, Ş.& Demirel, F. (2014). Bilimsel araştırma yöntemleri (16. edition). Pegem.
6. Chance, P. (1986). Thinking in the classroom: A survey of programs. Teachers College, Columbia University.
7. Crocker, L. &Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart and Winston.
8. Cronbach, L. J. (1984). Essentials for psychological testing. Harper & Row.

9. Çıkrıkçı, N. (1993). Watson-Glaser eleştirel akıl yürütme gücü ölçeğinin (form Y M) lise öğrencileri üzerindeki ön deneme uygulaması. Ankara University Journal of Educational Sciences, 25(2), 559-569.
10. Çıkrıkçı, N. (1996, June). Eleştirel düşünme: Bir ölçme aracı ve bir araştırma. III. Ulusal Psikolojik Danışma ve Rehberlik Kongresi, Adana.
11. Çokluk, Ö., Şekercioğlu, G. &Büyüköztürk, Ş. (2012). Sosyal bilimler için çok değişkenli istatistik SPSS ve LISREL uygulamaları (2.edition). Pegem.
12. Çolakkadıoğlu, O. (2012). The reliability and validity study of adolescent decision making questionnaire for the high school students. Mustafa Kemal University Journal of Social Sciences Institute, 9(19), 387-403.
13. Demir, M. K. (2006). The research of fourth and fifth grade primary school students’ critical thinking levels in social studies lessons according to different variables. Journal of Gazi Educational Faculty, 26(3), 155-169.
14. Eğmir, E. &Ocak, G. (2017). Eleştirel düşünme öğretim programının öğrencilerin eleştirel düşünme becerisi ve özdeğerlendirme düzeylerine etkisi. Karaelmas Journal of Educational Sciences, 5, 138-156.
15. Ennis, R. H. &Weir, E. (1985). The Ennis-Weir critical thinking essay test. Pacific Grove, CA: Critical Thinking Press and software.
16. Ennis, R. H. (1985). A logical basis for measuring critical thinking skills. Educational Leadership, 43(2), 44-48.
17. Ennis, R. H., Millman, J. &Tomko, T. N. (2005). Cornell critical thinking tests Level X and Level Z manual (5th edition). The Critical Thinking Co.
18. Epstein, R. L. &Kernberger, C. (2012). Critical thinking. Advanced Reasoning Forum.
19. Evcen, D. (2002). Adaptation of Watson-Glaser critical thinking apprasial test (form s) to [Unpublished master thesis]. Ankara University, Ankara.
20. Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction - executive summary - the Delphi report. The California Academic Press.
21. Facione, P. A. (2000). The disposition toward critical thinking: Its character, measurement and relationship to critical thinking skill. Journal of Informal Logic, 20(1), 61-84.
22. Fisher, A. (2001). Critical thinking: An introduction. Cambridge University Press.
23. Floyd, F. J. &Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286-299.
24. Gorsuch, R. L. (2014). Factor analysis. Routledge.
25. Gülveren, H. (2007). Critical thinking skills of education faculty students and factors influencing critical thinking skills [Doctoral dissertation]. Dokuz Eylül University, İzmir.
26. Halpern, D. (2003). Thought & knowledge: An introduction to critical thinking. Lawrence.
27. Jonassen, D. H. (2000). Computers as mindtools for schools: Engaging critical thinking. Prentice Hall.
28. Jones, E., Hoffman, S., Moore, L., Ratcliff, G., Tibbetts, S. &Click, B. (1995). National assessment of college student learning: Identifying the college graduates’ essential skills in writing, speech and listening and critical thinking. National Center for Educational Statistics.
29. Judge, B., Jones, P. &McCreery, E. (2009). Critical thinking skills for education students. Learning Matters.
30. Kan, A. (2011). Ölçme aracı geliştirme. S. Tekindal (Ed.). Eğitimde ölçme ve değerlendirme içinde (p. 239-276). Pegem.
31. Kennedy, M., Fisher, M. B. &Ennis, R. H. (1991). Critical thinking: Literature review and needed research. In L. Idol & B. Fly Jones (Eds.), Educational values and cognitive instruction: Implications for reform (pp. 11-40). Lawrence Erlbaum.
32. Kline, R. B. (2010). Principles and practice of structural equation modeling. Guilford Press.
33. Koç, C. (2007). The effects of active learning on reading comprehension, critical thinking and classroom interaction [Doctoral dissertation]. Dokuz Eylül University, İzmir.
34. Kurnaz, A. (2007). Effects of skill and content-based critical thinking training on students' critical thinking skills, achievement and attitudes in the fifth grade course of social knowledge of primary school [Doctoral dissertation]. Selçuk University, Konya.
35. Lewis, A. &Smith, D. (1993). Defining higher order thinking. Theory into Practice, 32(3), 131-137.
36. Mason, M. (2008). Critical thinking and learning. Blackwell Publishing.
37. Masterman, L. (1985). Teaching the media. Comedia Publishing Group.
38. Mecit, Ö. (2006). The effect of 7E learning cycle model on the improvement of fifth grade students’ critical thinking skills [Doctoral dissertation]. Middle East Technical University, Ankara.
39. Norris, S. P. &Ennis, R. H. (1989). Evaluating critical thinking. Critical Thinking Press and Software.
40. Open Society Institute. (2021). Media literacy index. Access link: https://osis.bg/?p=3750&lang=en.
41. Özçelik, D. A. (2013). Test hazırlama kılavuzu. Pegem.
42. Pascarella, E. T. &Terenzini, P. T. (1991). How college affects students: Findings and insights from twenty years of research. Jossey-Bass.
43. Paul, R., Binker, A. J. A., Jensen, K. &Kreklau, H. (1990). Critical thinking handbook: 4th.-6th grades a guide for remodellingg lesson plans in langıage, arts, social studies & science. Foundation for Critical Thinking, Sonoma State University.
44. Pérez Tornero, J. M. &Varis, T. (2010). Media literacy and new humanism. Moscow: UNESCO Institute for Information Technologies in Education.
45. Presseisen, B. Z. (1985). Thinking skills throughout the K-12 curriculum: A conceptual design. Philadelphia: Research for Better Schools.
46. Shipman, V. (1983). New Jersey test of reasoning skills. Upper Montclair NJ: IAPC, Test Division, Montclair State College.
47. Stevens, J. P. (2009). Applied multivariate statistics for the social sciences. Routledge.
48. Swartz, R. J. &Parks, D. (1994). Infusing the teaching of critical and creative thinking in elementary instruction. Critical Thinking Books & Software.
49. Şenturan, L. (2006). Critical thinking in nursing students [Doctoral dissertation]. Marmara University, İstanbul.
50. Tabachnick, B. G. &L. S. Fidell. (2012). Using multivariate statistics (6. edition). Pearson.
51. Tekin, H. (2019). Eğitimde ölçme ve değerlendirme (27. edition). Yargı Yayınları.
52. Terzi, R. (2019). Nicel veri toplama teknikleri. S. Şen & İ. Yıldırım (Eds.), Eğitimde araştırma yöntemleri içinde (p.357-382). Nobel.
53. Tufan, D. (2008). Critical thinking skills of prospective teachers: foreign language education case at the Middle East Technical University [Unpublished master thesis]. Middle East Technical University, Ankara.
54. Turgut M. F. (1995). Eğitimde ölçme ve değerlendirme metodları. Yargıcı Matbaası.
55. Wagner, T. (2010). The global achievement gap. Basic Books.
56. Watson, G. & Glaser, M. E. (1964). Watson-Glaser critical thinking appraisal manual. Harcourt, Brace & World Inc.
57. Watson, G. &Glaser, M. E. (1994). Watson-Glaser critical thinking appraisal form S manual. San Antonio: The Psychological Corporation.
58. We Are Social. (2020). Digital 2020. Access link: https://wearesocial.com/digital-2020.
59. Wood, D. (1998). Understanding children's worlds. How children think and learn: The social contexts of cognitive development (2. edition). Blackwell Publishing.
60. Yıldırım, B. (2010). The effect of skill based critical thinking education on the development of critical thinking in nurse students [Doctoral dissertation]. Ege University, İzmir.