Development of context and question evaluation scales for measuring higher order thinking skills

Öz

In measuring higher-order thinking skills, modern education emphasizes contexts requiring knowledge use and various skills like transfer, analytical solution, and critical thinking. Students must transfer knowledge across contexts, develop strategies, and analyze information analytically, creating different solutions. Quality contexts are clear, understandable, and class-appropriate, with interdependent questions that are also clear and context-appropriate. This research, conducted under the TÜBİTAK 1001 project 120K850,and derived from the first author’s PhD thesis, developed scales for experts to evaluate context and questions quantitatively. Principal components analysis confirmed both scales are unidimensional. The Cronbach's alpha value of the context quality assessment scale consisting of 6 items was .81, while it was observed that appropriate fit indices were reached in the confirmatory factor analysis; χ2/sd = 2.58, RMSEA = .09, SRMR = .05, CFI = .99, GFI = .97. The reliability value of the question quality assessment scale consisting of a total of 7 items was found to be 0.84, while the fit indices were also within acceptable limits; χ2/sd = 4.49, RMSEA = .08, SRMR = .03, CFI = .99, GFI = .96. A moderate positive correlation was found between the two scales; r = .57, p < .001.

Anahtar Kelimeler

• Higher Order Thinking Skills
• Education
• Context and Question Quality
• Scale

Üst düzey düşünme becerilerinin ölçümüne yönelik bağlam ve soru değerlendirme ölçeklerinin geliştirilmesi

Öz

Eğitimde üst düzey düşünme becerilerin ölçülmesinde ve değerlendirilmesinde modern eğitim anlayışı bağlamlara dayanmaktadır. Bu bağlamlar salt bilginin kullanılmasını değil transfer, analitik çözümleme, eleştirel düşünme gibi çeşitli üst düzey düşünme becerilerin de kullanılmasını gerektirmektedir. Öğrencilerin farklı bağlamlar arasında bilgi transferi yapması, bağlamdaki bilgiler üzerinde stratejiler geliştirerek analitik olarak çözümlemesi ve farklı çözüm alternatifleri geliştirebilmesi beklenmektedir. Üst düzey düşünme becerileri ölçen, açık, anlaşılır ve düzeye uygun bağlamlar kaliteli bağlamlar olarak değerlendirilirken, bu bağlam ile karşılıklı bağımlılık gösteren, bağlama uygun, açık ve anlaşılır sorular da kaliteli olarak değerlendirilmektedir. . Bu araştırma “Çevrimiçi Bilişsel Tanıya Dayalı İzleme Modelinin Üst Düzey Düşünme Becerilerine Etkisi” başlıklı 120K850 numaralı TÜBİTAK 1001 projesi kapsamında yürütülmüş ve ilk yazarın doktora tez çalışmasından türetilmiş olup, uzmanların niceliksel olarak bağlam ve sorularını değerlendirebileceği ölçekler geliştirilmiştir. Bağlam kalitesini değerlendirme ölçeğinin Cronbach alfa değeri .81 olarak belirlenirken doğrulayıcı faktör analizinde uygun uyum indekslerine ulaşıldığı gözlenmiştir; χ2/sd = 2.58, RMSEA = .09, SRMR = .05, CFI = .99, GFI = 0.97. soru kalitesini değerlendirme ölçeğinin de Alfa değerinin .84 olduğu tespit edilirken uyum indekslerinin kabul edilebilir sınırlarda olduğu görülmüştür; χ2/sd = 4.49, RMSEA = 0.08, SRMR = .03, CFI = .99, GFI = .96. İki ölçek arasında orta düzeyde pozitif yönlü bir ilişki tespit edilmiştir; r = .57, p < .001.

Anahtar Kelimeler

• Üst Düzey Düşünme Becerileri
• Eğitim
• Bağlam ve Soru Kalitesi
• Ölçek

Kaynakça

1. Alaimi, M., Law, E., Pantasdo, K. D., Oudeyer, P.-Y., & Sauzeon, H. (2020). Pedagogical agents for fostering question-asking skills in children. arXiv preprint arXiv. https://doi.org/10.48550/arXiv.2004.03472
2. Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn?: A taxonomy for far transfer. Psychological Bulletin, 128(4), 612-637. http://doi.org/10.1037/0033-2909.128.4.612
3. Baysal, E. A., & Ocak, G. (2022). University Students' Cognitive Bias in the Context of Their Analytical Thinking Skills: A Reliability and Validity Study. International Journal of Progressive Education, 18(3), 205–225. https://doi.org/10.29329/ijpe.2022.439.14
4. Beno, P., & Kovacova, B. (2020). Structured, Analytical and Critical Thinking in the Educational Process of Future Teachers. Acta Educationis Generalis, 10(3), 111–118. https://doi.org/10.2478/atd-2020-0024
5. Biggs, J., & Tang, C. (2011). Teaching for Quality Learning at University (Eds. 4). Open University Press.
6. Bloom, B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. David McKay Co Inc.
7. Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education, 24(1), 61-100.
8. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How People Learn: Brain, Mind, Experience, and School. National Academy Press.

9. Brown, A. L. (1987). Metacognition, executive control, self-regulation , and other more mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds.1), Metacognition, Motivation, and Understanding, 65-116.
10. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42. https://doi.org/10.3102/0013189X018001032
11. Butterfield, E. C., & Nelson, G. D. (1989). Theory and practice of teaching for transfer. Educational Technology Research and Development, 37(3), 5-38. http://doi.org/10.1007/BF02298274
12. Butterfield, E. C., Nelson, T. O., & Peck, V. (1988). Developmental aspects of the feeling of knowing. Developmental Psychology, 24(5), 654–663. http://doi.org/10.1037/0012-1649.24.5.654
13. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Educational Technology Research and Development, 37, 5-38. http://doi.org/10.1207/s15516709cog1302_1
14. Elballah, K., Alkhalifah, N., Alomari, A. & Alghamdi, A. (2024). Enhancing cognitive dimensions in gifted students through future problem-solving enrichment programs. Discover Sustainability, 5, 248. https://doi.org/10.1007/s43621-024-00470-5
15. Elder, L. & Paul, R. (2008). Critical thinking: the nuts and bolts of education. Optometric Education, 33(3), 88-91.
16. Ennis, R. H. (1996). Critical Thinking. Prentice Hall.
17. Facione, P. A. (1990). Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. The Delphi Report. http://doi.org/10.1037/0022-0663.91.2.260
18. Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906-911. https://doi.org/10.1037/0003-066X.34.10.906
19. Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52, 67-86. http://doi.org/10.1007/BF02504676
20. Halpern, D. F. (2013). Thought and Knowledge: An Introduction to Critical Thinking (5th ed.). Psychology Press.
21. Hasanah, S., & Retnawati, H. (2022). Assessment of contextual learning in mathematics education. AIP Conference Proceedings, 2575(1), 040018. https://doi.org/10.1063/5.0111142
22. Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. http://doi.org/10.3102/003465430298487
23. Herrington, J., & Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational Technology Research and Development, 4, 23-48. http://doi.org/10.1007/BF02319856
24. Hidayat, R., Nugroho, I., Zainuddin, Z., & Ingai, T. A. (2024). A systematic review of analytical thinking skills in STEM education settings. Information and Learning Sciences, 125(7/8), 565–586. https://doi.org/10.1108/ILS-06-2023-0070
25. IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp.
26. Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48, 63-85. http://doi.org/10.1007/BF02300500
27. Jöreskog, K.G. & Sörbom, D. (2006). LISREL 8.80 for Windows [Computer software]. Lincolnwood, IL: Scientific Software International, Inc.
28. Krell, M., Kind, V., & Krüger, D. (2024). Validation of the Scientific Reasoning Competencies Instrument: Relationships with Epistemological Beliefs and Analytical Thinking. International Journal of Science and Mathematics Education. https://doi.org/10.1007/s10763-024-10482-2
29. Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28(2), 16-25. http://doi.org/10.3102/0013189X028002016
30. Lave, J., & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge University Press.
31. Ferrando & Lorenzo-Seva (2017). Program FACTOR at 10: Origins, development and future directions.
32. Psicothema, 29(2), 236-240. http://doi.org/10.7334/psicothema2016.304.
33. Mayer, R. E. (2002). Rote versus meaningful learning. Theory into Practice, 41(4), 226-232. http://doi.org/10.1207/s15430421tip4104_4
34. Paris, S. G., & Winograd, P. (1990). How metacognition can promote academic learning and instruction. In B. F.
35. Jones & L. Idol (Eds.1), Dimensions of Thinking and Cognitive Instruction, 15-51.
36. Paul, R., & Elder, L. (2008). The Miniature Guide to Critical Thinking: Concepts and Tools. Foundation for Critical Thinking Press.
37. Perkins, D. N., & Salomon, G. (1988). Teaching for Transfer. Educational Leadership, 46(1), 22-32.
38. Rutumalessy, M., Sudyana, I. N., Azis, A. A., Suharyatun, S., & Suroso, A. (2023). The Implementation of Contextual Learning and Teaching Method in Improving Students Learning Achievement. Journal on Education, 5(4), 16262–16267. https://jonedu.org/index.php/joe/article/view/2771
39. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119-144. http://doi.org/10.1007/BF00117714
40. Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4), 219-225. http://doi.org/10.1207/s15430421tip4104_3
41. Schraw, G., & Moshman, D. (1995). Metacognitive Theories. Educational Psychology Review, 7(4), 351-371. http://doi.org/10.1007/BF02212307
42. Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. K. (1988). Cognitive Flexibility Theory: Advanced Knowledge Acquisition in Ill-Structured Domains. In V. Patel (Ed.1), Proceedings of the Tenth Annual Conference of the Cognitive Science Society, 375-383.
43. Sternberg, R. J. (1986). Critical Thinking: Its Nature, Measurement, and Improvement. National Institute of Education.
44. Swanson, H. L. (1990). Influence of metacognitive knowledge and aptitude on problem solving. Journal of Educational Psychology, 82(2), 306-314. http://doi.org/10.1037/0022-0663.82.2.306
45. Tobias, S., & Everson, H. T. (2000). Assessing metacognitive knowledge monitoring. In G. Schraw & J. Impara (Eds.2), Issues in the Measurement of Metacognition, 147-222.
46. Veenman, M. V. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3-14. http://doi.org/10.1007/s11409-006-6893-0
47. Wang, Y., & Chen, L. (2023). Enhancing student's computational thinking skills with student-generated questions strategy in a game-based learning platform. Computers & Education, 200. https://doi.org/10.1016/j.compedu.2023.104794
48. Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds. 1), Metacognition in Educational Theory and Practice, 277-304.
49. Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89-100. http://doi.org/10.1111/j.1469-7610.1976.tb00381.x
50. Yu, X., Wang, T.-Y., & Yuizono, T. (2023). Creativity development through questioning activity in second language education. Frontiers in Education, 8, 1178655. https://doi.org/10.3389/feduc.2023.1178655
51. Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64-70. http://doi.org/10.1207/s15430421tip4102_2