PSCP Cognitive Engagement Scale: A Scale Development Study

Year 2025, Issue: 56, 80 - 90, 11.03.2025

Filiz Özek , Seval Fer

https://doi.org/10.33418/education.1527281

Abstract

This study aimed to develop and validate an instrument to measure students' cognitive engagement in teaching-learning environments. An exploratory correlational method was employed to develop the scale. 446 university pre‐degree students learning English as a foreign language participated in the study. A pilot study was conducted with 117 students to explore the item and factor structure of the scale, resulting in the removal of eight items from the scale. A subsequent study with 329 students was conducted to confirm the scale's item and factor structure. Results showed that the scale demonstrated content validity, with a content validity index of .94. The scale consisted of nine items and two factors, identified as cognitive attention and cognitive effort. Convergent validity was established, as evidenced by composite reliability values of .83 and .84 for each factor, with average variance extracted of .55 and .51, respectively. Corrected item-total correlation values ranged from .54 to .71, and inter-item correlation exceeded .30. Reliability analysis revealed high internal consistency, with each factor demonstrating reliability of .83 and .85, resulting in an overall scale reliability of .89. In conclusion, the findings indicate that the developed PSCP Cognitive Engagement Scale is a valid and reliable scale for measuring cognitive engagement in learning environments.

Keywords

Cognitive engagement, scale development, factor analysis, reliability, validity

PSCP Bilişsel Katılım Ölçeği: Bir Ölçek Geliştirme Çalışması

Abstract

Bu çalışmanın amacı, öğrencilerin derslerdeki bilişsel katılımlarını ölçmek için güvenilir ve geçerli bir ölçek geliştirmektir. Ölçek geliştirilirken nicel araştırma yöntemlerinden biri olan açımlayıcı ilişkisel yöntem takip edilmiştir. Araştırmaya, iki üniversitede İngilizceyi yabancı dil olarak öğrenen 446 üniversite hazırlık sınıfı öğrencisi katılmıştır. Ölçeğin madde ve faktör yapısını incelemek amacıyla 117 öğrenci ile bir pilot çalışma yürütülmüş ve bu çalışma sonucunda ölçekten sekiz madde çıkarılmıştır. Ardından, ölçeğin madde ve faktör yapısını doğrulamak için 329 öğrenci ile ikinci çalışma yapılmıştır. Elde edilen sonuçlar, ölçeğin kapsam geçerlilik indeksini .94 olarak ortaya koymuştur. Ölçek, bilişsel dikkat ve bilişsel çaba olarak tanımlanan iki faktör ve dokuz madde içermektedir. Her faktör için bileşik güvenilirlik değerleri sırasıyla 0.83 ve 0.84; ortalama varyans değerleri ise sırasıyla 0.55 ve 0.51 olarak belirlenmiştir. Ölçek maddelerinin madde-toplam korelasyon değerleri 0.54 ile 0.71 arasında değişirken maddeler arası korelasyon değerlerinin 0.30’in üzerinde olduğu bulgulanmıştır. Güvenilirlik analizi, birinci faktör için 0.83; ikinci faktör için 0.85 ve ölçeğin tamamı için 0.89 olarak hesaplanmıştır. Sonuç olarak, bu bulgular, derslerde öğrencilerin bilişsel katılımlarını ölçmek üzere geliştirilen PSCP Sosyal Varlık Ölçeği'nin geçerli ve güvenilir bir ölçme aracı olduğunu göstermektedir.

Keywords

Bilişsel Katılım, ölçek geliştirme, faktör analizi, güvenilirlik, geçerlik

References

• Agustini, K., Santyasa, I. W., Tegeh, I. M., Santyadiputra, G. S., & Mertayasa, I. N. E. (2022). Quantum flipped learning and students’ cognitive engagement in achieving their critical and creative thinking in learning. International Journal of Emerging Technologies in Learning, 17(18), 4. https://doi.org/https://doi.org/10.3991/ijet.v17i18.32101
• Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of school psychology, 44(5), 427-445. https://doi.org/10.1016/j.jsp.2006.04.002
• Archambault, I., Janosz, M., Morizot, J., & Pagani, L. (2009). Adolescent behavioral, affective, and cognitive engagement in school: Relationship to dropout. Journal of School Health, 79(9), 408-415. https://doi.org/10.1111/j.1746-1561.2009.00428.x
• Barlow, A., Brown, S., Lutz, B., Pitterson, N., Hunsu, N., & Adesope, O. (2020). Development of the student course cognitive engagement instrument (SCCEI) for college engineering courses. International Journal of STEM Education, 7(1), 1-20. https://doi.org/https://doi.org/10.1186/s40594-020-00220-9
• Blumenfeld, P. C., Kempler, T. M., & Krajcik, J. S. (2006). Motivation and cognitive engagement in learning environments. In The Cambridge handbook of: The learning sciences. (pp. 475-488). Cambridge University Press.
• Burch, G. F., Heller, N. A., Burch, J. J., Freed, R., & Steed, S. A. (2015). Student engagement: Developing a conceptual framework and survey instrument. Journal of Education for Business, 90(4), 224-229. https://doi.org/https://doi.org/10.1080/08832323.2015.1019821
• Büyüköztürk, Ş. (2018). Data analysis handbook for social sciences. Pegem.
• Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219-243. https://doi.org/10.1080/00461520.2014.965823
• Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Pearson Education Limited.
• DeVellis, R. F. (2003). Scale development: theory and applications. SAGE Publications.
• Earl, S. R., Taylor, I. M., Meijen, C., & Passfield, L. (2023). Trajectories in cognitive engagement, fatigue, and school achievement: The role of young adolescents' psychological need satisfaction. Learning and Individual Differences, 101, 1-11. https://doi.org/https://doi.org/10.1016/j.lindif.2022.102248
• Finn, J. D., & Zimmer, K. S. (2012). Student engagement: What is it? Why does it matter? In Handbook of research on student engagement. (pp. 97-131). Springer Science + Business Media. https://doi.org/10.1007/978-1-4614-2018-7_5
• Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research, 74(1), 59-109. https://doi.org/10.3102/00346543074001059
• Fredricks, J. A., & McColskey, W. (2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In Handbook of research on student engagement (pp. 763-782). Springer.
• Greene, B., & Miller, R. (1996). Influences on course performance: Goals, perceived ability, and self-regulation. Contemporary Educational Psychology, 21(2), 181-192. https://doi.org/https://doi.org/10.1006/ceps.1996.0015
• Greene, B. A. (2015). Measuring cognitive engagement with self-report scales: Reflections from over 20 years of research. Educational Psychologist, 50(1), 14-30. https://doi.org/https://doi.org/10.1080/00461520.2014.989230
• Gunuc, S., & Kuzu, A. (2015). Student engagement scale: development, reliability and validity. Assessment & Evaluation in Higher Education, 40(4), 587-610. https://doi.org/https://doi.org/10.1080/02602938.2014.938019
• Guthrie, R., & Carlin, A. (2024). Waking the dead: Using interactive technology to engage passive listeners in the classroom. Proceedings of the Tenth Americas Conference on Information Systems. http://aisel.aisnet.org/amcis2004/358
• Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2013). Multivariate data analysis. Pearson education limited. https://books.google.ca/books?id=VvXZnQEACAAJ
• Helme, S., & Clarke, D. (2001). Identifying cognitive engagement in the mathematics classroom. Mathematics Education Research Journal, 13(2), 133-153. https://doi.org/https://doi.org/10.1007/BF03217103
• Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015). Measuring student engagement in technology-mediated learning: A review. Computers and Education, 90, 36-53. https://doi.org/10.1016/j.compedu.2015.09.005
• Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS. British Journal of Educational Technology, 47(2), 320-341. https://doi.org/10.1111/bjet.12235
• Huang, B. Y., Hew, K. F., & Lo, C. K. (2019). Investigating the effects of gamification-enhanced flipped learning on undergraduate students' behavioral and cognitive engagement. Interactive Learning Environments, 27(8), 1106-1126. https://doi.org/10.1080/10494820.2018.1495653
• Husni, N. A., Jumaat, N., & Tasir, Z. (2022). Investigating student’s cognitive engagement, motivation and cognitive retention in learning management system. International Journal of Emerging Technologies in Learning, 17(9), 184-200. https://doi.org/https://doi.org/10.3991/ijet.v17i09.29727
• Kline, R. B. (2023). Principles and practice of structural equation modeling. Guilford.
• Kong, S., & Hoare, P. (2011). Cognitive content engagement in content-based language teaching. Language Teaching Research, 15(3), 307-324. https://doi.org/https://doi.org/10.1177/1362168811401152
• Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
• Liem, G. A. D., & Martin, A. J. (2012). The motivation and engagement scale: Theoretical framework, psychometric properties, and applied yields. Australian Psychologist, 47(1), 3-13. https://doi.org/https://doi.org/10.1111/j.1742-9544.2011.00049.x
• Liu, K., Yao, J., Tao, D., & Yang, T. (2023). Influence of individual-technology-task-environment fit on university student online learning performance: The mediating role of behavioral, emotional, and cognitive engagement. Education and Information Technologies, 28, 1-20. https://doi.org/https://doi.org/10.1007/s10639-023-11833-2
• Meece, J. L., Blumenfeld, P. C., & Hoyle, R. H. (1988). Students' goal orientations and cognitive engagement in classroom activities. Journal of Educational Psychology, 80(4), 514. https://doi.org/https://doi.org/10.1037/0022-0663.80.4.514
• Shukor, N. A., Tasir, Z., Van der Meijden, H., & Harun, J. (2014). A predictive model to evaluate students’ cognitive engagement in online learning. Procedia - Social and Behavioral Sciences, 116, 4844-4853. https://doi.org/https://doi.org/10.1016/j.sbspro.2014.01.1036
• Silviana, M., Mafruhah, O. R., & Ningrum, V. D. A. (2024). Validation of questionnaires and the effect of educational videos on the knowledge of hyperlipidemia patients at banjarbaru utara primary health care. Tropical Health and Medical Research, 6(1), 27-41. https://doi.org/https://doi.org/10.35916/thmr.v6i1.113
• Voelkl, K. E. (1997). Identification with School. American Journal of Education, 105(3), 294 - 318. https://doi.org/https://doi.org/10.1086/444158
• Yazzie-Mintz, E. (2007). Voices of students on engagement: A report on the 2006 high school survey of student engagement (Center for Evaluation and Education Policy, Indiana University, Issue.