Inside the black box: do teachers practice assessment as learning?

Abstract

The conceptual development of assessment literature in recent years has been remarkable. One of the latest concepts to have emerged in parallel with this development is Assessment as Learning (AsL). This study investigated how AsL pertains to classroom practices within its conceptual framework by examining teacher reports. Case study design, a qualitative research method, was used to collect detailed information about in-class teacher practices. The teachers were interviewed with semi-structured interview forms and the data obtained were then analyzed using content analysis. The results revealed that in-class teacher practices were incapable of supporting AsL and promoting self-regulated behaviors and that many of the activities conducted in class were teacher-centered. Teachers did not apply self-assessment or peer-assessment practices, and the feedback they gave to students was mainly based on measurement scores. The researchers discussed the results in relation to the relevant literature and offered some suggestions for applying AsL in practice.

Keywords

• Formative assessment
• Assessment for learning
• Assessment as learning
• Self-regulation
• Self-assessment.

Kara kutunun içine bakmak: öğretmenler öğrenmeyi destekleyici değerlendirme yapıyor mu?

Abstract

Öğrenme kuramlarındaki kuramsal tartışmalara bağlı olarak durum belirleme (assessment) literatüründe öne çıkan kavramlardan biri öğrenmeyi destekleyici değerlendirme (assessment as learning- AsL) dir. Pratik öğretmen uygulamalarında bu kavramın nasıl karşılık bulduğunu belirlemek amacıyla orta okullarda çalışan öğretmenlerin sınıf içi öğretim ve değerlendirme etkinlikleri hakkında bilgi toplanmış ve bunlar AsL’in kavramsal çerçevesine uyumu bakımından değerlendirilmiştir. Bu amaca uygun olarak nitel araştırma yöntemlerinden durum belirleme çalışması deseni kullanılmıştır. Yarı yapılandırılmış görüşme formuyla öğretmenlerden veri toplanmış ve veriler içerik analizi ile çözümlenmiştir. Araştırma sonuçları sınıf içi öğretmen uygulamalarının AsL’yi desteklenme ve öz düzenleyici öğrenme davranışlarını harekete geçirme konusunda yetersiz kaldığını ve sınıf içi birçok etkinliğin öğretmen merkezli yürütüldüğünü ortaya koymuştur. Öğretmenlerin öz değerlendirme ve akran değerlendirme etkinliklerinin sıklığının çok az ve öğrenci performansıyla ilgili kullandıkları geri bildirim mekanizmasının AsL’yi desteklemekten uzak olduğu dikkat çekmiştir. Araştırma sonuçları ilgili alan yazınla ilişkili tartışılmış ve pratik uygulamalar için bazı öneriler sunulmuştur.

Keywords

• Sürece dayalı değerlendirme
• Öğrenme için değerlendirme
• Öğrenmeyi destekleyici değerlendirme
• Öz düzenleme
• Öz değerlendirme

References

1. Akıncı, B., Uzun, N., & Kışoğlu, M. (2015). Fen bilimleri öğretmenlerinin meslekte karşılaştıkları problemler ve fen öğretiminde yaşadıkları zorluklar [The problems experienced by science teachers in their profession and difficulties they are confronted with in science teaching]. International Journal of Human Sciences, 12 (1), 1189-1215. https://www.j-humansciences.com/ojs/index.php/IJHS/article/view/3188
2. Andrade, H.L. (2019). A Critical review of research on student self-assessment. Frontiers in Education, 4(87). https://doi.org/10.3389/feduc.2019.00087
3. Andrade, H., & Brookhart, S.M. (2016). The role of classroom assessment in supporting self-regulated learning. In D. Laveault & L. Allal (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 293–309). Springer.
4. Assessment Reform Group (1999). Assessment for learning: Beyond the black box. University of Cambridge School of Education.
5. Assessment Reform Group (2002). Assessment for Learning: 10 principles. http://www.assessment-reform-group.org/CIE3.PDF.
6. Baird, J.A., Andrich, D., Hopfenbeck, T.N., & Stobart, G. (2017). Assessment and learning: Fields apart?. Assessment in Education: Principles, Policy & Practice, 24(3), 317-350. https://doi.org/10.1080/0969594X.2017.1319337
7. Balbağ, M.Z., & Karaer, G. (2017). Sınıf öğretmenlerinin fen öğretiminde karşılaştıkları sorunlar [The problems of primary school teacher faced in the science teaching process]. Trakya Üniversitesi Eğitim Fakültesi Dergisi, 8(1), 28 46. https://doi.org/10.24315/trkefd.364015
8. Bandura, A. (1997). Self-efficacy: The exercise of control. W H Freeman/Times Books/ Henry Holt & Co.

9. Bennett, R. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5 25. https://doi.org/10.1080/0969594X.2010.513678
10. Berg, B.L., & Howard, L. (2016). Qualitative research methods for the social sciences (8th ed.). Pearson.
11. Black, P., & Jones, J. (2006). Formative assessment and the learning and teaching of MFL: sharing the language learning road map with the learners. Language Learning Journal, 34(1), 4-9. https://doi.org/10.1080/09571730685200171
12. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: putting it into practice. Open University Press.
13. Black, P., McCormick, R., James, M., & Pedder, D. (2006). Learning how to learn and assessment for learning. Research Papers in Education, 21(2), 119 132. https://doi.org/10.1080/02671520600615612
14. Black, P., & Wiliam, D. (2006). Developing a theory of formative assessment. In J. Gardner (Ed.), Assessment and learning (pp. 81–100). SAGE Publications, Inc.
15. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7 74. https://doi.org/10.1080/0969595980050102
16. Bloom, B. (1974). An introduction to mastery learning theory. In J. Block (Ed.) Schools, society and mastery learning. Holt, Rinehart & Winston, Inc.
17. Bloxham, S., & West, A. (2004). Understanding the rules of the game: making peer assessment as a medium for developing students’ conceptions of assessment. Assessment & Evaluation in Higher Education, 29 (6),721 733. https://doi.org/10.1080/0260293042000227254
18. Brookhart, S.M. (2018). Summative and formative feedback. In A. Lipnevich & J. Smith (Eds.), The Cambridge handbook of instructional feedback (pp. 52–78). Cambridge University Press.
19. Brown, G.T.L., & Harris, L.R. (2013). Student self-assessment. In J. McMillan (Ed.), The SAGE handbook of research on classroom assessment (pp. 367 393). SAGE Publications, Inc.
20. Boud, D., & N. Falchikov, N. (2006). Aligning assessment with long term learning. Assessment & Evaluation in Higher Education, 31 (4),399 413. https://doi.org/10.1080/02602930600679050
21. Butler, D.L., & Winne, P.H. (1995). Feedback and self regulated learning: A theoretical synthesis. Review of Educational Research, 65, 245 281. https://doi.org/10.3102/00346543065003245
22. Clark, I. (2012). Formative assessment: Assessment is for self-regulated learning. Educational Psychology Review, 24(2), 205–249. https://doi.org/10.1007/s10648-011-9191-6
23. Credé, M., & Phillips, L.A. (2011). A meta-analytic review of the motivated strategies for Learning Questionnaire. Learning and Individual Differences, 21(4), 337–346. https://doi.org/10.1016/j.lindif.2011.03.002
24. Creswell, J.W. (2018) Research design: Qualitative, quantitative, and mixed methods aproaches (4th Edition). SAGE Publications, Inc.
25. Creswell, J. W., & Miller, D.L. (2000). Determining validity in qualitative inquiry. Theory into practice, 39(3), 124-130. https://doi.org/10.1207/s15430421tip3903_2
26. Dann, R. (2014). Assessment as learning: Blurring the boundaries of assessment and learning for theory, policy and practice. Assessment in Education: Principles, Policy & Practice, 21(2), 149-166. https://doi.org/10.1080/0969594X.2014.898128
27. Davies, A., & LeMahieu, P. (2003). Assessment for learning: Reconsidering portfolios and research evidence. In M. Segers, F. Dochy, and E. Cascallar (Eds.), In optimising new modes of assessment: In search of qualities and standards, (pp. 141–69). Kluwer Academic.
28. DeLuca, C., Chapman-Chin, A., LaPointe-McEwan, D., & Klinger, D.A. (2018). Student perspectives on assessment for learning. The Curriculum Journal, 29(1), 77 94. https://doi.org/10.1080/09585176.2017.1401550
29. Dignath, C., & Büttner, G. (2008). Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and secondary school level. Metacognition and Learning, 3(3), 231-264. https://doi.org/10.1007/s11409-008-9029-x
30. Dinsmore, D.L., Alexander, P.A., & Loughlin, S.M. (2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20(4), 391–409. https://doi.org/10.1007/s10648-008-9083-6
31. Earl, L.M. (2003). Assessment as learning using classroom assessment to maximise student learning. Corwin Press.
32. Earl, L. M. (2013). Assessment as learning: Using classroom assessment to maximize student learning (2nd Edition). Corwin Press.
33. Fenwick, L. (2017). Promoting assessment for learning through curriculum-based performance standards: Teacher responses in the northern territory of Australia. Curriculum Journal, 28(1), 41–58. https://doi.org/10.1080/09585176.2016.1260486
34. Ferris, D., & Hedgcock, J. (2014). Teaching L2 composition: Purpose, process, and practice (3rd Edition). Routledge.
35. Francis, J. J., Johnston, M., Robertson, C., Glidewell, L., Entwistle, V., Eccles, M. P., & Grimshaw, J. M. (2010). What is an adequate sample size? Operationalising data saturation for theory based interview studies. Psychology and Health, 25(10), 1229 1245. https://doi.org/10.1080/08870440903194015
36. García-Pérez, D., Fraile, J., & Panadero, E. (2021). Learning strategies and self-regulation in context: How higher education students approach different courses, assessments, and challenges. European Journal of Psychology of Education, 36(2), 533-550. https://doi.org/10.1007/s10212-020-00488-z
37. Gelbal, S., & Kelecioğlu, H. (2007). Öğretmenlerin ölçme-değerlendirme yöntemleri hakkındaki yeterlik algıları ve karşılaştıkları sorunlar [Teachers’ proficiency perceptions of about the measurement and evaluation techniques and the problems they confront]. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 33, 135 145. http://efdergi.hacettepe.edu.tr/yonetim/icerik/makaleler/1017-published.pdf
38. Graham, S., Hebert, M., & Harris, K.R. (2015). Formative assessment and writing: A meta-analysis. The Elementary School Journal, 115(4), 523 547. https://doi.org/10.1086/681947
39. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
40. Hattie, J., & Jaeger, R. (1998). Assessment and classroom learning: A deductive approach. Assessment in Education: Principles, Policy & Practice, 5, 111 122. https://doi.org/10.1080/0969595980050107
41. Hawe, E., & Parr, J. (2014). Assessment for learning in the writing classroom: An incomplete realization. Curriculum Journal, 25(2), 210 237. https://doi.org/10.1080/09585176.2013.862172
42. Hennink, M.M., Kaiser, B.N., & Marconi, V.C. (2017). Code saturation versus meaning saturation: how many interviews are enough?. Qualitative Health Research, 27(4), 591-608. https://doi.org/10.1177/1049732316665344
43. Karatay, H., & Dilekçi, A. (2019). Türkçe öğretmenlerinin dil becerilerini ölçme ve değerlendirme yeterlikleri [Competencies of turkish teachers in measuring and evaluating language skills]. Milli Eğitim Dergisi, 48(1), 685 716. https://dergipark.org.tr/tr/pub/milliegitim/issue/51765/674598
44. Kim, M., & Ryu, J. (2013). The development and implementation of a web-based formative peer assessment system for enhancing students’ metacognitive awareness and performance in ill-structured tasks. Educational Technology Research and Development. 61(4), 549–561. https://doi.org/10.1007/s11423-012-9266-1
45. Kovach, R.A., Resch, D.S., & Verhulst, S.J. (2009). Peer assessment of professionalism: A five-year experience in medical clerkship. Journal of General Internal Medicine. 24(6), 742–746. https://doi.org/10.1007/s11606-009-0961-5
46. Klenowski, V. (2009) Assessment for learning revisited: An Asia Pacific perspective. Assessment in Education, 16(3), 263–268. https://doi.org/10.1080/09695940903319646
47. Klute, M., Apthorp, H., Harlacher, J., & Reale, M. (2017). Formative assessment and elementary school student academic achievement: A Review of the Evidence. REL 2017-259. Regional Educational Laboratory Central.
48. Kruger, A., & DeNisi, A. (1996). The effects of feedback interventions on performance: An historical review, meta-analysis and preliminary feedback theory. Psychological Bulletin, 119, 254-285. https://doi.org/10.1037/0033-2909.119.2.254
49. Lam, R. (2013). Formative use of summative tests: Using test preparation to promote performance and self-regulation. The Asia-Pacific Education Researcher, 22(1), 69–78. https://doi.org/10.1007/s40299-012-0026-0
50. Lam, R. (2014). Promoting self-regulated learning through portfolio assessment: Testimony and recommendations. Assessment & Evaluation in Higher Education, 39(6), 699–714. https://doi.org/10.1080/02602938.2013.862211
51. Lam, R. (2020). Investigating assessment as learning in second language writing: A qualitative research perspective. International Journal of Qualitative Methods, 19, 1 10. https://doi.org/10.1177/1609406920938572
52. Lee, H., Chung, H. Q., Zhang, Y., Abedi, J., & Warschauer, M. (2020). The effectiveness and features of formative assessment in US K-12 education: A systematic review. Applied Measurement in Education, 33(2), 124 140. https://doi.org/10.1080/08957347.2020.1732383
53. Leirhaug, P.E., & Annerstedt, C. (2016). Assessing with new eyes? Assessment for learning in Norwegian physical education. Physical Education and Sport Pedagogy, 21(6), 616-631. https://doi.org/10.1080/17408989.2015.1095871
54. Li, H., Xiong, Y., Zang, X., Kornhaber, M., Lyu, Y., Chung, K., & Suen, H.K. (2016). Peer assessment in a digital age: A Meta-analysis comparing peer and teacher ratings. Assessment & Evaluation in Higher Education, 41(2), 245 264. https://doi.org/10.1080/02602938.2014.999746
55. Liu, L., & Ji, X. (2018). A Study on the acceptability and validity of peer scoring in Chinese university EFL writing classrooms. Foreign Language World, 5, 63 70. https://doi.org/10.1016/j.jslw.2006.09.004
56. Manuel, A.K. (2015). The effects of immediate feedback using a student response system on math achievement of eleventh grade students (Unpublished doctoral dissertation). Mercer University, Macon, GA.
57. Marshall, B., & Jane Drummond, M. (2006). How teachers engage with assessment for learning: Lessons from the classroom. Research Papers in Education, 21(02), 133-149. https://doi.org/10.1080/02671520600615638
58. McCaslin, M., & Hickey, D.T. (2001). Educational psychology, social constructivism, and educational practice: A case of emergent identity. Educational Psychologist, 36(2), 133-140. https://doi.org/10.1207/S15326985EP3602_8
59. Panadero, E., Jonsson, A., & Botella, J. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: four meta-analyses. Educational Research Review, 22, 74–98. https://doi.org/10.1016/j.edurev.2017.08.004
60. Panadero, E., Andrade, H., & Brookhart, S.M. (2018). Fusing self-regulated learning and formative assessment: A roadmap of where we are, how we got here, and where we are going. The Australian Educational Researcher, 45(1), 13 31. https://doi.org/10.1007/s13384-018-0258-y
61. Panadero, E., Broadbent, J., Boud, D., & Lodge, J.M. (2019). Using formative assessment to influence self-and co-regulated learning: The role of evaluative judgement. European Journal of Psychology of Education, 34(3), 535-557. https://doi.org/10.1007/s10212-018-0407-8
62. Pintrich, P. (1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31(6), 459 470. https://doi.org/10.1016/S0883-0355(99)00015-4
63. Pintrich, P.R. (2000). The role of goal orientation in self-regulated learning. In M. Boekhaerts, P.R. Pintrich, & M. Zeidner (Eds.), Handbook of self regulation (pp. 451 502). Academic Press.
64. Popham, J. (1978). Criterion-referenced measurement. Prentice-Hall.
65. Rovers, S.F.E., Stalmeijer, R.E., van Merriënboer, J.J.G., Savelberg, H.H.C.M., & de Bruin, A.B.H. (2018). How and why do students use learning strategies? A mixed methods study on learning strategies and desirable difficulties with effective strategy users. Frontiers in Psychology, 9, 2501. https://doi.org/10.3389/fpsyg.2018.02501
66. Sadler, R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144. https://doi.org/10.1007/BF00117714
67. Sadler, D. (1998). Formative assessment: Revisiting the territory. Assessment in Education, 5(1), 77–84. https://doi.org/10.1080/0969595980050104
68. Schellekens, L.H., Bok, H.G., de Jong, L.H., van der Schaaf, M.F., Kremer, W.D., & van der Vleuten, C.P. (2021). A scoping review on the notions of assessment as learning (AaL), assessment for learning (AfL), and assessment of learning (AoL). Studies in Educational Evaluation, 71. https://doi.org/10.1016/j.stueduc.2021.101094
69. Sung, Y.T., Lin, C.S., Lee, C.L., & Chang, K.E. (2003). Evaluating proposals for experiments: An application of web based self assessment and peer assessment. Teaching of Psychology, 30(4), 331 334. https://doi.org/10.1207/S15328023TOP3004_06
70. Swaffield, S. (2011). Getting to the heart of authentic Assessment for Learning. Assessment in Education, 18(4), 433–449. https://doi.org/10.1080/0969594X.2011.582838
71. Swart, E.K., Nielen, T.M., & Sikkema-de Jong,M.T. (2019). Supporting learning from text: A meta-analysis on the timing and content of effective feedback. Educational Research Review, 28, 100296. https://doi.org/10.1016/j.edurev.2019.100296
72. Tan, K. (2013). A framework for assessment for learning: Implications for feedback practices within and beyond the gap. ISRN Education, 2013, 1 6. https://doi.org/10.1155/2013/640609
73. Theobald, M. (2021). Self-regulated learning training programs enhance university students’ academic performance, self-regulated learning strategies, and motivation: A meta-analysis. Contemporary Educational Psychology, 66, 1 19. https://doi.org/10.1016/j.cedpsych.2021.101976
74. Thompson, C.S. (2017). An Exploration of faculty involvement in and attitudes toward strategic planning in their institutions. Educational Planning, 24(1), 7 21. https://files.eric.ed.gov/fulltext/EJ1208234.pdf
75. Thompson, J., Houston, D., Dansie, K., Rayner, T., Pointon, T., Pope, S., … Grantham, H. (2017). Student & tutor consensus: A partnership in assessment for learning. Assessment and Evaluation in Higher Education, 42(6), 942 952. https://doi.org/10.1080/02602938.2016.1211988
76. Topping, K.J. (1998). Peer assessment between students in colleges and universities. Review of Educational Research 68(3), 249 276. https://doi.org/10.3102/00346543068003249
77. Topping, K.J., & Ehly, S.W. (2001). Peer assisted learning: A framework for consultation. Journal of Educational and Psychological Consultation, 12(2),113 132. https://doi.org/10.1207/S1532768XJEPC1202_03
78. Torrance, H. (1991). Records of achievement and formative assessment: some complexities of practice, in: R. Stake (Ed.) Advances in program evaluation: Using assessment policy to reform education (pp. 231-245). JAI Press.
79. Torrance, H. (2012). Formative assessment at the crossroads: Conformative, deformative and transformative assessment. Oxford Review of Education, 38, 323 342. https://doi.org/10.1080/03054985.2012.689693
80. Torrance, H. (2007). Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post-secondary education and training can come to dominate learning. Assessment in Education: Principles, Policy & Practice, 14 (3), 281–94. https://doi.org/10.1080/09695940701591867
81. van Gennip, N.A.E., Segers, M.S.R., & Tillema, H.H. (2009). Peer assessment for learning from a social perspective: The influence of interpersonal variables and structural features. Educational Research Review 4(1), 41–54. https://doi.org/10.1016/j.edurev.2008.11.002
82. Vygotsky, L.S. (1978). Interaction between learning and development. In M. Cole, V. JohnSteiner, S. Scribner, & E. Souberman (Eds.), Mind in society: The development of higher psychological processes (pp. 79–91). Harvard University Press.
83. Vygotsky, L. (1986). Thought and language. MIT Press.
84. Ward, M., Gruppen, L., & Regehr, G. (2002). Measuring self-assessment: current state of the art. Advances in Health Sciences Education, 7(1), 63 80. https://doi.org/10.1023/A:1014585522084
85. Wiliam, D. (2018). Feedback: At the heart of – But definitely not all of – Formative assessment. In A. Lipnevich & J. Smith (Eds.), The Cambridge handbook of instructional feedback (pp. 3–28). Cambridge University Press. https://doi.org/10.1017/9781316832134.003
86. Weinstein, C.E., Acee, T.W., & Jung, J. (2011). Self-regulation and learning strategies. New Directions for Teaching and Learning, 126, 45–53. https://doi.org/10.1002/tl.443
87. Wolters, C.A. (2003). Regulation of motivation: Evaluating an underemphasized aspect of self-regulated learning. Educational Psychologist, 38(4), 189 205. https://doi.org/10.1207/S15326985EP3804_1
88. Wyse, D., & Torrance, H. (2009). The development and consequences of national curriculum assessment for primary education in England. Educational Research, 51(2), 213–238. https://doi.org/10.1080/00131880902891479
89. Zimmerman, B.J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P.R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13–40). Academic Press.
90. Zimmerman, B.J., & Pons, M.M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23(4), 614–628. https://doi.org/10.3102/00028312023004614