Çok Yönlü Rasch Modeli ile Akran Değerlendirme Analizi

Öz

Akran değerlendirme, dinamik bir yaklaşım olarak eğitim araştırmalarında giderek daha fazla ilgi görmektedir. Bu bağlamda, bu çalışma üniversite öğrencilerinin drama becerilerine yönelik akran değerlendirmelerini çok yönlü Rasch modelini kullanarak analiz etmeyi amaçlamaktadır. Bu çalışmada betimsel desenli nicel araştırma yöntemi kullanılmıştır. Çalışma grubu, 2024-2025 akademik yılında bir devlet üniversitesinde öğrenim gören 10 üniversite öğrencisinden oluşmaktadır. Veriler, beşli Likert ölçeğine göre puanlanan 32 madde içeren “Drama Becerileri Akran Değerlendirme Formu” kullanılarak toplanmıştır. Analizler FACETS programı kullanılarak gerçekleştirilmiştir. Öğrenci, değerlendirici ve madde olmak üzere üç boyut belirlenmiştir. Tüm katılımcılar değerlendirme formu üzerinden birbirleri için akran değerlendirmeleri yapmış ve 2880 (10x9x32) veri elde edilmiştir. Bulgular, öğrencilerin drama becerilerinde, değerlendiricilerin puanlamadaki katılık/cömertliklerinde ve maddelerin zorluk düzeylerinde anlamlı istatistiksel farklılıklar olduğunu ortaya koymuştur. Çalışma, akran değerlendirme sürecini etkileyen faktörlerin daha fazla incelenmesi gerektiğini göstermiştir.

Anahtar Kelimeler

• akran değerlendirme
• drama becerileri
• çok yönlü Rasch modeli
• yükseköğretim
• performans analizi

Etik Beyan

Bu çalışma, Tokat Gaziosmanpaşa Üniversitesi Sosyal ve Beşeri Bilimler Araştırmaları Etik Kurulu'nun 14.01.2025 tarihli ve 01.56 numaralı onayı ile gerçekleştirilmiştir.

Peer Assessment Analysis via the Many-Facet Rasch Model

Öz

Peer assessment has gained increasing attention in educational research as a dynamic approach. In this regard, this paper aims to analyze peer assessments of university students' drama skills using the many-facet Rasch model. In this study, quantitative research method with a descriptive design was used. The study group comprised 10 university students enrolled at a state university during the 2024-2025 academic year. Data were collected using the "Drama Skills Peer Assessment Form," which includes 32 items scored on a five-point Likert scale. The analysis was conducted using the FACETS program. Three facets were determined as student, assessor, and item. All participants conducted peer assessments for each other on the assessment form and 2880 (10x9x32) data were obtained. The findings revealed significant statistical differences in students' drama skills, assessors' strictness/generosity in scoring, and the levels of difficulty for the item. The study highlighted the need for further examination of the factors influencing peer assessment process.

Anahtar Kelimeler

• peer assessment
• drama skills
• many-facet Rash model
• higher education
• performance analysis

Etik Beyan

This study was carried out with the approval of Tokat Gaziosmanpaşa University Ethics Council for the Research in Social and Human Sciences dated 14.01.2025 and numbered 01.56.

Kaynakça

1. Adachi, C., Tai, J., & Dawson, P. (2018). A framework for designing, implementing, communicating and researching peer assessment. Higher Education Research & Development, 37(3), 453-467. https://doi.org/10.1080/07294360.2017.1405913
2. Adams, J., & Owens, A. (2016). Creativity, education and democracy: The practices and politics of learning in the arts. Routledge.
3. Armengol-Asparó, C., Mercader, C., & Ion, G. (2022). Making peer-feedback more efficient: What conditions of its delivery make the difference? Higher Education Research & Development, 41(2), 226-239. https://doi.org/10.1080/07294360.2020.1840527
4. Asimidou, A., Lenakakis, A., & Tsiaras, A. (2021). The contribution of drama pedagogy in developing adolescents’ self-confidence: A case study. NJ: Drama Australia Journal, 45(1), 45-58. https://doi.org/10.1080/14452294.2021.1978145
5. Badea, G., & Popescu, E. (2022). A dynamic review allocation approach for peer assessment in technology enhanced learning. Education and Information Technologies, 27(9), 13131-13162. https://doi.org/10.1007/s10639-022-11175-5
6. Barrera, F., Venegas-Muggli, J. I., & Nuñez, O. (2021). The impact of role-playing simulation activities on higher education students’ academic results. Innovations in Education and Teaching International, 58(3), 305-315. https://doi.org/10.1080/14703297.2020.1740101
7. Belliveau, G., & Kim, W. (2013). Drama in L2 learning: A research synthesis. Scenario 7(2), 7-27. https://doi.org/10.33178/scenario.7.2.2
8. Bessadet, L. (2022). Drama-based approach in English language teaching. Arab World English Journal, 13(1) 525-533. https://dx.doi.org/10.24093/awej/vol13no1.34

9. Bijami, M., Kashef, S. H., & Nejad, M. S. (2013). Peer feedback in learning English writing: Advantages and disadvantages. Journal of Studies in Education, 3(4), 91. https://doi.org/10.5296/jse. v3i4.431
10. Bradley, S., Kirby, E., & Madriaga, M. (2015). What students value as inspirational and transformative teaching. Innovations in Education and Teaching International, 52(3), 231-242. https://doi.org/10. 1080/14703297.2014.880363
11. Brown, T., Rongerude, J., Leonard, B., & Merrick, L. C. (2021). Best practices for online team-based learning: Strengthening teams through formative peer evaluation. New Directions for Teaching and Learning, 2021, 53-64. https://doi.org/10.1002/tl.20436
12. Conde, M.´A., S´anchez-Gonz´alez, L., Matellan-Olivera, V., & Lera, F. J. R. (2017). Application of peer review techniques in engineering education. International Journal of Engineering Education, 33(2), 918-926.
13. Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Sage Publications.
14. Deeley, S. J., & Bovill, C. (2015). Staff student partnership in assessment: enhancing assessment literacy through democratic practices. Assessment & Evaluation in Higher Education, 42(3), 463-477. https://doi.org/10.1080/02602938.2015.1126551
15. Deng, Y., Liu, D., & Feng, D. (2023). Students’ perceptions of peer review for assessing digital multimodal composing: the case of a discipline-specific English course. Assessment & Evaluation in Higher Education, 48(8), 1254-1267. https://doi.org/10.1080/02602938.2023.2227358
16. Devisakti, A., & Ramayah, T. (2021). Sense of belonging and grit in e-learning portal usage in higher education. Interactive Learning Environments, 31(8), 4850-4864. https://doi.org/10.1080/10494820.2021.1983611
17. Double, K. S., McGrane, J. A., & Hopfenbeck, T. N. (2020). The impact of peer assessment on academic performance: A meta-analysis of control group studies. Educational Psychology Review, 32, 481-509. https://doi.org/10.1007/s10648-019-09510-3
18. Fleming, M. (2006). Drama and language teaching: the relevance of Wittgenstein’s concept of language games. Humanising Language Teaching, 8(4), 1-12.
19. Garaigordobil, M., & Berrueco, L. (2011). Effects of a play program on creative thinking of preschool children. The Spanish Journal of Psychology, 14(2), 608-618. https://doi.org/10.5209/rev_SJOP.2011.v14.n2.9
20. Gaynor, J. W. (2019). Peer review in the classroom: Student perceptions, peer feedback quality and the role of assessment. Assessment & Evaluation in Higher Education, 45(5), 758-775. https://doi.org/10. 1080/02602 938. 2019.1697424
21. Guelfi, M. R., Formiconi, A. R., Vannucci, M., Tofani, L., Shtylla, J., & Masoni, M. (2021). Application of peer review in a university course: Are students good reviewers? Journal of E-Learning and Knowledge Society, 17(2), 1-8. https://doi.org/10.20368/1971-8829/1135380
22. Hagenauer G., Muehlbacher F., & Ivanova, M. (2023).“It’s where learning and teaching begins - is this relationship”- insights on the teacher-student relationship at university from the teachers’ perspective. Higher Education, 85, 819-835. https://doi.org/10.1007/s10734-022-00867-z
23. Hammond, N. (2015). Forum theatre for children: Enhancing social, emotional and creative development. Institute of Education Press.
24. Heffernan, T., Morrison, M., Sweeney, A., & Jarratt, D. (2010). Personal attributes of effective lecturers: The importance of dynamism, communication, rapport and applied knowledge. The International Journal of Management Education, 8(3), 13-27. https://doi.org/10.3794/ijme.83.275
25. Humberstone, J. H. B., Zhao, C., Liu, D., & Elizabeth, M. (2024). Stimulating music educator worldview change through a pedagogy of provocation, critical thinking and peer-review. International Journal of Music Education, 42(1), 47-67. https://doi.org/10.1177/02557614221149223
26. Isyar, O., & Akay, C. (2017). The use of drama in education in primary schools from the viewpoint of the classroom teachers: A mixed method research. Journal of Education & Practice, 8(28), 215-230.
27. Jiang, J. P., Hu, J. Y., Zhang, Y. B., & Yin, X. C. (2022). Fostering college students’ critical thinking skills through peer assessment in the knowledge building community. Interactive Learning Environments, 31(10), 6480-6496. https://doi.org/10.1080/10494 820. 2022.20399 49
28. Jin, Z. (2000). The learning experience of students in Middlesex University Business School (MUBS): Why do they enjoy some modules/lectures and dislike others? The International Journal of Management Education, 1(1), 22-36. https://doi.org/10.3794/ijme.11.c
29. Jongsma, M. V., Scholten, D. J., van Muijlwijk-Koezen, J. E., & Meeter, M. (2023). Online versus offline peer feedback in higher education: A meta-analysis. Journal of Educational Computing Research, 61(2), 329-354. https://doi.org/10.1177/07356331221114181
30. Kim, M., & Ryu, J. (2013). The development and implementation of a web-based formative peer assessment system for enhancing students’ metacognitive awareness and performance in ill-structured tasks. Educational Technology Research and Development, 61, 549-561. https://doi.org/10.1007/s11423-012-9266-1
31. Linacre, J. M. (1994). Many-facet rasch model (2nd ed.). Mesa Press.
32. Linacre, J. M. (1993). Rasch-based generalizability theory. Rasch Measurement Transactions, 7(1), 283-284. https://www.rasch.org/rmt/rmt71h.htm
33. Lubicz-Nawrocka, T., & Bunting, K. (2019). Student perceptions of teaching excellence: An analysis of student-led teaching award nomination data. Teaching in Higher Education, 24(1), 63-80. https://doi.org/10.1080/13562517.2018.1461620
34. Maley, A., & Duff, A. (2005). Drama techniques: A resource book of communication activities for language teachers. Cambridge University Press. https://doi.org/10.1017/CB09780511733079
35. Mercer, S., & Dornyei, Z. (2020). Engaging language learners in contemporary classrooms. Cambridge University Press.
36. Mumpuni, K. E., Priyayi, D. F., & Widoretno, S. (2022). How do students perform a peer assessment? International Journal of Instruction, 15(3), 751-766. https://doi.org/10.29333/iji.2022.15341a
37. Nicholson, H. (2009). Theatre and education. Palgrave Macmillan.
38. Noroozi, O., Banihashem, S. K., Biemans, H. J. A., Smits, M., Vervoort, M. T. W., & Verbaan, C.-L. (2023). Design, implementation, and evaluation of an online supported peer feedback module to enhance students’ argumentative essay quality. Education and Information Technologies, 28(10), 12757-12784. https://doi.org/10.1007/s10639-023-11683-y
39. Panadero, E., & Alqassab, M. (2019). An empirical review of anonymity effects in peer assessment, peer feedback, peer review, peer evaluation and peer grading. Assessment & Evaluation in Higher Education, 44(8), 1253-1278. https://doi.org/10.1080/02602938.2019.1600186
40. Panadero, E., & Broadbent, J. (2018). Developing evaluative judgement: A self-regulated learning perspective. In D. Boud, R. Ajjawi, P. Dawson, & J. Tai (Eds.), Developing Evaluative Judgement in Higher Education: Assessment for Knowing and Producing Quality Work (pp. 81-89). Routledge.
41. Patchan, M. M., Schunn, C. D., & Correnti, R. J. (2016). The nature of feedback: How peer feedback features affect students’ implementation rate and quality of revisions. Journal of Educational Psychology, 108(8), 1098-1120. https://doi.org/10.1037/edu0000103
42. Pinciotti, P. (1993). Creative drama and young children: The dramatic learning connection. Arts Education Policy Review, 94(6), 24-28.
43. Poitras, J., Stimec, A., & Hill, K. (2013). On teaching: Fostering student engagement in negotiation role plays. Negotiation Journal, 29(4), 439-462. https://doi.org/10.1111/nejo.12036
44. Prins, F. J., Sluijsmans, D. M. A., Kirschner, P. A., & Strijobs, J. W. (2005). Formative peer assessment in a CSCL environment: a case study. Assessment and Evaluation in Higher Education, 30(4), 417-444. https://doi.org/10.1080/02602930500099219
45. Reddy, K., Harland, T., Wass, R., & Wald, N. (2021). Student peer review as a process of knowledge creation through dialogue. Higher Education Research & Development, 40(4), 825-837. https://doi.org/10.1080/07294360.2020.1781797
46. Reinholz, D. (2016). The assessment cycle: a model for learning through peer assessment. Assessment & Evaluation in Higher Education, 41(2), 301-315. https://doi.org/10.1080/02602938.2015.1008982
47. Sengül, K., & Sünbül, A. M. (2015). An investigation into comprehension competence and grammar achievements of learners of Turkish as a foreign language in Turkey. Educational Alternatives, 13, 573-580.
48. Seppanen, S. (2022). What would happen if I said yes? Measuring the immediate and long-term impact of improvisation training on student teachers’ subjective, neuroendocrine and psychophysiological responses [Unpublished doctoral dissertation]. University of Helsinki.
49. Smith, H., Cooper, A., & Lancaster, L. (2002). Improving the quality of undergraduate peer assessment: A case for student and staff development. Innovations in Education and Teaching International, 39(1), 71-81. https://doi.org/10.1080/13558 000110102904
50. Stevens, R. (2015). Role-play and student engagement: Reflections from the classroom. Teaching in Higher Education, 20(5), 481-492. https://doi.org/10.1080/13562517.2015.1020778
51. Sudweeks, R. R., Reeve, S., & Bradshaw, W. S. (2004). A comparison of generalizability theory and many-facet Rasch measurement in an analysis of college sophomore writing. Assessing Writing, 9(3), 239-261. https://doi.org/10.1016/j.asw.2004.11.001
52. Thom, M. (2019). Drama for teaching and working in academia - using stagecraft to develop academics and their teaching practice in higher education. Educational Developments, 20(1), 18-22.
53. Topping, K. J. (2009). Peer assessment. Theory into Practice, 48(1), 20-27. https://doi.org/10.1080/00405840802577569
54. Topping, K. J. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249-276. https://doi.org/10.3102/00346543068003249
55. Weng, F., Ye, S. X., & Xue, W. (2023). The effects of peer feedback on L2 students’ writing motivation: An experimental study in China. The Asia-Pacific Education Researcher, 32, 473-483. https://doi.org/10.1007/s40299-022-00669-y
56. Wilson, M. J., Diao, M. M., & Huang, L. (2015). ‘I’m not here to learn how to mark someone else’s stuff’: An investigation of an online peer-to-peer review workshop tool. Assessment & Evaluation in Higher Education, 40(1), 15-32. https://doi.org/10.1080/02602938.2014.881980
57. Winstone, N., & Carless, D. (2020). Designing effective feedback processes in higher education: A learning-focused approach. Routledge.
58. Wright, B.D., & Linacre, J.M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370-382. https://www.rasch.org/rmt/rmt83b.htm
59. Yao, C. (2017). A case study on the factors affecting Chinese adult students’ English acquisition in a blended learning environment. International Journal of Continuing Engineering Education and Life Long Learning, 27(1/2), 22-44. https://doi.org/10.1504/IJCEELL.2017.080993
60. Yao, C., & Shao, S. (2024). Factors affecting teacher-student interactions in a foreign language teaching class: a language socialisation theory perspective. Humanities and Social Sciences Communications, 11, 1358. https://doi.org/10.1057/s41599-024-03900-x
61. Zhan, Y. (2021). Are they ready? An investigation of university students’ difficulties in peer assessment from dual perspectives. Teaching in Higher Education, 29(4), 823-840. https://doi.org/10. 1080/13562517.2021.2021393
62. Zhan, Y., Yan, Z., Wan, H. Z., Wang, X., Zeng, Y., Yang, M., & Yang, L. (2023). Effects of online peer assessment on higher-order thinking: A meta-analysis. British Journal of Educational Technology, 54(4), 817-835. https://doi.org/10.1111/bjet.13310
63. Zheng, L., Chen, N. S., Cui, P., & Zhang, X. (2019). A systematic review of technology-supported peer assessment research. The International Review of Research in Open and Distributed Learning, 20(5), 168-191. https://doi.org/10.19173/irrodl.v20i5.4333