Çok Yönlü Rasch Modeli ile Akran Değerlendirme Analizi

Year 2025, Volume: 21 Issue: 2, 631 - 646, 20.08.2025

Burak Ayçiçek

https://doi.org/10.17860/mersinefd.1642676

https://izlik.org/JA99BL38NM

Abstract

Akran değerlendirme, dinamik bir yaklaşım olarak eğitim araştırmalarında giderek daha fazla ilgi görmektedir. Bu bağlamda, bu çalışma üniversite öğrencilerinin drama becerilerine yönelik akran değerlendirmelerini çok yönlü Rasch modelini kullanarak analiz etmeyi amaçlamaktadır. Bu çalışmada betimsel desenli nicel araştırma yöntemi kullanılmıştır. Çalışma grubu, 2024-2025 akademik yılında bir devlet üniversitesinde öğrenim gören 10 üniversite öğrencisinden oluşmaktadır. Veriler, beşli Likert ölçeğine göre puanlanan 32 madde içeren “Drama Becerileri Akran Değerlendirme Formu” kullanılarak toplanmıştır. Analizler FACETS programı kullanılarak gerçekleştirilmiştir. Öğrenci, değerlendirici ve madde olmak üzere üç boyut belirlenmiştir. Tüm katılımcılar değerlendirme formu üzerinden birbirleri için akran değerlendirmeleri yapmış ve 2880 (10x9x32) veri elde edilmiştir. Bulgular, öğrencilerin drama becerilerinde, değerlendiricilerin puanlamadaki katılık/cömertliklerinde ve maddelerin zorluk düzeylerinde anlamlı istatistiksel farklılıklar olduğunu ortaya koymuştur. Çalışma, akran değerlendirme sürecini etkileyen faktörlerin daha fazla incelenmesi gerektiğini göstermiştir.

Keywords

akran değerlendirme , drama becerileri , çok yönlü Rasch modeli , yükseköğretim , performans analizi

Ethical Statement

Bu çalışma, Tokat Gaziosmanpaşa Üniversitesi Sosyal ve Beşeri Bilimler Araştırmaları Etik Kurulu'nun 14.01.2025 tarihli ve 01.56 numaralı onayı ile gerçekleştirilmiştir.

Peer Assessment Analysis via the Many-Facet Rasch Model

https://izlik.org/JA99BL38NM

Abstract

Peer assessment has gained increasing attention in educational research as a dynamic approach. In this regard, this paper aims to analyze peer assessments of university students' drama skills using the many-facet Rasch model. In this study, quantitative research method with a descriptive design was used. The study group comprised 10 university students enrolled at a state university during the 2024-2025 academic year. Data were collected using the "Drama Skills Peer Assessment Form," which includes 32 items scored on a five-point Likert scale. The analysis was conducted using the FACETS program. Three facets were determined as student, assessor, and item. All participants conducted peer assessments for each other on the assessment form and 2880 (10x9x32) data were obtained. The findings revealed significant statistical differences in students' drama skills, assessors' strictness/generosity in scoring, and the levels of difficulty for the item. The study highlighted the need for further examination of the factors influencing peer assessment process.

Keywords

peer assessment , drama skills , many-facet Rash model , higher education , performance analysis

Ethical Statement

This study was carried out with the approval of Tokat Gaziosmanpaşa University Ethics Council for the Research in Social and Human Sciences dated 14.01.2025 and numbered 01.56.

References

• Adachi, C., Tai, J., & Dawson, P. (2018). A framework for designing, implementing, communicating and researching peer assessment. Higher Education Research & Development, 37(3), 453-467. https://doi.org/10.1080/07294360.2017.1405913
• Adams, J., & Owens, A. (2016). Creativity, education and democracy: The practices and politics of learning in the arts. Routledge.
• Armengol-Asparó, C., Mercader, C., & Ion, G. (2022). Making peer-feedback more efficient: What conditions of its delivery make the difference? Higher Education Research & Development, 41(2), 226-239. https://doi.org/10.1080/07294360.2020.1840527
• Asimidou, A., Lenakakis, A., & Tsiaras, A. (2021). The contribution of drama pedagogy in developing adolescents’ self-confidence: A case study. NJ: Drama Australia Journal, 45(1), 45-58. https://doi.org/10.1080/14452294.2021.1978145
• Badea, G., & Popescu, E. (2022). A dynamic review allocation approach for peer assessment in technology enhanced learning. Education and Information Technologies, 27(9), 13131-13162. https://doi.org/10.1007/s10639-022-11175-5
• Barrera, F., Venegas-Muggli, J. I., & Nuñez, O. (2021). The impact of role-playing simulation activities on higher education students’ academic results. Innovations in Education and Teaching International, 58(3), 305-315. https://doi.org/10.1080/14703297.2020.1740101
• Belliveau, G., & Kim, W. (2013). Drama in L2 learning: A research synthesis. Scenario 7(2), 7-27. https://doi.org/10.33178/scenario.7.2.2
• Bessadet, L. (2022). Drama-based approach in English language teaching. Arab World English Journal, 13(1) 525-533. https://dx.doi.org/10.24093/awej/vol13no1.34
• Bijami, M., Kashef, S. H., & Nejad, M. S. (2013). Peer feedback in learning English writing: Advantages and disadvantages. Journal of Studies in Education, 3(4), 91. https://doi.org/10.5296/jse. v3i4.431
• Bradley, S., Kirby, E., & Madriaga, M. (2015). What students value as inspirational and transformative teaching. Innovations in Education and Teaching International, 52(3), 231-242. https://doi.org/10. 1080/14703297.2014.880363
• Brown, T., Rongerude, J., Leonard, B., & Merrick, L. C. (2021). Best practices for online team-based learning: Strengthening teams through formative peer evaluation. New Directions for Teaching and Learning, 2021, 53-64. https://doi.org/10.1002/tl.20436
• Conde, M.´A., S´anchez-Gonz´alez, L., Matellan-Olivera, V., & Lera, F. J. R. (2017). Application of peer review techniques in engineering education. International Journal of Engineering Education, 33(2), 918-926.
• Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Sage Publications.
• Deeley, S. J., & Bovill, C. (2015). Staff student partnership in assessment: enhancing assessment literacy through democratic practices. Assessment & Evaluation in Higher Education, 42(3), 463-477. https://doi.org/10.1080/02602938.2015.1126551
• Deng, Y., Liu, D., & Feng, D. (2023). Students’ perceptions of peer review for assessing digital multimodal composing: the case of a discipline-specific English course. Assessment & Evaluation in Higher Education, 48(8), 1254-1267. https://doi.org/10.1080/02602938.2023.2227358
• Devisakti, A., & Ramayah, T. (2021). Sense of belonging and grit in e-learning portal usage in higher education. Interactive Learning Environments, 31(8), 4850-4864. https://doi.org/10.1080/10494820.2021.1983611
• Double, K. S., McGrane, J. A., & Hopfenbeck, T. N. (2020). The impact of peer assessment on academic performance: A meta-analysis of control group studies. Educational Psychology Review, 32, 481-509. https://doi.org/10.1007/s10648-019-09510-3
• Fleming, M. (2006). Drama and language teaching: the relevance of Wittgenstein’s concept of language games. Humanising Language Teaching, 8(4), 1-12.
• Garaigordobil, M., & Berrueco, L. (2011). Effects of a play program on creative thinking of preschool children. The Spanish Journal of Psychology, 14(2), 608-618. https://doi.org/10.5209/rev_SJOP.2011.v14.n2.9
• Gaynor, J. W. (2019). Peer review in the classroom: Student perceptions, peer feedback quality and the role of assessment. Assessment & Evaluation in Higher Education, 45(5), 758-775. https://doi.org/10. 1080/02602 938. 2019.1697424
• Guelfi, M. R., Formiconi, A. R., Vannucci, M., Tofani, L., Shtylla, J., & Masoni, M. (2021). Application of peer review in a university course: Are students good reviewers? Journal of E-Learning and Knowledge Society, 17(2), 1-8. https://doi.org/10.20368/1971-8829/1135380
• Hagenauer G., Muehlbacher F., & Ivanova, M. (2023).“It’s where learning and teaching begins - is this relationship”- insights on the teacher-student relationship at university from the teachers’ perspective. Higher Education, 85, 819-835. https://doi.org/10.1007/s10734-022-00867-z
• Hammond, N. (2015). Forum theatre for children: Enhancing social, emotional and creative development. Institute of Education Press.
• Heffernan, T., Morrison, M., Sweeney, A., & Jarratt, D. (2010). Personal attributes of effective lecturers: The importance of dynamism, communication, rapport and applied knowledge. The International Journal of Management Education, 8(3), 13-27. https://doi.org/10.3794/ijme.83.275
• Humberstone, J. H. B., Zhao, C., Liu, D., & Elizabeth, M. (2024). Stimulating music educator worldview change through a pedagogy of provocation, critical thinking and peer-review. International Journal of Music Education, 42(1), 47-67. https://doi.org/10.1177/02557614221149223
• Isyar, O., & Akay, C. (2017). The use of drama in education in primary schools from the viewpoint of the classroom teachers: A mixed method research. Journal of Education & Practice, 8(28), 215-230.
• Jiang, J. P., Hu, J. Y., Zhang, Y. B., & Yin, X. C. (2022). Fostering college students’ critical thinking skills through peer assessment in the knowledge building community. Interactive Learning Environments, 31(10), 6480-6496. https://doi.org/10.1080/10494 820. 2022.20399 49
• Jin, Z. (2000). The learning experience of students in Middlesex University Business School (MUBS): Why do they enjoy some modules/lectures and dislike others? The International Journal of Management Education, 1(1), 22-36. https://doi.org/10.3794/ijme.11.c
• Jongsma, M. V., Scholten, D. J., van Muijlwijk-Koezen, J. E., & Meeter, M. (2023). Online versus offline peer feedback in higher education: A meta-analysis. Journal of Educational Computing Research, 61(2), 329-354. https://doi.org/10.1177/07356331221114181
• Kim, M., & Ryu, J. (2013). The development and implementation of a web-based formative peer assessment system for enhancing students’ metacognitive awareness and performance in ill-structured tasks. Educational Technology Research and Development, 61, 549-561. https://doi.org/10.1007/s11423-012-9266-1
• Linacre, J. M. (1994). Many-facet rasch model (2nd ed.). Mesa Press.
• Linacre, J. M. (1993). Rasch-based generalizability theory. Rasch Measurement Transactions, 7(1), 283-284. https://www.rasch.org/rmt/rmt71h.htm
• Lubicz-Nawrocka, T., & Bunting, K. (2019). Student perceptions of teaching excellence: An analysis of student-led teaching award nomination data. Teaching in Higher Education, 24(1), 63-80. https://doi.org/10.1080/13562517.2018.1461620
• Maley, A., & Duff, A. (2005). Drama techniques: A resource book of communication activities for language teachers. Cambridge University Press. https://doi.org/10.1017/CB09780511733079
• Mercer, S., & Dornyei, Z. (2020). Engaging language learners in contemporary classrooms. Cambridge University Press.
• Mumpuni, K. E., Priyayi, D. F., & Widoretno, S. (2022). How do students perform a peer assessment? International Journal of Instruction, 15(3), 751-766. https://doi.org/10.29333/iji.2022.15341a
• Nicholson, H. (2009). Theatre and education. Palgrave Macmillan.
• Noroozi, O., Banihashem, S. K., Biemans, H. J. A., Smits, M., Vervoort, M. T. W., & Verbaan, C.-L. (2023). Design, implementation, and evaluation of an online supported peer feedback module to enhance students’ argumentative essay quality. Education and Information Technologies, 28(10), 12757-12784. https://doi.org/10.1007/s10639-023-11683-y
• Panadero, E., & Alqassab, M. (2019). An empirical review of anonymity effects in peer assessment, peer feedback, peer review, peer evaluation and peer grading. Assessment & Evaluation in Higher Education, 44(8), 1253-1278. https://doi.org/10.1080/02602938.2019.1600186
• Panadero, E., & Broadbent, J. (2018). Developing evaluative judgement: A self-regulated learning perspective. In D. Boud, R. Ajjawi, P. Dawson, & J. Tai (Eds.), Developing Evaluative Judgement in Higher Education: Assessment for Knowing and Producing Quality Work (pp. 81-89). Routledge.
• Patchan, M. M., Schunn, C. D., & Correnti, R. J. (2016). The nature of feedback: How peer feedback features affect students’ implementation rate and quality of revisions. Journal of Educational Psychology, 108(8), 1098-1120. https://doi.org/10.1037/edu0000103
• Pinciotti, P. (1993). Creative drama and young children: The dramatic learning connection. Arts Education Policy Review, 94(6), 24-28.
• Poitras, J., Stimec, A., & Hill, K. (2013). On teaching: Fostering student engagement in negotiation role plays. Negotiation Journal, 29(4), 439-462. https://doi.org/10.1111/nejo.12036
• Prins, F. J., Sluijsmans, D. M. A., Kirschner, P. A., & Strijobs, J. W. (2005). Formative peer assessment in a CSCL environment: a case study. Assessment and Evaluation in Higher Education, 30(4), 417-444. https://doi.org/10.1080/02602930500099219
• Reddy, K., Harland, T., Wass, R., & Wald, N. (2021). Student peer review as a process of knowledge creation through dialogue. Higher Education Research & Development, 40(4), 825-837. https://doi.org/10.1080/07294360.2020.1781797
• Reinholz, D. (2016). The assessment cycle: a model for learning through peer assessment. Assessment & Evaluation in Higher Education, 41(2), 301-315. https://doi.org/10.1080/02602938.2015.1008982
• Sengül, K., & Sünbül, A. M. (2015). An investigation into comprehension competence and grammar achievements of learners of Turkish as a foreign language in Turkey. Educational Alternatives, 13, 573-580.
• Seppanen, S. (2022). What would happen if I said yes? Measuring the immediate and long-term impact of improvisation training on student teachers’ subjective, neuroendocrine and psychophysiological responses [Unpublished doctoral dissertation]. University of Helsinki.
• Smith, H., Cooper, A., & Lancaster, L. (2002). Improving the quality of undergraduate peer assessment: A case for student and staff development. Innovations in Education and Teaching International, 39(1), 71-81. https://doi.org/10.1080/13558 000110102904
• Stevens, R. (2015). Role-play and student engagement: Reflections from the classroom. Teaching in Higher Education, 20(5), 481-492. https://doi.org/10.1080/13562517.2015.1020778
• Sudweeks, R. R., Reeve, S., & Bradshaw, W. S. (2004). A comparison of generalizability theory and many-facet Rasch measurement in an analysis of college sophomore writing. Assessing Writing, 9(3), 239-261. https://doi.org/10.1016/j.asw.2004.11.001
• Thom, M. (2019). Drama for teaching and working in academia - using stagecraft to develop academics and their teaching practice in higher education. Educational Developments, 20(1), 18-22.
• Topping, K. J. (2009). Peer assessment. Theory into Practice, 48(1), 20-27. https://doi.org/10.1080/00405840802577569
• Topping, K. J. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249-276. https://doi.org/10.3102/00346543068003249
• Weng, F., Ye, S. X., & Xue, W. (2023). The effects of peer feedback on L2 students’ writing motivation: An experimental study in China. The Asia-Pacific Education Researcher, 32, 473-483. https://doi.org/10.1007/s40299-022-00669-y
• Wilson, M. J., Diao, M. M., & Huang, L. (2015). ‘I’m not here to learn how to mark someone else’s stuff’: An investigation of an online peer-to-peer review workshop tool. Assessment & Evaluation in Higher Education, 40(1), 15-32. https://doi.org/10.1080/02602938.2014.881980
• Winstone, N., & Carless, D. (2020). Designing effective feedback processes in higher education: A learning-focused approach. Routledge.
• Wright, B.D., & Linacre, J.M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370-382. https://www.rasch.org/rmt/rmt83b.htm
• Yao, C. (2017). A case study on the factors affecting Chinese adult students’ English acquisition in a blended learning environment. International Journal of Continuing Engineering Education and Life Long Learning, 27(1/2), 22-44. https://doi.org/10.1504/IJCEELL.2017.080993
• Yao, C., & Shao, S. (2024). Factors affecting teacher-student interactions in a foreign language teaching class: a language socialisation theory perspective. Humanities and Social Sciences Communications, 11, 1358. https://doi.org/10.1057/s41599-024-03900-x
• Zhan, Y. (2021). Are they ready? An investigation of university students’ difficulties in peer assessment from dual perspectives. Teaching in Higher Education, 29(4), 823-840. https://doi.org/10. 1080/13562517.2021.2021393
• Zhan, Y., Yan, Z., Wan, H. Z., Wang, X., Zeng, Y., Yang, M., & Yang, L. (2023). Effects of online peer assessment on higher-order thinking: A meta-analysis. British Journal of Educational Technology, 54(4), 817-835. https://doi.org/10.1111/bjet.13310
• Zheng, L., Chen, N. S., Cui, P., & Zhang, X. (2019). A systematic review of technology-supported peer assessment research. The International Review of Research in Open and Distributed Learning, 20(5), 168-191. https://doi.org/10.19173/irrodl.v20i5.4333