Assessing feedback practices in higher education: Development and validation of the Perceived Quality of Feedback Scale

Abstract

Despite the importance of feedback quality in educational contexts, the availability of comprehensive, valid, and reliable tools for assessing feedback quality remains limited. The objective of this study was to develop a valid and reliable measurement tool for evaluating the perceived quality of feedback in a comprehensive manner. The study was conducted with the participation of 847 pre-service teachers from a range of grade levels and teacher education programmes (322 second-year, 272 third-year, 253 fourth-year; 52.18% female, 47.82% male) studying at a public university's Faculty of Education. Participants were selected from various teaching programs. The scale was developed in three stages: item development, scale development, and scale evaluation. Exploratory and confirmatory factor analyses were performed to examine the factor structure of the assessment tool. Exploratory factor analysis (EFA) was conducted with data collected from 531 undergraduate students, followed by confirmatory factor analysis (CFA) with data from another group of 316 undergraduate students. The EFA results revealed a five-factor structure consistent with the theoretical framework: cognitive orientation, metacognitive orientation, affective orientation, quality of presentation, and quality of content. These five factors explained 52.53% of the total variance. The CFA demonstrated that the model's fit indices exceeded the acceptable thresholds (CFI=.95, RMSEA=.05, TLI=.94). The reliability analyses showed that the Cronbach's Alpha (.78 - .93) and McDonald's Omega (.77 - .93) coefficients for the scale dimensions were within acceptable ranges, confirming the scale's reliability. Consequently, a valid and reliable 25-item scale with five dimensions was developed to measure the perceived quality of feedback.

Keywords

• Feedback quality
• Functions of feedback
• Content of feedback
• Presentation of feedback
• Scale development

References

1. Akbulut, Y., Saykılı, A., Öztürk, A., & Bozkurt, A. (2023). What if it’s all an illusion? To what extent can we rely on self-reported data in open, online, and distance education systems? The International Review of Research in Open and Distributed Learning, 24(3), 1-17. https://doi.org/10.19173/irrodl.v24i3.7321
2. Akkuzu, N., & Uyulgan, M.A. (2014). Toward making the invisible visible using a scale: Prospective teachers’ thoughts and affective reactions to feedback. Irish Educational Studies, 33(3), 287–305. https://doi.org/10.1080/03323315.2014.923184
3. Baydas Onlu, O., Abdusselam, M.S., & Yilmaz, R.M. (2022). Students’ perception of instructional feedback scale: Validity and reliability study. Contemporary Educational Technology, 14(3), Article ep368. https://doi.org/10.30935/cedtech/11811
4. Beydogan, H.Ö. (2016). Feedback correction perception scale for teacher candidates. Ahi Evran University Journal of Kırşehir Education Faculty, 17(2), 297–314.
5. Boateng, G.O., Neilands, T.B., Frongillo, E.A., Melgar‐Quiñonez, H., & Young, S.L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, Article 149. https://doi.org/10.3389/fpubh.2018.00149
6. Brookhart, S.M. (2008). How to give effective feedback to your students. Association for Supervision and Curriculum Development.
7. Byrne, B.M. (1994). Structural equation modeling with EQS and EQS/Windows. Sage.
8. Costello, A.B., & Osborne, J.W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(1), Article 7. https://doi.org/10.7275/jyj1-4868

9. Deeley, S.J., Fischbacher-Smith, M., Karadzhov, D., & Koristashevskaya, E. (2019). Exploring the ‘wicked’ problem of student dissatisfaction with assessment and feedback in higher education. Higher Education Pedagogies, 4(1), 385 405. https://doi.org/10.1080/23752696.2019.1644659
10. Deci, E.L., & Ryan, R.M. (1985). Intrinsic motivation and self-determination in human behavior. Springer Science & Business Media. https://doi.org/10.1007/978-1-4899-2271-7
11. Fabrigar, L.R., Wegener, D.T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272
12. Flora, D.B., LaBrish, C., & Chalmers, R.P. (2012). Old and new ideas for data screening and assumption testing for exploratory and confirmatory factor analysis. Frontiers in Psychology, 3, Article 55. https://doi.org/10.3389/fpsyg.2012.00055
13. Fornell, C., & Larcker, D.F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
14. George, D., & Mallery, M. (2010). SPSS for Windows step by step: A simple guide and reference, 17.0 update (10th ed.) Pearson.
15. Glazzard, J., & Stones, S. (2019). Student perceptions of feedback in higher education. International Journal of Learning, Teaching and Educational Research, 18(11), 38-52. https://doi.org/10.26803/ijlter.18.11.3
16. Hair, J.F., Black, W.C., Babin, B.J. and Anderson, R.E. (2010) Multivariate Data Analysis (7th ed.). Pearson.
17. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
18. Hayton, J.C., Allen, D.G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7(2), 191–205. https://doi.org/10.1177/1094428104263675
19. Henderson, M., Ryan, T., & Phillips, M. (2019). The challenges of feedback in higher education. Assessment & Evaluation in Higher Education, 44(8), 1237–1252. https://doi.org/10.1080/02602938.2019.1599815
20. Henseler, J., Ringle, C.M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8
21. Holstun, V., Rigsbee, N., & Bohecker, L. (2021). Encouragement is not enough: perceptions and attitudes towards corrective feedback and their relationship to self-efficacy. Teaching and Supervision in Counseling, 3(3), Article 2. https://doi.org/10.7290/tsc030302
22. Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
23. Jöreskog, K.G., & Sörbom, D. (1984). LISREL VI user’s guide (3rd ed.). Scientific Software.
24. Erdem Kara, B., & Akbulut, Y. (2025). Prevalence and psychometric implications of careless responses in an online student survey. Journal of Psychoeducational Assessment, 43(5), 467-487. https://doi.org/10.1177/07342829251328132
25. Kara, F.M., Kazak, F.Z., & Asci, F.H. (2018). Perceived Teacher Feedback Scale: The validity and reliability study. Hacettepe Journal of Sport Sciences, 29(2), 79 86. https://doi.org/10.17644/sbd.306544
26. Kartol, A., & Arslan, N. (2021). Turkish version of the Feedback Orientation Scale: Investigation of psychometric properties. International Journal of Turkish Literature Culture Education, 10(1), 321–329. https://doi.org/10.7884/teke.5129
27. Kılıç, A.F. (2022). Deciding the number of dimensions in explanatory factor analysis: A brief overview of the methods. Pamukkale University Journal of Social Sciences Institute, 51(1), 305–318.
28. Kline, P. (1994). An easy guide to factor analysis. Routledge.
29. Kline, R.B. (2011). Principles and practice of structural equation modeling (3rd ed.). Guilford Press.
30. Kline, R.B. (2016). Principles and practice of structural equation modeling (4th ed.). Guilford Press.
31. Koka, A., & Hein, V. (2003). Perceptions of teacher’s feedback and learning environment as predictors of intrinsic motivation in physical education. Psychology of Sport and Exercise, 4(4), 333–346. https://doi.org/10.1016/S1469-0292(02)00012-2
32. Kumral, O., & Saracaloğlu, A.S. (2011). An evaluation of elementary school teacher’s teaching profession courses programme via educational criticism model. Education Sciences, 6(1), 106-118.
33. Lawshe, C.H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575.
34. London, M., & Smither, J.W. (2002). Feedback orientation, feedback culture, and the longitudinal performance management process. Human Resource Management Review, 12(1), 81-100.
35. MacCallum, R.C., Browne, M.W., & Sugawara, H.M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130-149. https://doi.org/10.1037/1082-989X.1.2.130
36. Mulliner, E., & Tucker, M. (2017). Feedback on feedback practice: perceptions of students and academics. Assessment & Evaluation in Higher Education, 42(2), 266-288. https://doi.org/10.1080/02602938.2015.1103365
37. Narciss, S., & Huth, K. (2004). How to design informative feedback for multimedia learning. In Niegemann, R. Brünken, & D. Leutner (Eds.), Instructional design for multimedia learning. Waxmann.
38. Nugraheny, E., Claramita, M., Rahayu, G.R., & Kumara, A. (2016). Feedback in the nonshifting context of the midwifery clinical education in Indonesia: A mixed methods study. Iranian Journal of Nursing and Midwifery Research, 21(6), 628. https://doi.org/10.4103/1735-9066.197671
39. Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
40. O’Donovan, B., Rust, C., & Price, M. (2016). A scholarly approach to solving the feedback dilemma in practice. Assessment & Evaluation in Higher Education, 41(6), 938-949. https://doi.org/10.1080/02602938.2015.1052774
41. Ocak, G., & Karafil, B. (2020). Development of Teacher Feedback Use Evaluation Scale. International Journal of Progressive Education, 16(1), 287 299. https://doi.org/10.29329/ijpe.2020.228.20
42. Ostaeyen, S.V., Embo, M., Rotsaert, T., Clercq, O.D., Schellens, T., & Valcke, M. (2023). A qualitative textual analysis of feedback comments in e-portfolios: Quality and alignment with the CanMEDS roles. Perspectives on Medical Education, 12(1). https://doi.org/10.5334/pme.1050
43. Öntaş, T., & Kaya, B. (2019). Investigation of the opinions of pre-service primary school teachers to give feedback in preparation of teaching materials. Journal of Milli Eğitim, 48(224), 59–73.
44. Pallant, J. (2016). SPSS survival manual: A step by step guide to data analysis using SPSS program (6th ed.). McGraw-Hill Education.
45. Panadero, E., & Lipnevich, A.A. (2022). A review of feedback models and typologies: Towards an integrative model of feedback elements. Educational Research Review, 35, Article 100416. https://doi.org/10.1016/j.edurev.2021.100416
46. Prilop, C.N., Weber, K.E., Prins, F.J., & Kleinknecht, M. (2021). Connecting feedback to self-efficacy: Receiving and providing peer feedback in teacher education. Studies in Educational Evaluation, 70, Article 101062. https://doi.org/10.1016/j.stueduc.2021.101062
47. Revelle, W. (2019). Psych: Procedures for psychological, psychometric, and personality research. Northwestern University. https://CRAN.R-project.org/package=psych
48. Sattora, A., & Bentler, P.M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514. https://doi.org/10.1007/BF02296192
49. Schumacker, R.E., & Lomax, R.G. (2004). A beginner’s guide to structural equation modeling (2nd ed.). Erlbaum.
50. Shute, V.J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189. https://doi.org/10.3102/0034654307313795
51. Silverstein, A.B. (1987). Note on the parallel analysis criterion for determining the number of common factors or principal components. Psychological Reports, 61(2), 351-354. https://doi.org/10.2466/pr0.1987.61.2.351
52. Şahin, M. (2015). Investigation of prospective teachers’ opinions about the feedback activity used in teaching and learning process. Bolu Avant Izzet Baysal University Journal of Faculty of Education, 15(USBES Special Issue I), 247–264.
53. Tabachnick, B.G., & Fidell, L.S. (2007). Using multivariate statistics (5th ed.). Allyn & Bacon/Pearson Education.
54. Tabachnick, B.G., & Fidell, L.S. (2013). Using multivariate statistics (6th ed.). Pearson.
55. Williams, A. (2024). Delivering effective student feedback in higher education: An evaluation of the challenges and best practice. International Journal of Research in Education and Science (IJRES), 10(2), 473-501. https://doi.org/10.46328/ijres.3404
56. Winstone, N., Nash, R.A., Rowntree, J., & Parker, M. (2017). ‘It'd be useful, but i wouldn't use it’: barriers to university students’ feedback seeking and recipience. Studies in Higher Education, 42(11), 2026-2041. https://doi.org/10.1080/03075079.2015.1130032
57. Yang, L.; Sin, K.; Li, X.; Guo, J. & Lui, M. (2014). Understanding the power of feedback in education: A validation study of the Feedback Orientation Scale (FOS) in classrooms. The International Journal of Educational and Psychological Assessment, 16(1), 21-46.
58. Yuan, K.-H., & Bentler, P.M. (1998). Robust mean and covariance structure analysis. British Journal of Mathematical and Statistical Psychology, 51(1), 63 88. https://doi.org/10.1111/j.2044-8317.1998.tb00667.x
59. Yuan, K.-H., & Bentler, P.M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological Methodology, 30(1), 165–200. https://doi.org/10.1111/0081-1750.00078
60. Zinbarg, R.E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s alpha, Revelle’s beta, and McDonald’s omega: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123–133. https://doi.org/10.1007/s11336-003-0974-7
61. Zhou, M., & Winne, P.H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413 419. https://doi.org/10.1016/j.learninstruc.2012.03.004
62. Zwick, W.R., & Velicer, W.F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99(3), 432–442. https://doi.org/10.1037/0033-2909.99.3.432