students’ Evaluation of Teaching in Higher Education: Effects of Students’ Perception on Ratings

Abstract

This research aims at identifying the situations influential over the ratings given by students to the items on questionnaires employed to evaluate instructors and teaching in higher education institutions as well as reasons for giving the ratings, the process of initial comments of the items on first reading and choosing the appropri-ate rate, and the current research also aims at analyzing the relations between the reasons. The research sample was composed of the junior students of Elementary Education Class Teaching, Pre-School Education and Social Sciences Teacher Education branches of Primary School Teaching Department of Gazi Teacher Training College of Gazi University. Research data were gathered in two successive semesters in the same academic year. The same tool of data collection was em-ployed in both of these two semesters, and the content analysis of the data was conducted. Following a content analysis of the reasons for attributing rates, findings were obtained, and a matrix for each item was formed. The fact that reasons for rating were various and too many, that students gave rates even when they were not sure, that overlapping items caused indecisiveness in rating, that they gave rates even to those items of which they had no knowledge, that they rated higher to those instructors who gave them the marks they had expected, that there were contrasts among the reasons for the items which are clearly agreed upon, and that a different halo effect emerged in this research was of importance in this research.

Keywords

• assessment of instruction in higher education , instructional assessment scales , assessment of instruction by students

Yükseköğretimde Öğretimi Değerlendirme Anketlerinde Öğrenci Algılarındaki Farklılıkların Puanlamaya Etkileri

Öz

Bu çalışma, yükseköğretim kurumlarında kullanılan öğretim elemanlarını ve öğretimi değerlendirme anketlerindeki maddelere öğrenciler tarafından verilen puanların, puanlara verilen nedenlerin, nedenler arasındaki ilişkilerin analizinin yapılmasını ve öğrencilerin maddeleri okuduklarında, maddeler hakkındaki ilk yorumlarından, puanların birinin seçimine karar verme sürecinde etkili olan durumları saptamayı amaçlamıştır. Örneklem Gazi Üniversitesi Gazi Eğitim Fakültesi İlköğretim Bölümü Sınıf Öğretmenliği, Okul Öncesi Eğitimi ve Sosyal Bilgiler Öğretmenliği Anabilim Dalı 3. sınıf öğrencilerinden oluşmuştur. Çalışmanın verileri aynı yıl içerisinde ardışık olarak iki farklı dönemde toplanmıştır. Bu iki ayrı dönemde yürütülen araştırmada aynı veri toplama aracı kullanılarak veriler içerik analizi ile çözümlenmiştir. Bulgular puanlar için verilen nedenlerin (yorumların) içerik analizi yapılması sonucu elde edilmiş ve her bir madde için matris oluşturulmuştur. Ankette her madde için puan verme nedenlerinin farklılığı ve fazlalığı, öğrencilerin hatırlayamadıkları ve emin olmadıkları durumlarda bile puan verdiği, maddedeki binişikliğin, puanlamada kararsızlığa neden olduğu, öğrencilerin bilgileri olmadığını belirtikleri maddelere puan verdikleri, öğrenciler sınavlarda bekledikleri notu aldıklarında öğretim elemanına yüksek not verdiği, açık olarak kabul edilen maddelerde bile nedenler arasında zıtlıkların olduğu ve diğer araştırmalardan daha farklı hale etkisinin ortaya çıkması bu çalışmanın dikkat çekici sonuçlarıdır. Sonuçlar kötü yapılandırılmış problem çözme süreçleri temele alınarak yorumlanmıştır.

Anahtar Kelimeler

• yükseköğretimde öğretimi değerlendirme
• öğretimi değerlendirme anketi
• öğretimin öğrenciler tarafından değerlendirilmesi

Kaynakça

1. Abrami, P.C & D. Mizener. (1983). “Does the attitude similarity of college professors and their students produce ‘bias’ in course evaluations?” American Educational Research Journal, 20, 123-136.
2. Abrami, P. C. (1989). How should we use student ratings to evaluate teaching? Research in Higher Education. 30, 221-227.
3. Abrami, P. C. (1989). SEEQing the truth about student ratings of instruction. Educational Researcher. 18 (1), 43-45.
4. Abrami, P. C., Perry, R. P., & Leventhal, L. (1982). The relationship between student personality characteristics, teacher ratings and student achievement. Journal of Educational Psychology, 74 (1), 111-125.
5. Abrami, P.C. & d'Apollonia, S. (1991). Multidimensional students' evaluations of teaching effectiveness; the generalizability of "n = 1" research: comment on marsh (1991). Journal of Educational Psychology. 83, 411-415.
6. Alderson, J. C. (1992). Validating questionnaires. centre for research in language education. Working Papers No. 15. University of Lancaster.
7. Aleamoni, L. M. (1981). Student ratings of instruction. In J. Millman (Ed.), Handbook of teacher evaluation. (pp. 110-145). Beverly Hills, California: Sage.
8. Bedggood, R.E., & Polard, R.J. (1999). Uses and misuses of student opinion surveys in eight Australian Universities. Australian Journal of Education, 43 (2), 129-156.

9. Block, D. (1998). Exploring interpretations of questionnaire items. System, 26 (3), 403-425.
10. Braskamp, L. A., & Ory, J. C. (1994). Assessing faculty work: enhancing individual and institutional performance. San Francisco: Jossey-Bass.
11. Braskamp, L. A., Ory, J. C. & Pieper, D. M. (1981). Student written comments: dimensions of instructional quality, Journal of Educational Psychology, 73, 65-70.
12. Cashin, W.E. (1990). Student rating of teaching: Recommendations for use. IDEA Paper no 22. Kansas State University,Center for Faculty Evaluation & Development.
13. Cashin, W. E. (1996). Developing an effective faculty evaluation system, IDEA Paper no 33. Kansas State University,Center for Faculty Evaluation & Development.
14. Cashin,W. E. (1995). Student Ratings of Teaching: A summary of The Research. IDEA Paper no. Kansas State University, Center for Faculty Evaluation & Development.
15. Centra, J. A. (1993). Reflective faculty evaluation: enhancing teaching and determining faculty effectiveness. San Francisco: Jossey-Bass.
16. Centra, J. A. & Creech, F. R. (1976). The relationship between student, teachers, and course characteristics and student ratings of teacher effectiveness. Project Report 76- 1, Educational Testing Service, Princeton, NJ.
17. Chen, Y., Hoshower, L. B. (2003). Student Evaluation of Teaching Effectiveness: an assessment of student perception and motivation, Assessment & Evaluation in Higher Education, 28 (1), 71-88.
18. Clouder, L. (1988). Getting The ‘Right Answer’: Student Evaluation As A Reflection On Intellectual Development, Teaching in Higher Education, 3 (2), 185-196.
19. Cohen, P. A. (1981). Student ratings of instruction and student achievement: a meta-analysis of multisection validity studies. Review of Educational Research, 51, 281-309.
20. Damron, J. C. (1995). The three faces of teaching evaluation, Unpublished Manuscript, Douglas College, New Westminster, British Columbia.
21. DeCanio, S. J. (1986). Student evaluations of teaching – a multinomial logit approach, Journal of Economic Education. 17, 165-175.
22. Feldman, K. A. (1976). The superior college teacher from the student’s view, Research in Higher Education, 5, 243-288.
23. Feldman, K. A. (1977). Consistency and variability among college students in rating their teachers and courses: a review and analysis. Research in Higher Education, 6 (3), 223-274.
24. Feldman, K. A. (1978). “Course characteristics and college students ratings of their teachers: what we know and what we don’t”. Research in Higher Education, 9, 199-242.
25. Feldman, K. A. (1987). Research productivity and scholarly accomplishment of college teachers as related to their instructional effectiveness: a review and exploration. Research in Higher Education, 26 (3), 227-298.
26. Feldman, K. A. (1989a). The Association between student ratings of specific instructional dimensions and student achievement: refining and extending the synthesis of data from multisection validity studies. Research in Higher Education, 30 (6), 583-645.
27. Feldman, K. A. (1989b). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators and external (neutral) observers. Research in Higher Education, 30, 583-645.
28. Feldman, K. A. (1993). College students' views of male and female college teachers: part ii- evidence from students' evaluations of their classroom teachers. Research in Higher Education, 34 (2), 151-211.
29. Felton, J., Mitchell, J., & Stinson, M. (2004) Web-based student evaluation of professors: therelations between perceived quality, easiness and sexiness, Assessment and Evaluation in Higher Education, 29 (1), 91-108.
30. Gick, M. L. (1986). Problem Solving Strategies. Educational Psychologist, 21 (1&2), 99-120.
31. Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Ground Theory. Chicago: Aldine.
32. Gramlich, E. M. and Greenlee, G. A. (1993). Measuring Teaching Performance, Journal of Economic Education, 24, 3-13.
33. Grant, H. (1998). Academic contests: Merit pay in Canadian universities, Relations Industrially / Industrial Relations 53 (4), 647-664.
34. Green, B. P., Caleron, T. G.& Reider B. P. (1998). A Content analysis of teaching evaluation instruments used in accounting departments, Issues in Accounting Education, 13 (1), 15–30.
35. Haskell, R. E. (1997a). Academic freedom, tenure, and student evaluations of faculty: galloping polls in the 21st century, Education Policy Analysis Archives 5 (6), February 12.
36. Haskell, R. E. (1997b). Academic Freedom, Promotion, Reappointment, Tenure, and The Administrative Use of Student Evaluation of Faculty (SEF): (Part II) Views from court, Education Policy Analysis Archives 5 (6), August 25.
37. Haskell, R. E. (1997c). Academic Freedom, Promotion, Reappointment, Tenure, and The Administrative Use of Student Evaluation of Faculty (SEF): (Part III) Analysis and implications of views from the court in relation to accuracy and psychometric validity, Education Policy Analysis Archives 5 (6), August 25.
38. Haskell, R. E. (1997d). Academic Freedom, Promotion, Reappointment, Tenure, and The Administrative Use of Student Evaluation of Faculty (SEF): (Part IV) Analysis and implications of views from the court in relation to academic freedom, standards, and quality of instruction, Education Policy Analysis Archives 5 (6), November 25.
39. Haynes, B. (2002). The use of student evaluations as an indicator of teaching quality in higher education. Paper presented at the Conference of Australians Association for Research in Education, Brisbane.
40. Hobson, S. M. & Talbot, D. M. (2001). Understanding student evaluations, College Teaching, 49 (1), 26–31.
41. Hoffman, F. E. & Kremer, L. (1980). Attitudes toward higher education and course evaluation, Journal of Educational Psychology, 72 (5), 610–617.
42. Hong, S. N. (1998). The Relationship between Well- Structured and IllStructured Problem Solving in Multimedia Simulation. Unpublished Doctoral Thesis. The Pennsylvania State University, The Graduate School College of Education.
43. Howard, G. S., & Maxwell, S. E. (1980). Correlation between student satisfaction and grades: a case of mistaken causation? Journal of Educational Psychology, 72, 810-820.
44. Howard, G. S., & Maxwell, S. E. (1982). Do grades contaminate student evaluations of instruction? Research in Higher Education, 16,175-188.
45. Husbands, C. T. (1996). Variations in students’ evaluations of teachers’ lecturing and small group teaching: A study at the London school of economics and political science. Higher Education, 33(1), 51-70.
46. Husbands, C. T. & Fosh, P. (1993). student evaluations of teaching in higher education: Experiences from four European countries and some implications of the practice. Assessment and Evaluation in Higher Education, 18(2), 95-114.
47. Johnson, R. (2000). The authority of the student evaluation questionnaire. Teaching in Higher Education; 5 (4), 419-434.
48. Jonassen, D. H. & Grabowski, B. L. (1993). Handbook of Individual Differences, Learning and Instruction. Hillsdale, NJ: Lawrence Erlbaum Associates.
49. Jonassen, D. H. & Tessmer, M. (1996/1997). An outcomes-based taxonomy for instructional systems design, evaluation, and research. Training Research Journal, 2, 11-46.
50. Jonassen, D. H. (1997).Instructional design model for well-structured and illstructured problem-solving learning outcomes. Educational Technology: Research and Development. 45 (1), 65-95.
51. Jonassen, D. H. (2000a). Integrating Problem Solving Into Instructional Design. in R.A. Reiser & J. Dempsey (Eds.), Trends and issues ion instructional design and technology. Upper Saddle River, NJ: Prentice-Hall.
52. Jonassen, D. H. (2000b). Towards a design theory of problem solving. Educational Technology Research and Development 48 (4), 63-85.
53. Kember, D. & Wong A, (2000). Implication for evaluation from a study of students’ perceptions of good and poor teaching, Higher Education, 40 (1), 69-97.
54. Kemp, B. and G.Kumar (1990). Student evaluations: are we using them correctly, Journal of Education for Business, 65, 106-111.
55. Kitchener, K. S. (1983). Cognition, metacognition, and epidemic cognition: a tree-level model of cognitive processing, Human Development, 4, 222-232.
56. Kolitch, E & Dean, A. V. (1999). Students’ ratings of instruction in the usa: hidden assumptions and missing conceptions about ‘good’ teaching, Studies in Higher Education. 24 (1), 27-42.
57. Kulik, J. & McKeachie, W. (1975). The Evaluation of Teachers in Higher Education. Ed. Kerlinger, F. (Ed). Review of Research in Education. Itasca: IL: F.E. Peacock.
58. LaForge, M. C. (2003). Student mood and teaching evaluation ratings. Journal of The Academy of Business Education, 4.
59. Lechtreck, R. (1990). College Faculty Evaluation By Students - an Opportunity For Bias. College Student Journal, 24 (September), 297-299.
60. Lewis. K (1991). Making sense of student written comments, New Direction for Teaching and Learning, 87, 25-32.
61. Low, G. (1988). The semantics of questionnaires rating scales’ Evaluation and Research in Education, 2, 69-79.
62. Low, G. (1991).Talking to Questionnaires Pragmatic Models in Questionnaire Design. In B Heaton, P Adams and P. Howarth (eds). Sociocultural aspect of english for academic purposes. London, Macmillan.
63. Low, G. (1996). Intensifiers and hedges questionnaire items and the lexical invisibility hypothesis, Applied Linguistics, 17 (1), 1-36.
64. Low, G. (1999). What respondents do with questionnaires: accounting for incongruity and fluidity. Applied Linguistics, 20 (4), 503-533.
65. Marsh, H. W. (1984). Students' evaluations of university teaching: dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76, 707-754.
66. Marsh, H. W. (1987). Students' evaluations of university teaching: research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11 (3), 253-388.
67. Marsh, H. W., & Bailey, M. (1993). Multidimensional students’ evaluations of teaching effectiveness. Journal of Higher Education, 64(1), 1-18.
68. Marsh, H. W., & Dunkin, M. (1992). Students' evaluations of university teaching: a multi-dimensional perspective. In J. C. Smart (Ed.), Higher Education: Handbook on Theory and Research, New York: Agathon Press, pp. 143-234.
69. Marsh, H. W. & Roche, L. A. (1997). Making students’ evaluations of teaching effectiveness effective, American Psychologist, 52 (11), 1187-1197.
70. Martinson, D. L. (2000). Student evaluations of teaching and their short term validity. Journalism and Mass Communication Educator, 54 (4), 77-82.
71. McKeachie, W. (1997). Students ratings: the validity of use, American Psychologist, 52 (11), 1218-1225.
72. McKeachie, W. J. (1979). Student ratings of faculty: a reprise. Academe: Bulletin of the AAUP, 65(6), 384-397.
73. Murray, H. G. (1983). Low-inference classroom teaching behaviors and student ratings of college teaching effectiveness. Journal of Educational Psychology, 75 (1), 138-149.
74. Neumann, R.(2000). Communicating student evaluations of teaching Assessment and Evaluation in Higher Education, 25 (2), 121-134.
75. Newell, A. & Simon, H. A. (1972). Human Problem Solving (Englewood Cliffs, NJ; Prentice Hall).
76. Platt, M. (1993). What student evaluations teach, Perspectives in Political Science 22 (1), 29-40.
77. Polson, P. & Jeffries, R.(1985). Instruction in problem- solving skills: an analysis of four approaches. In J. W. Segal, S. F. Chipmen, & R. Glaser (eds.), Thinking and learning skills. 1,417-455. Hillsdale, NJ: Lawrence Erlbaum.
78. Remmers, H.H. (1927). The Purdue rating scale for instructors. Educational Administration and Supervision,6,399-406.
79. Remmers, H.H. (1928).The relationship between students’ marks and students’ attitudes toward instructors. School and Society, 28,759-60.
80. Remmers, H.H. (1930). To what extent do grades influence student ratings of instructors? Journal of Educational Psychology, 21, 314-16.
81. Remmers, H.H., & Brandenburg, G.C. (1927). "Experimental data on the Purdue rating scale for instructors." Educational Administration and Supervision, 13, 519-27.
82. Remmers, H.H., Martin F.D., & Elliot D.N. (1949). Are students ratings of their instructors related to their grades? Purdue University Studies in Higher Education. 44,17-26.
83. Robertson, I. (2004). Student perceptions of student perception of module questionnaires: questionnaire completion as problem solving, Assessment and Evaluation in Higher Education, 29 (6), 662-669.
84. Seldin, P. (1985). Current practices in evaluating business school faculty. Pleasantville, NY: Center for Applies Research, Lubin School of Business Administration, Pace University.
85. Shevlin, M., Banyard,P., Davies,M., & Griffiths,M. (2000). The validity of student evaluation of teaching in higher education: Love me love my lectures? Assessment and Evaluation in Higher Education,25(4), 397- 405.
86. Sinnott, J. D. (1989). A model for solution of ill-structured problems: implications for everyday and abstract problem solving. In J. D. Sinnott (Ed.), Everyday problem solving:Ttheory and applications, 72-99. New York: Praeger.
87. Sixbury, G. R., & Cashin, W. E. (1995a). IDEA technical report no. 9: Description of database for the IDEA Diagnostic Form. Manhattan: Kansas State University, Center for Faculty Evaluation and Development.
88. Sixbury, G. R., & Cashin, W. E. (1995b). IDEA technical report no. 10: Comparative data by academic field. Manhattan: Kansas State University, Center for Faculty Evaluation and Development.
89. Solas, J. (1990). Effective teaching as construed by social work students. Journal of Social Work Education, 26, 145-154.
90. Tagomori, H. & Bishop, L. (1995). Student evaluation of teaching: flaws in the instruments, Thought and Action, 11, 63-78.
91. Van Manen, M. (1990). Researching lived experience: human science for an action based pedagogy. London, Ont: The Althouse Press.
92. Voss, J. F., & Post, T. A. (1988). On the solving of ill-structured problems. In M.T.H. Chi, R. Glaser and M.J Farr (Eds), The nature of expertise (261285). Hillsdale, NJ:Lawrence Erlbaum Associates.
93. Wagenaar, T. A. (1995). Student evaluation of teaching: some cautions and suggestions. Teaching Sociology, 23 (l), 64-8.
94. Wagner, Z.M.(1999). Using student journals for course evaluation. Assessment and Evaluation in Higher Education, 24(3),261-272.
95. Wetzstein, M. E., Broder J. M., & Wilson G. (1984). Bayesian Inference and student evaluations of teachers and courses. Journal of Economic Education 15 (Winter): 40-45.
96. Whetley, G. H. (1984). Problem solving in school mathematics, MEPS Technical Report no. 84.01 (West Lafeyette, in, Purdue University School of Mathematics and Science Center).
97. Wigington, H., Tollefson, N., & Rodriguez, E. (1989). Students' ratings of instructors revisited: interactions among class and instructor variables. Research in Higher Education, 30 (3), 331-44.
98. Worthington,A.C(2002). The impact of student perceptions and characteristics on teaching evaluations: a case study in finance education. Assessment and Evaluation in Higher Education,27(1), 49-64.
99. Wood, P. K. (1983). Aspects of problem solving: an inquiry systems perspective of decision theory. Human. Development.