Course and Instructor Characteristics Distinguishing Highest and Lowest Student Ratings of Instructors

Abstract

Problem Statement: One way to delineate the main characteristics of effective teaching within the higher education system is to gather college students’ opinions of an effective instructor. Research based on students’ perceptions of efficient teaching revealed a series of teaching behaviors setting the distinction between good and poor teaching. However, studies also indicate differences across culture, and in Turkey, there has been little research on the topic.

Purpose of Study: The goal of this study was to determine instructor and course characteristics and teaching dimensions that discriminate between instructors who received the highest and the lowest student ratings within a Turkish college setting, by incorporating both quantitative and qualitative methods.

Method: A total of 23,814 students across different departments in the university rated 630 instructors on a scale developed to assess students’ perceptions of instructors’ performance. In addition, students were asked to respond to an open-ended question to provide their own impressions of each instructor. Then, students’ ratings were analyzed by means of discriminative functional analysis, and written statements provided by students were analyzed via content-analysis techniques by using a combination of manual and computer-assisted methods (NVivo 9).

Findings and Results: According to the quantitative analyses, although course and instructor characteristics were weak in discriminating the groups, all teaching dimensions (relationships with students, effective teaching, exams and evaluation, contribution to generic skills, class interaction, and organization and planning) were very useful in discriminating the instructors who received the best and the poorest ratings. Also, qualitative analysis revealed 4 themes consistently distinguishing the two groups: lecturing, relationship with the students, knowledge and expertise, and exams and evaluation.

Conclusions and Recommendations: This study replicates the existing literature on student perceptions of effective teaching, with a culturally different, large sample. It also adds support to the notion that there are teaching behaviors, such as lecturing skills, fair evaluations, respect and interest toward students, and demonstrating expertise, that help draw the distinction between good and poor teaching in the eyes of students and that could therefore assist the improvements efforts of teacher education.

Keywords:  Instructor effectiveness, poor teaching, college teaching, student evaluations

Problem Statement: One way to delineate the main characteristics of effective teaching within the higher education system is to gather college students’ opinions of an effective instructor. Research based on students’ perceptions of efficient teaching revealed a series of teaching behaviors setting the distinction between good and poor teaching. However, studies also indicate differences across culture, and in Turkey, there has been little research on the topic.

Purpose of Study: The goal of this study was to determine instructor and course characteristics and teaching dimensions that discriminate between instructors who received the highest and the lowest student ratings within a Turkish college setting, by incorporating both quantitative and qualitative methods.

Method: A total of 23,814 students across different departments in the university rated 630 instructors on a scale developed to assess students’ perceptions of instructors’ performance. In addition, students were asked to respond to an open-ended question to provide their own impressions of each instructor. Then, students’ ratings were analyzed by means of discriminative functional analysis, and written statements provided by students were analyzed via content-analysis techniques by using a combination of manual and computer-assisted methods (NVivo 9).

Findings and Results: According to the quantitative analyses, although course and instructor characteristics were weak in discriminating the groups, all teaching dimensions (relationships with students, effective teaching, exams and evaluation, contribution to generic skills, class interaction, and organization and planning) were very useful in discriminating the instructors who received the best and the poorest ratings. Also, qualitative analysis revealed 4 themes consistently distinguishing the two groups: lecturing, relationship with the students, knowledge and expertise, and exams and evaluation.

Conclusions and Recommendations: This study replicates the existing literature on student perceptions of effective teaching, with a culturally different, large sample. It also adds support to the notion that there are teaching behaviors, such as lecturing skills, fair evaluations, respect and interest toward students, and demonstrating expertise, that help draw the distinction between good and poor teaching in the eyes of students and that could therefore assist the improvements efforts of teacher education.

Keywords:  Instructor effectiveness, poor teaching, college teaching, student evaluations

References

1. Acker, J. R. (2003). Class acts: Outstanding college teachers and the difference they make. Criminal Justice Review, 28, 215-231.
2. Akpinar, B. & Aydin, K. (2007). Egitimde degisim ve ogretmenlerin degisim algilari, [Change in education and teachers' perceptions of change]. Egitim ve Bilim, 32 (144), 71-80.
3. Bail, F. T. & Mina, S. S. (1981).Filipino and American student perceptions of teacher effectiveness. Research in Higher Education, 14 (2), 135-145.
4. Beran, T. & Violato, C. (2005). Rating of university teacher instruction: How much do student and course characteristics really matter? Assessment and Evaluation in Higher Education, 30, 593‐601.
5. Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014). Measuring the impacts of teachers II: teacher value added and student out comes in adulthood. American Economic Review, 104 (9), 2633–2679.
6. Cashin, W. E. (1995). Student ratings of teaching: The research revisited. IDEA Paper No. 32. Manhattan, KS: Kansas State University, Center for Faculty Evaluation and Development.
7. Check, J.F. (2001). Positive traits of the effective teacher - negative traits of the ineffective one. Education, 106 (3), 326-334.
8. Chickering, A. W. & Reisser, L. (1983). Education and Identity. (2nd ed.) San Francisco: Jossey-Bass.

9. Epting, L. K., Zinn, T. E., Buskist, C., & Buskist, W. (2004).Student perspectives on the distinction between ideal and typical teachers. Teaching of Psychology, 31 (3), 181-183.
10. Feldman, K. A. (2007). Identifying exemplary teachers and teaching: Evidence from student ratings. In R. Perry & J. Smart (Eds.), The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective (pp. 93-129). Dordrecht, The Netherlands: Springer.
11. Fortson, S. B. & Brown, W. E. (1998). Best and worst university instructors: The opinions of graduate students. College Student Journal, 32, 572-576.
12. Greenwald, A. G. & Gillmore, G. M. (1997) Grading lenience is a removable contaminant of student ratings. American Psychologist, 52(11): 1209-1217.
13. Hanushek, E.A. (2002). Evidence, politics, Oxford and the class size debate, in L. Mishel & R. Rothstein (Eds.) The Class Size Debate. Washington, DC: Economic Policy Institute.
14. Johnston III, G.P., (1990). Best liked/least liked teacher attributes: Herzberg's two-factor theory of job satisfaction. Journal of Education for Business, 66 (2), 121-129.
15. Khandelwal, K. A. (2009). Effective teaching behaviors in the college classroom: a critical incidents technique from students' perspectives. International Journal of Teaching and Learning in Higher Education, 21(3), 299-309.
16. Marsh, H. W. (1977). The validity of students’ evaluations: Classroom evaluations of instructors independently nominated as best and worst teachers by graduating seniors. American Educational Research Journal, 14, 441-447.
17. Marsh, H. W. (1980). The influence of student, course and instructor characteristics on evaluations of university teaching. American Educational Research Journal, 17, 219-237.
18. Marsh, H.W. (1984). Students’ evaluation of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76, 707-754.
19. Marsh, H. W., & Dunkin, M. (1997). Student evaluation of university teaching: A multidimensional perspective. In Perry, P. R., & Smart, J. C. (Eds.), Effective Teaching in Higher Education: Research and Practice, (pp. 241-320).Agathon, New York.
20. Marsh, H. W. & Roche, L.A (1997). Making students’ evaluations of teaching effectiveness effective. American Psychologist, 52, 1187-1197.
21. Marsh, H.W. & Roche, L.A. (1999). Rely upon SET research. American Psychologist, 54, 517–518.
22. Marsh, H. W., & Roche, L. A. (2000). Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, and innocent bystanders. Journal of Educational Psychology, 92, 202-22.
23. Miller, J. L., Dzindolet, M. T., Weinstein, L., Xie, X., & Stones, C. R. (2001). Faculty and students views of teaching effectiveness in the United States, China and South Africa. Teaching of Psychology, 28(2), 138-142.
24. Okpala, C.O. & Ellis, R. R. (2005). The perceptions of college students on teacher quality: A focus on teacher qualifications. Education. 126 (2), 374-383.
25. Patrick, C.L. (2011). Student evaluations of teaching: Effects of the big five personality traits, grades, and the validity hypothesis. Assessment & Evaluation in Higher Education, 36 (2), 239-249.
26. Pozo-Muñoz, C., Rebolloso-Pacheco, E., & Fernández-Ramírez, B. (2000).The 'Ideal Teacher' implications for student evaluation of teacher effectiveness. Assessment & Evaluation in Higher Education, 24(1), 253-263.
27. Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16, 129-50.
28. Slate, J., LaPrairie, K. N., Schulte, D. P., & Onwuegbuzie, A. J. (2011).Views of effective college faculty: a mixed analysis. Assessment & Evaluation in Higher Education, 36 (3), 331-346.
29. Tunca, N., Alkin-Sahin, S., Oguz, A. & Bahar-Guner, H. O. (2015). Qualities of Ideal Teacher Educators. Turkish Online Journal of Qualitative Inquiry, 6(2), 123-148.
30. Young, S. & Shaw, D. G. (1999). Profiles of effective college and university teachers. The Journal of Higher Education, 70 (6), 670-686.
31. Watkins, D. & Akande, A. (1992). Student evaluations of teaching effectiveness: A Nigerian investigation. Higher Education, 24, 453-463.
32. Zhang, S., Fike, D., & DeJesus, G. (2015). Qualities university students seek in a Teacher. Journal of Economics and Economic Education Research ,16 (1), 42-54.