RELIABILITY OF INTERVIEWS USED FOR ADMISSION TO TEACHER EDUCATION PROGRAMS IN NORTHERN CYPRUS: GENERALIZABILITY THEORY

Year 2024, , 67 - 91, 25.06.2024

Hasan Özder , Aygil Takır

https://doi.org/10.48166/ejaes.1415158

Abstract

This study examined the reliability of raters' assessments of prospective teachers for admission to a teacher training college. A total of fifty-eight prospective teachers and 10 raters participated in the study. The raters gave their ratings according to five dimensions, namely general culture (GC), language (L), self-image (SI), hobbies (H) and attitude towards the teaching profession (ATTP) in two interviews (Interview 1 and Interview 2). The data were analysed using the Mann-Whitney U-test and Generalizability Theory. The results of the study show that there is no significant difference between the raters' ratings in each interview. The G coefficients were unacceptable for the GC and SI dimensions in Interview 1. In addition, the G coefficients were relatively low for the L, H, and ATTP dimensions in Interview 1 and all dimensions in Interview 2.

Keywords

Admission to teacher education programs, interview, reliability, generalizability theory

References

• Andrew, M. D., Cobb, C. D., & Giampietro, P. J. (2005). Verbal ability and teacher effectiveness. Journal of teacher education, 56(4), 343-354. https://doi.org/10.1177/0022487105279928
• Arnon, S. and Reichel, N. (2007). Who is the ideal teacher? Am I? Similarity and differences in perception of students of education regarding the qualities of a good teacher and their qualities as teachers. Teachers and Teaching: Theory and Practice, 13 (5), 441-446. https://doi.org/10.1080/13540600701561653
• Atılgan, H. (2008). Using generalizability theory to assess the score reliability of the special ability selection examinations for music education programmes in higher education. International Journal of Research & Method in Education, 31(1), 63-76. https://doi.org/10.1080/17437270801919925
• Atılgan, H. & Tezbaşaran, A. A. (2005). An Investigation on consistency of g and phi coefficients obtained by generalizability theory alternative decisions study for scenarios and actual cases. Eurasian Journal of Educational Research, 18, 28-40.
• Bardach, L. & Klassen, R. M. (2020). Smart teachers, successful students? A systematic review of the literature on teachers’ cognitive abilities and teacher effectiveness. Educational Research Review, 30, 100312. Advance Online Publication. https://doi.org/10.1016/j.edurev.2020.100312
• Brennan, R. (1983). Elements of generalizability theory. Iowa City, IA: American College Testing.
• Brennan, R. L. (2001) Generalizability theory. New York: Springer-Verlag.
• Briesch, A. M., Swaminathan, H., Welsh, M., & Chafouleas, S. M. (2014). Generalizability theory: A practical guide to study design, implementation, and interpretation. Journal of School Psychology, 52(1), 13-35. https://doi.org/10.1016/j.jsp.2013.11.008
• Byrnes, D., Kiger, G., & Shechtman, Z. (2003). Evaluating the use of group interviews to select students for teacher-education programs. Journal of Teacher Education, 54(2), 163-172. https://doi.org/10.1177/0022487102250310
• Cardinet, J., Johnson, S. & Pini, G. (2011). Applying generalizability theory using EduG, New York: Taylor and Francis.
• Casey, C., & Childs, R. (2007). Teacher education program admission criteria and what beginning teachers need to know to be successful teachers. Canadian Journal of Educational Administration and Policy, (67).
• Caskey, M. M., Peterson, K. D., & Temple, J. B. (2001). Complex admission selection procedures for a graduate preservice teacher education program. Teacher Education Quarterly, 28(4), 7-21.
• Corcoran, R. P., & O’Flaherty, J. (2018). Factors that predict pre-service teachers’ teaching performance. Journal of Education for Teaching, 44(2), 175-193. https://doi.org/10.1080/02607476.2018.1433463
• Crocker, L. M., & Algina, L. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.
• Cronbach, L.J., Rajarathnam, N. & Gleser, G.C. (1963). Theory of generalizability: a liberalization of reliability theory. British Journal of Statistical Psychology, 16(2), 137–163.
• Donnon, T., & Paolucci, E. O. (2008). A generalizability study of the medical judgment vignettes interview to assess students' noncognitive attributes for medical school. BMC Medical Education, 8(1), 1-7. https://doi.org/10.1186/1472-6920-8-58
• Ebmeier, H., & Ng, J. (2005). Development and field test of an employment selection instrument for teachers in urban school districts. Journal of Personnel Evaluation in Education, 18(3), 201e218. https://doi.org/10.1007/s11092-006-9021-4
• Faulk, L. G. (2008). Predicting on-the-job teacher success based on a group assessment procedure used for admission to teacher education (Doctoral dissertation). Available from ProQuest Dissertations and Thesis database (UMI No. 3297518).
• Furr, M. (2011). Scale construction and psychometrics for social and personality psychology, London: Sage.
• Goho, J. & Blackman, A. (2006). The effectiveness of academic admission interviews: an exploratory meta-analysis. Medical Teacher, 28(4), 335-340. https://doi.org/10.1080/01421590600603418
• Güler, N., Uyanık, G. K. & Teker, G. T. (2012). Generalizability theory. Ankara: Pegem Academic Publishing.
• Gürlen, E., Boztunç-Öztürk, N., & Eminoğlu, E. (2019). Investigation of the reliability of teachers, self and peer assessments at the primary school level with generalizability theory. Journal of Measurement and Evaluation in Education and Psychology, 10(4), 406-421. https://doi.org/10.21031/epod.583891
• Haberman, M. & Post, L. (1998).Teachers for multicultural schools: the power of selection. Theory into Practice, 37(2), 96-104. https://doi.org/10.1080/00405849809543792
• Huffcutt, A., & Woehr, D. (1999). Further analysis of employment interview validity: A quantitative evaluation of interviewer-related structuring methods. Journal of Organizational Behavior, 20, 549–560.
• Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144. https://doi.org/10.1016/j.edurev.2007.05.002
• Kim, L. E., Jörg, V., & Klassen, R. M. (2019). A meta-analysis of the effects of teacher personality on teacher effectiveness and burnout. Educational psychology review, 31, 163-195. https://doi.org/10.1007/s10648-018-9458-2
• Leger, K. E. (2014). Defining teaching excellence: A phenomenological study of 2013 highly effective louisiana value-added model teachers with perfect evaluation scores. (Unpublished doctoral dissertation). Faculty of the College of Graduate Studies, Lamar University.
• Lupascu, A. R., Pânisoarâ, G., & Pânisoarâ, I-O. (2014). Characteristics of effective teacher. Procedia. Social and Behavioral Sciences, 127, 534-538. https://doi.org/10.1016/j.sbspro.2014.03.305
• Marcoulides, G. A. (1993). Maximizing Power in Generalizability Studies under Budget Constraints, Journal of Educational Statistics, 18(2), 197-206.
• Mathis, R. L., Jackson, J. H., Valentine, S. R., & Meglich, P. (2016). Human resource management. Cengage Learning.
• McDaniel, M. A., Whetzel, D. J., Schmidt, F. T., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599–616. https://doi.org/10.1037/0021-9010.79.4.599
• Petrarca, D., & LeSage, A. (2014). Should it stay or should it go? Re-considering the Pre-service Teacher Education Admissions Interview. ICET 2014, 252.
• Pellicer, L. O. (1981). Improved teacher selection with the structured interview. Educational Leadership, 38(6), 492-94.
• Pursell, E. D., Campion, M. A., & Gaylord, S. R. (1980). Structured interviewing: Avoiding selection problems. Personnel Journal, 59, 907–912.
• Schulte, D. P., Slate, J. R., & Onwuegbuzie, A. J. (2008). Effective high school teachers: A mixed investigation. International Journal of Educational Research, 47(1), 351-361. https://10.1016/j.ijer.2008.12.001
• Shavelson, R. J. & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage
• Shechtman, Z. (1992). Interrater reliability of a single group assessment procedure administered in several educational settings. Journal of Personnel Evaluation in Education, 6(1), 31-39.
• Smith, H. A., & Pratt, D. (1996). The use of biodata in admissions to teacher education. Journal of Teacher Education, 47(1), 43-52. https://doi.org/10.1177/0022487196047001008
• Stronge, J. H. (2007). Qualities of effective teachers. Alexandria, VA: Association for Supervision and Curriculum Development.
• Taneri, P. O. (2017). The viewpoints of instructors about the effects of teacher education programs on prospective teachers’ affective characteristics. Eurasian Journal of Educational Research, 17 (70), 105-120. http://dx.doi.org/10.14689/ejer.2017.70.6
• Taşdelen Teker, G., Şahin, M. G., & Baytemir, K. (2016). Using generalizability theory to investigate the reliability of peer assessment applications. Journal of Human Sciences, 13(3), 5574-5586. http://dx.doi.org/10.14687/jhs.v13i3.4155
• Thomson, D., Cummings, E., Ferguson., A. K., Miyuki Moizumi, E., Sher, Y., Wang, X., & Childs, R. A. (2011). A role for research in initial teacher education admissions: A case study from one Canadian university. Canadian Journal of Educational Administration and Policy, 121, 1–23.
• Tran, T., & Blackman, M. C. (2006). The dynamics and validity of the group selection interview. The Journal of Social Psychology, 146, 183–201. https://10.3200/SOCP.146.2.183-201
• Webb, N. M., & Shavelson, R. J. (2005). Generalizability theory: overview. Wiley StatsRef: Statistics Reference Online.
• Wing Tat-Chiu, C. (2001). Scoring performance assessments based on judgements generalizability theory. Springer Science+Business Media, LLC
• Yılmaz, N.F. & Başbaşa, N.B. (2015). Assessment of sewing and picking skills station reliability with generability theory. Journal of Measurement and Evaluation in Education and Psychology, 6(1): 107-116.
• Yılmaz, N.F. & Gelbal, S. (2011). Comparison of different patterns with generality theory in the case of communication skills station. Hacettepe University Journal of Education Faculty, 41: 509-518.
• Yılmaz, N. F. & Tavsancıl, E. (2014). Comparison of balanced and unbalanced patterns in generalizable theory with intramuscular injection station data. Education and Science, 39(175): 285-295.
• Uzun, N. B., Aktaş, M., Aşiret, S., & Yorulmaz, S. (2018). Using generalizability theory to assess the score reliability of communication skills of dentistry students. Asian Journal of Education and Training, 4(2), 85–90. https://doi.org/10.20448/journal.522.2018.42.85.90