Developing a Proof-of-Concept Selection Test for Entry into Primary Teacher Education Programs

Öz

The purpose of this article is to report on the development of a proof-of-concept situational judgment test (SJT) to assist in the selection of candidates for primary teacher education (ITE) programs. Nine development steps involving practising teachers, teacher educators, and applicants to ITE programs were carried out to establish target attributes and to develop content for the test. The results from administering the test to 124 primary ITE candidates showed a near-normal distribution, high levels of reliability, and significant positive correlations with a range of concurrently administered interview scores. We conclude with a description of the necessary next steps needed to implement evidence-supported teacher education selection processes in a range of international settings.

Anahtar Kelimeler

• teacher selection
• initial teacher education
• situational judgment tests
• teacher effectiveness
• recruitment
• teacher characteristics

Developing a Proof-of-Concept Selection Test for Entry into Primary Teacher Education Programs

Abstract

The purpose of this article is to report on the development of a proof-of-concept situational judgment test (SJT) to assist in the selection of candidates for primary teacher education (ITE) programs. Nine development steps involving practising teachers, teacher educators, and applicants to ITE programs were carried out to establish target attributes and to develop content for the test. The results from administering the test to 124 primary ITE candidates showed a near-normal distribution, high levels of reliability, and significant positive correlations with a range of concurrently administered interview scores. We conclude with a description of the necessary next steps needed to implement evidence-supported teacher education selection processes in a range of international settings.

Keywords

• teacher selection
• initial teacher education
• situational judgment tests
• teacher effectiveness
• recruitment
• teacher characteristics

References

1. Anderson, L. & Wilson, S. (1997) Critical incident technique. In D.L. Whetzel and G.R. Wheaton (eds.), Applied Measurement Methods in Industrial Psychology. Palo Alto, CA: Davies-Black.
2. Atteberry, A., Loeb, S., & Wyckoff, J. (2015). Do first impressions matter? Improvement in early career teacher effectiveness. AERA Open, 1(4), 1-23.
3. Barber, M. & Mourshed, M. (2007). How the world's best performing school systems come out on top. London: McKinsey & Company.
4. Bauer, T. N., & Truxillo, D. M. (2006). Applicant reactions to situational judgment tests: Research and related practical issues. In J. A. Weekley & R. E. Ployhart (Eds.) Situational judgment tests: Theory, measurement, and application (pp. 233-249). Mahwah, NJ: Lawrence Erlbaum Associates.
5. Bergman, M. E., Drasgow, F., Donovan, M. a., Henning, J. B., & Juraska, S. E. (2006). Scoring situational judgment tests: Once you get the data, your troubles begin. International Journal of Selection and Assessment, 14, 223-235.
6. Campion, M. C., Ployhart, R. E., & MacKenzie, W. I. (2014). The state of research on situational judgment tests: A content analysis and directions for future research. Human Performance, 27, 283–310.
7. Casey, C. E., & Childs, R. A. (2007). Teacher education program admission criteria and what beginning teachers need to know to be successful teachers. Canadian Journal of Educational Administration and Policy, 67, 1-24.
8. Christian, M. S., Edwards, B. D., & Bradley, J. C. (2010). Situational judgment tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology, 63, 83-117.

9. Department for Education (2016). Statistics at DfE. Retrieved from: https://www.gov.uk/government/organisations/department-for-education/about/statistics
10. Elliott, J. G., Stemler, S. E., Sternberg, R. J., Grigorenko, E. L., & Hoffman, N. (2011). The socially skilled teacher and the development of tacit knowledge. British Educational Research Journal, 37, 88-103.
11. Goldhaber, D., Grout, C., & Huntington-Klein, N. (2014). Screen twice, cut once: Assessing the predictive validity of teacher selection tools. CEDR Working Paper No. 2014-9: University of Washington, Seattle, WA.
12. Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition attitudes, self-esteem, and stereotypes. Psychological Review, 102, 4-27.
13. Hanushek, E. A., & Rivkin, S. G. (2011). The Distribution of Teacher Quality and Implications for Policy. Annual Review of Economics, 4, 131-157.
14. Hobson, A. J., Ashby, P., McIntyre, J., & Malderez, A. (2010). International approaches to teacher selection and recruitment. OECD Education Working Paper No.47: Organisation for Economic Co-operation and Development.
15. Hooper, A. C., Jackson, H. L., & Motowidlo, S. J. (2004). Situational judgment measures of personality and work-relevant performance. Paper presented at the 112th annual meeting of the American Psychological Association, Honolulu, HI.
16. Johnson, R. E., & Saboe, K. N. (2011). Measuring implicit traits in organizational research: Development of an indirect measure of employee implicit self-concept. Organizational Research Methods, 14, 530-547.
17. Kane, T. J., & Staiger, D. O. (2012). Gathering Feedback for Teaching: Combining High-Quality Observations with Student Surveys and Achievement Gains. Research Paper. MET Project. Bill & Melinda Gates Foundation.
18. Klassen, R. M., & Dolan, R. (2015, September). Selection for teacher education in the UK and the Republic of Ireland: A proposal for innovation. Presented at the meeting of the European Conference on Educational Research, Budapest, Hungary.
19. Klassen, R. M., & Durksen, T. L. (2014). Weekly self-efficacy and work stress during the final teaching practicum: A mixed methods study. Learning and Instruction, 33, 158-169.
20. Klassen, R.M., Durksen, T.L., Rowett, E., & Patterson, F. (2014). Applicant reactions to a situational judgment test used for selection into initial teacher training. International Journal of Educational Psychology, 3, 104-125.
21. Knight, F. B. (1922). Qualities related to success in elementary school teaching. The Journal of Educational Research, 5, 207-216.
22. Kunter, M., Kleickmann, T., Klusmann, U., & Richter, D. (2013). The development of teachers’ professional competence. In M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss, & M. Neubrand (Eds.), Cognitive activation in the mathematics classroom and professional competence of teachers (pp. 63-77). New York, NY: Springer.
23. Lievens, F., Corstjens, J., Sorrel, M. A., Abad, F. J., Olea, J., & Ponsoda, V. (2015). The cross-cultural transportability of situational judgment tests: How does a US-based integrity situational judgment test fare in Spain. International Journal of Selection and Assessment, 23, 361-372.
24. Lievens, F., & Patterson, F. (2011). The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection. Journal of Applied Psychology, 96, 927-940.
25. Lievens, F., Peeters, H., & Schollaert, E. (2008). Situational judgment tests: A review of recent research. Personnel Review, 37, 426-441.
26. Lievens, F., & Sackett, P. R. (2012). The validity of interpersonal skills assessment via situational judgment tests for predicting academic succss and job performance. Journal of Applied Psychology, 97, 460-468.
27. McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel psychology, 60, 63-91.
28. McDaniel, M. A., Whetzel, D., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-616.
29. Mikitovics, A., & Crehan, K. D. (2002). Pre-professional skills test scores as college of education admissions criteria. The Journal of Educational Research, 95, 215-223.
30. Motowidlo, S. J., & Beier, M. E. (2010). Differentiating specific job knowledge from implicit trait policies in procedural knowledge measured by a situational judgment test. Journal of Applied Psychology, 95, 321-333.
31. Patterson, F., Ashworth, V., Mehra, S., & Falcon, H. (2012). Could situational judgement tests be used for selection into dental foundation training? British Dental Journal, 213, 23–26.
32. Patterson, F., Zibarras, L., & Ashworth, V. (2016). Situational judgement tests in medical education and training: Research, theory and practice. AMEE Guide No. 100. Medical Teacher, 38, 3-17.
33. Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J. (2016). How effective are selection methods in medical education and training? Evidence from a systematic review. Medical Education, 50, 36–60.
34. Pianta, R. C., & Hamre, B. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38, 109-119.
35. Rockoff, J. E., Jacob, B. A., Kane, T. J., & Staiger, D. O. (2011). Can you recognize an effective teacher when you recruit one? Education, 6, 43-74.
36. Sahlberg, P. (2014). Finnish lessons 2.0: What can the world learn from educational change in Finland? New York, NY: Teachers College Press.
37. Schmitt, N., & Chan, D. (2006). Situational judgment tests: Method or construct? In J. A. Weekley & R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 135-155). Mahwah, NJ: Erlbaum.
38. Sclafani, S. K. (2015). Singapore chooses teachers carefully. Phi Delta Kappan, 97(3), 8-13.
39. Shultz, M. M., & Zedeck, S. (2012). Admission to Law School: New Measures. Educational Psychologist, 47, 51-65.
40. Staiger, D. O., & Kane, T. J. (2015). Making Decisions with Imprecise Performance Measures. In T. Kane, K. Kerr, R. Pianta, & Bill and Melinda Gates Foundation (Eds.), Designing Teacher Evaluation Systems (pp. 144-169). San Francisco, CA: Jossey-Bass.
41. Staiger, D. O., & Rockoff, J. E. (2010). Searching for effective teachers with imperfect information. Journal of Economic Perspectives, 24(3), 97-118.
42. Thomson, D., Cummings, E., Ferguson, A. K., Moizumi, E. M., Sher, Y., Wang, X., Broad, K., & Childs, R. A. (2011). A role for research in initial teacher education admissions: A case study from one Canadian university. Canadian Journal of Educational Administration and Policy, 67, 1–24
43. Walker, H. J., Bauer, T. N., Cole, M. S., Bernerth, J. B., Feild, H. S., & Short, J. C. (2013). Is this how I will be treated? Reducing uncertainty through recruitment interactions. Academy of Management Journal, 56, 1325-1347.
44. Watt, H. M. G., Richardson, P. W., & Wilkins, K. (2014). Profiles of professional engagement and career development aspirations among USA preservice teachers. International Journal of Educational Research, 65, 23-40.
45. Whetzel, D., & McDaniel, M. A. (2009). Situational judgment tests: An overview of current research. Human Resource Management Review, 19, 188-202.