Measuring Teaching Best Practice in the Induction Years: Development and Validation of an Item-Level Assessment

Abstract

Schools and teacher induction programs around the world routinely assess teaching best practice to inform accreditation, tenure/promotion, and professional development decisions. Routine assessment is also necessary to ensure that teachers entering the profession get the assistance they need to develop and succeed. We introduce the Item-Level Assessment of Teaching practice (I-LAST) as a flexible framework-based approach for quantitative evaluation of teaching best practice in the induction stages. We based the I-LAST on a novel framework for teaching best practice, and used Fuller’s scale as a framework for understanding the potential of the I-LAST in providing longitudinal measures for growth. Using the context of a year-long teacher induction program in the Midwestern United States, we collected data through an online survey from 46 teaching supervisors who were asked to evaluate their interns. We used the Rasch partial credit model as a criterion for construct validity, and measured dimensionality and reliability from both Rasch and classical frameworks. The I-LAST was found to be a unidimensional, valid, and reliable measure for teaching best practice. It demonstrated the ability to provide reliable scores for specific sub- dimensions of best practice, including those which manifest at various stages along Fuller’s scale. Potential uses of the I-LAST to advance understanding of the role of teacher induction programs in fostering productive growth in new teachers is discussed.

Keywords

• Teacher induction, development, assessment, Rasch modeling, teaching internship, teacher attrition

References

1. Akiba, M., LeTendre, G. K., & Scribner, J. P. (2007). Teacher quality, opportunity gap, and national achievement in 46 countries. Educational Researcher, 36(7), 369-387.
2. Andrich, D. (2004). Controversy and the Rasch model: A characteristic of incompatible paradigms? Medical Care, 42(1), Supplement: Applications of Rasch Analysis in Health Care, 17–116.
3. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: PrenticeHall.
4. Berliner, D.C. (1988). Implications of studies on expertise in pedagogy for teacher education and evaluation. In New directions for teacher assessment (Proceedings of the 1988 ETS Invitational Conference, pp. 39-68). Princeton, NJ: Educational Testing Service.
5. Bitner, T., & Kratzner, R. (1995). A Primer on Building Teacher Evaluation Instruments.
6. Burnett, P. C., & Meacham, D. (2002). Measuring the quality of teaching in elementary school classrooms. Asia-Pacific Journal of Teacher Education,30(2), 141-153.
7. Capie, W. (1978). Teacher Performance Assessment Instruments: Plans for Practice Rating.
8. Cattell, R.B. (1966). The meaning and strategic use of factor analysis. In: Handbook of Multivariate Experimental Psychology. Chicago: Rand McNally.

9. Cochran-Smith, M., & Fries, M. K. (2001). Sticks, stones, and ideology: The discourse of reform in teacher education. Educational Researcher, 30(8), 3-15.
10. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd. Ed.) Lawrence Erlbaum and Associates.
11. Commission on No Child Left Behind. (2007). Beyond NCLB: Fulfilling the promise to our nation’s children. Washington, DC: The Aspen Institute.
12. Darling-Hammond, L. (2003). Keeping good teachers: Why it matters, what leaders can do. Educational Leadership, 6(8), 6-13.
13. Darling-Hammond, L., & Youngs, P. (2002). Defining “Highly Qualified Teachers”: What does “Scientifically-Based Research” actually tell us? The Educational Researcher, 31(9), 13-25.
14. Darling-Hammond, L., Holtzman, D.J., Gatlin, S.J. and Heilig, J.V. (2005). Does
15. teacher preparation matter? Evidence about teacher certification, Teach for America, and teacher effectiveness. Education Policy Analysis Archives, 13(42), Retrieved 9/21/2011 from http://epaa.asu.edu/epaa/v13n42.
16. De Fina, A. A. (1992). Portfolio Assessment: Getting Started. Teaching Strategies. Scholastic Inc. Jefferson City, Missouri.
17. D'Onofrio, C. N. (1989). The use of self‐reports on sensitive behaviors in health program evaluation. New Directions for Program Evaluation, 1989(43), 59-74.
18. Duffield, S.K. (2005). Swimming in the water: Immersing teacher candidates in the environment of a school. Current Issues in Education [on-line], 8 (11).
19. ETUCE. (2008). Teacher Education in Europe: an ETUCE policy paper. Brussels, Belgium.
20. Finn, C. E., Jr. (2003). High Hurdles. Education Next 3(2): 62–67.
21. Fuller, F. (1969). Concerns of teachers: A developmental conceptualization. American Educational Research Journal, 6, 207-226.
22. Gibson, S., & Dembo, M. H. (1984). Teacher efficacy: A construct validation.Journal of educational psychology, 76(4), 569.
23. Glaser, B., & Strauss, A. (1967). The Discovery of Grounded Theory. Chicago, IL: Aldine.
24. Goldhaber, D. (April, 2006). Everybody’s Doing It, But What Does Teacher Testing Tell Us About Teacher Effectiveness? Center on Reinventing Public Education. Paper presented at the AERA annual meeting.
25. Goodlad, J. (1994). Educational renewal. San Fransisco, CA: Jossey-Bass.
26. Goodlad, J. (1990). Teachers for our nation’s schools. San Francisco, CA: Jossey-Bass.
27. Gordon, R., T.J. Kane, and D.O. Staiger. (2006). Identifying effective teachers using performance on the job. Washington, DC: The Brookings Institution.
28. Helig, J. (June, 2010). Teach for America: A review of the evidence. Great Lakes Center for Educational Research and Practice: Lansing, MI.
29. Hoffman, J., Roller, C., Maloch, B., Sailors, M., Duffy, G., and Beretvas, S.N., The National Commission on Excellence in Elementary Teacher Preparation for Reading Instruction. (2005). Teachers’ preparation to teach reading and their experiences and practices in the first three years of teaching. The Elementary School Journal, 105(3), 267-287.
30. The Holmes Group. (1986). Tomorrow’s teachers. East Lansing, MI: Author.
31. Ingersoll, R. & Smith, T. (2003). The wrong solution to the teacher shortage. Educational Leadership, 60(8), 30-33.
32. Kagan, D. (1992). Professional growth among preservice and beginning teachers. Review of Educational Research, 62(2), 129-169.Linacre, J.M. (2010). Winsteps® (Version 3.70.0) [Computer Software]. Available from www.winsteps.com.
33. Linacre, J.M. (2012). Winsteps® Rasch Tutorial 3. Available from http://www.winsteps.com/a/winsteps-tutorial-3.pdf.
34. Martin, S. (2004). Finding balance: Impact of classroom management conceptions on development of teacher practice. Teaching and Teacher Education, 20, 405-422.
35. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174.
36. Mullen, C. & Farinas, J. (2003). What constitutes a highly qualified teacher: A review of teacher education standards and trends. Teacher Education and Practice, 16(4), 318-330.
37. NCATE. (2006). What makes a teacher effective? A summary of key research findings on teacher preparation. Washington DC: National Council for Accreditation of Teacher Education.
38. No Child Left Behind. (2001). Public Law, 107-110.
39. Penuel, W. R., Boscardin, C. K., Masyn, K., & Crawford, V. M. (2007). Teaching with student response systems in elementary and secondary education settings: A survey study. Educational Technology Research and Development, 55(4), 315-346.
40. Raiche, G. (2005). Critical eigenvalue sizes in standardized residual principle components analysis. Rasch Measurement Transactions, 19, 1012.
41. Ross, J. A., McDougall, D., Hogaboam-Gray, A., & LeSage, A. (2003). A survey measuring elementary teachers' implementation of standards-based mathematics teaching. Journal for Research in Mathematics Education, 344-363.
42. Savery, J. (2006). Overview of problem-based learning: definitions and distinctions. The Interdisciplinary Journal of Problem-Based Learning, 1(1), 9-20.
43. Smith, A. B., Rush, R., Fallowfield, L. J., Velikova, G., & Sharpe, M. (2008). Rasch fit statistics and sample size considerations for polytomous data. BMC Medical Research Methodology, 8(1), 33.
44. Smith, T. & Ingersoll, R. (2004). What are the effects of induction and mentoring on beginning teacher turnover? American Educational Research Journal, 41(3), 681-714.
45. Stulac, J. F. (1982). Procedure and Results of Measurement in the South Carolina Assessments of Performance in Teaching.
46. Tennant, A., & Connaghan, P.G. (2007). The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Care and Research, 57(8), 1358–1362.
47. Thomas, J. A., & Montgomery, P. (1998). On becoming a good teacher: reflective practice with regard to children's voices. Journal of Teacher Education, 49(5), 372-80.
48. Thomas, T. P., & Schubert, W. H. (2001). Reinterpreting teacher certification standards. In J. K. Kincheloe & D. Weil (Eds.), Standards and schooling in the United States: An encyclopedia, 1 (pp. 229-243). Santa Barbara, CA: ABC-CLIO.
49. Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and teacher education, 17(7), 783-805.
50. Wenglinsky, H. (2000). How teaching matters: bringing the classroom back into discussions of teacher quality. Princeton, NJ: Educational Testing Service.
51. Wolf, D. P. (1989). Portfolio assessment: Sampling student work. Educational leadership, 46(7), 35-39.
52. Wright, B.D., & Stone, M.A. (1979). Best test design. Chicago, IL: MESA Press.
53. Wright, B. D. (1996). Comparing Rasch measurement and factor analysis. Structural Equation Modeling: A Multidisciplinary Journal, 3(1), 3-24.
54. Wright, B.D. (1997). A history of social science measurement. Educational Measurement Issues and Practice, 16, 33-45.
55. Zeichner, K. (2010). Rethinking the connections between campus courses and field experiences in college and university based teacher education. Journal of Teacher Education, 61: 89-99.
56. Zemelman, S., Daniels, H., & Hyde, A. (2005). Best practice: Today’s standards for teaching and learning in America’s schools. Portsmouth, NH: Heinemann.