Using the 2006 PISA Questionaire to Evaluate the Measure of Educational Resources: A Rasch Measurement Approach

Öz

School educational resources are key when studying school improvement due to their influence on learning outcomes. Because of this, careful attention should be given to the way educational resources are operationalized and measured. Using the 2006 PISA American sample containing 166 schools, this study aims to validate the 13-item PISA School Educational Resource Scale with Rasch analysis. Winsteps software was used in the analysis and results were used to evaluate how well the instrument measured the construct of school educational resource. Findings revealed that the PISA 2006 data gave an overall indication of good fit to the model, despite the instrument not separating respondents well. In regards to the quality of the scale, the majority of items perform consistently with the model. However, for schools above the average educational resource threshold, it appears there is a need for more items to discriminate the situation.

Anahtar Kelimeler

• School educational resource
• PISA
• Rasch rating scale model

Using the 2006 PISA Questionaire to Evaluate the Measure of Educational Resources: A Rasch Measurement Approach

Abstract

School educational resources are key when studying school improvement due to their influence on learning outcomes. Because of this, careful attention should be given to the way educational resources are operationalized and measured. Using the 2006 PISA American sample containing 166 schools, this study aims to validate the 13-item PISA School Educational Resource Scale with Rasch analysis. Winsteps software was used in the analysis and results were used to evaluate how well the instrument measured the construct of school educational resource. Findings revealed that the PISA 2006 data gave an overall indication of good fit to the model, despite the instrument not separating respondents well. In regards to the quality of the scale, the majority of items perform consistently with the model. However, for schools above the average educational resource threshold, it appears there is a need for more items to discriminate the situation.

Keywords

• School educational resource
• PISA
• Rasch rating scale model

References

1. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 357-74.
2. Andrich, D. & Luo, G. (2003). Conditional pairwise estimation in the Rasch model for ordered response Categories using principal components. Journal of Applied Measurement, 4(3), 205-221.
3. Ammermueller, A., Heijke, H., Woessmann, L. (2005). Schooling quality in Eastern Europe: Educational production during transition. Economical Educational Review, 24 (5), 579 599.
4. Bond, T., & Fox, C. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
5. Card, D., & Krueger, A. (1996). School resources and student outcomes: An overview of the literature and new evidence from North and South Carolina. Journal of Economic Perspectives. doi: jep.10.4.31
6. Coleman, J. S., Hoffer, C., & York, R. (1966). The equality of educational opportunity study. Washington, DC: United States Department of Health, Education, and Welfare.
7. Dodson, C. K. (2005). The Relationship between School Effectiveness and Teachers' Job Satisfaction in North Mississippi Schools. Unpublished Doctoral Dissertation, Mississippi University, Oxford.
8. Eliot, M., Cornell, D., Gregory, A., & Fan, X. (2010). Supportive school climate and student willingness to seek help for bullying and threats of violence. Journal of School Psychology, 48, 533-553. doi:10.1016/j.jsp.2010.07.001

9. Fan, M. (2013). Stability of academic performance across science subjects among Chinese students (Unpublished master’s theses). University of Kentucky, Lexington, KY.
10. Hanushek, E. (1997). Assessing the effects of school resources on student performance: An update. Educational Evaluation and Policy Analysis, 19(2), 141-164. doi:10.2307/1164207
11. Hanushek, E. A., & Luque, J. A. (2003). Efficiency and equity in schools around the world. Economics of Education Review, 22, 481-502. doi: 10.1016/S0272-7757(03)00038-4
12. Knoeppel, R. C., Verstegen, D. A., & Rinehart, J. S. (2007). What is the relationship between resources and student achievement: A canonical analysis. Journal of Education Finance, 33(2), 183-202.
13. Jacob, B. & Ludwig, J. (2008). Improving Educational Outcomes for Poor Children. Cambridge, MA: National Bureau of Economic Research.
14. Johnson, A. D. (2008). The relationships among middle school student and staff perceptions of school effectiveness and student achievement. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses.
15. Koth, C., Bradshaw, C., & Leaf, P. (2008). A multilevel study of predictors of student perceptions of school climate: The effect of classroom-level factors. Journal of Educational Psychology, 100(1), 96-104. doi: 0022-0663.100.1.96
16. Lezotte, L. W. (2001). Revolutionary and Evolutionary: The Effective Schools Movement. Retrieved from http://www.edutopia.org/pdfs/edutopia.org-closing-achievement-gap lezotte-article.pdf.
17. Linacre, J. M. (2002).What do Infit and Outfit, Mean-square and Standardized mean? Rasch Measurement Transactions, 16(2), 878.
18. Linacre, J. M. (2009). A user's guide to Winsteps, Ministep, Rasch-model computer programs: Program manual 3.72.3. Retrieved from http://www.winsteps.com/a/winsteps-manual.pdf.
19. Ma, X. (2001). Stability of school academic performance across subject areas. Journal of Educational Measurement, 38(1), 1-18.
20. MacNeil, A., Prater, D., & Busch, S. (2009). The effects of school culture and climate on student achievement. International Journal of Leadership in Education, 12(1), 73–84. doi: 10.1080/13603120701576241
21. Mortimore, P., Sammons, P., Stoll, L., Lewis, D., & Ecob, R. (1988). School matters. Berkeley, CA: The University of California Press.
22. Murnane, R. (1981). Interpreting the evidence on school effectiveness. Teachers College Record, 83(1), 19-35.
23. Organization for Economic Cooperation and Development. (1994). Making education count: Developing and using international indicators. Paris: Author.
24. Organization for Economic Cooperation and Development (2010). PISA 2009 Results: What Makes a School Successful? – Resources, Policies and Practices (Volume IV). Paris: Organization for Economic Cooperation and Development.
25. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Chicago: The University of Chicago Press.
26. Reid, K., Hopkins, D., & Holly, P. (1987). Towards the effective school. Oxford: Blackwell.
27. Reynolds, D & Creemers, B. (1990). School effectiveness and school improvement: A mission statement, School Effectiveness & School Improvement, 1(1): 1-3. doi: 10.1080/0924345900010101
28. Reynolds, D., Creemers, B., Stringfield, S., Teddlie, C., & Schaffer, G. (2002). World class school: International perspectives on school effectiveness. London: Routledge Farmer.
29. Savasci, H. & Tomul, E. (2013). The relationship between educational resources of school and academic achievement. International Education Studies, 6(4), 114-123. doi:10.5539/ies.v6n4p114
30. Sala, M. (2014). Examining the effects of school-level variables on elementary school students' academic achievement: The use of structural equation modeling (Unpublished doctoral dissertation). Clemson University, Clemson, SC.
31. Smith, R. M. (1996). Polytomous mean-square fit statistics. Rasch Measurement Transactions, 10, 516-517.
32. Schneider, B. (1985). Further evidence of school effects. Journal of Educational Research, 78(6), 351-356.
33. Stanco, G. (2012). Using TIMSS 2007 data to examine STEM school effectiveness factors in an international context. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses.
34. Superfine, B. M. (2009). Deciding who decides questions at the intersection of school finance reform litigation and standards-based accountability policies. Educational Policy 23(3), 480-514. doi: 10.1177/0895904808314712
35. Teddlie, C., & Stringfield, S. (1993). Schools make a difference: lessons learned from a 10-year study of school effects. New York: Teachers College Press.
36. Teddlie, C., & Reynolds, D. (2000). The international handbook of school effectiveness research. London: Falmer Press.
37. Teddlie, C. (2004). Getting schools working in South Africa: A valuable addition to the SESI field. School Effectiveness and School Improvement, 15(2), 227-240.
38. Vandiver, B. (2011). The impact of school facilities on the learning environment (Doctoral dissertation). Retrieved from ProQuest Dissertations Publishing (UMI No. 3439537)
39. Waugh, R. & Chapman, E., (2005). An analysis of dimensionality using factor analysis (true score theory) and Rasch measurement: what is the difference? Which method is better? Journal of Applied Measurement, 6(1), 80-99.
40. Way, N., Reddy, R., Rhodes, J. (2007). Students’ perceptions of school climate during the middle school years: Associations with trajectories of psychological and behavioral adjustment. American Journal of Community Psychology, 40, 194–213.
41. Wright, B. D., & Linacre, J. M. (1989). Observations are always ordinal: Measurements, however, must be interval. Archives of Physical Medicine and Rehabilitation, 70(12), 857-860.
42. Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago, IL: MESA Press.
43. Yan, Z. & Mok, M. (2012). Validating the coping scale for Chinese athletes using multidimensional Rasch analysis. Psychology of Sport and Exercise, 13, 271-279. doi:10.1016/j.psychsport.2011.11.013.