The Effect of School and Student-Related Factors on PISA 2015 Science Performances in Turkey

Abstract

The Program for International Student Assessment (PISA) is a research project conducted by the Organization for Economic Co-operation and Development, which evaluates the knowledge and skills gained by 15-year-old students over three-year terms. Within this study’s' scope, the PISA 2015 data were analysed to determine whether school-related factors [including the schools’ economic, social, and cultural status (ESCS)] were related to Turkish students’ science performances. Due to its nested structure, the released PISA 2015 data were analysed using the hierarchical linear model (HLM). Two models were considered to examine how Aggregated ESCS at the school level makes a difference. Thereby in model 1 shortage of educational material, staff shortage, student behaviours, and teacher behaviours were included in the analysis; in addition to these variables listed, aggregated ESCS was also added to the analysis in Model 2. The results of the analysis revealed that school-related factors - in particular, staff shortage, student behaviours, and aggregated ESCS indexes - were statistically related to students’ science performances. When the aggregated ESCS was controlled, it is observed that the school-level variables had a higher effect on students’ science performances.

Keywords

• HLM
• PISA
• science literacy
• science performance
• school resources
• school ESCS

References

1. Agasisti, T., Avvisati, F., Borgonovi, F., & Longobardi, S. (2018). Academic resilience: What schools and countries do to help disadvantaged students succeed in PISA. OECD Education Working Papers, No. 167, Paris, OECD Publishing, https://doi.org/10.1787/e22490ac-en .
2. Algina, J., & Swaminathan, H. (2011). Centering in two-level nested designs. Handbook of advanced multilevel analysis, 285-312.
3. Baker, D. P., Goesling, B., & Letendre, G. K. (2002). Socioeconomic status, school quality, and national economic development: A cross‐national analysis of the "Heyneman‐Loxley Effect" on mathematics and science achievement. Comparative Education Review, 46, 291‐312.
4. Baker, M. L., Sigmon, J. N., & Nugent, E. M. (2001). Truancy reduction: Keeping students in school. Washington, DC: U.S. Department of Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention.
5. Barber, B. L., Stone, M. R., & Eccles, J. S. (2010). Protect, prepare, support, and engage. Handbook of Research On Schools, Schooling, And Human Development, 336-378.
6. Berberoğlu, G., Çalışkan, M., & Karslı, N. (2019). Variables predicting PISA scientific literacy scores in Turkey. International Journal of Science and Education, 2(2), 38-49.
7. Bouhlila, D. S. (2015). The Heyneman–Loxley effect revisited in the Middle East and North Africa: Analysis using TIMSS 2007 database. International Journal of Educational Development, 42, 85-95.
8. Brewer, D. J., & Stacz, C. (1996). Enhancing opportunity to learn measures in NCES data. Santa Monica, CA: RAND.

9. Bybee, R., Fensham, P. J., & Laurie, R. (2009). Scientific literacy and contexts in PISA 2006 science, Journal of Research in Science Teaching, 46(8), 862-864.
10. Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64(8), 723-733.
11. Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfall, F. D., York, R. L. (1966). The Equality of Educational Opportunity. United States Department of Health Education and Welfare, Washington DC.
12. Cooper, R., & Liou, D. (2007). The Structure and Culture of Information Pathways: Rethinking Opportunity to Learn in Urban High Schools during the Ninth Grade Transition. The High School Journal, 91(1), 43-56.
13. Çeçen, Y. (2015). Examination of the predictive powers of sociocultural and socioeconomic variables for PISA science literacy by years (Unpublished master’s thesis). İstanbul Aydın University, Graduate School of Social Sciences, İstanbul.
14. Çelebi, Ö. (2010). A cross-cultural comparison of the effect of human and physical resources on students' scientific literacy skills in the Programme for International Student Assessment (PISA) 2006 (Unpublished doctoral dissertation). Middle Eastern Technical University, Ankara.
15. Edmonds, R. R. (1979). Effective schools for the urban poor. Educational Leadership, 37(1), 15-27.
16. Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: a new look at an old issue. Psychological Methods, 12(2), 121.
17. Erbaş, K. C. (2005). Factors affecting scientific literacy of students in Turkey in programme for international student assessment (PISA) (Unpublished master’s thesis). Middle Eastern Technical University, Ankara
18. Fuller, B., & Clarke, P. (1994). Raising school effects while ignoring culture? Local conditions and the influence of classroom tools, rules, and pedagogy. Review of educational research, 64(1), 119-157.
19. Gamoran, A., & Long, D. A. (2007). Equality of Educational Opportunity A 40 Year Retrospective. In International Studies in Educational Inequality, Theory and Policy (pp. 23-47). Springer.
20. González de San Román, A., & de La Rica, S. (2016). Gender gaps in PISA test scores: The impact of social norms and the mother’s transmission of role attitudes. Estudios de Economía Aplicada, 34(1), 79-108.
21. Greenwald, R., Hedges, L., & Laine, R. (1996). The effect of school resources on student achievement. Review of Education Research, 66, 361-396.
22. Hallfors, D., Cho, H., Brodish, P. H., Flewelling, R., & Khatapoush, S. (2006). Identifying high school students “at risk” for substance use and other behavioral problems: Implications for prevention. Substance Use & Misuse, 41(1), 1-15.
23. Hanushek, E. A. (1997). Assessing the effects of school resources on student performance: An update. Educational Evaluation and Policy Analysis, 19(2), 141-164.
24. Heinrich, C. J., & Lynn Jr., L. E. (2001). Means and ends: A comparative study of empirical methods for investigating governance and performance. Journal of Public Administration Research and Theory, 11(1), 109-138.
25. Henry, K. L., & Huizinga, D. H. (2007). Truancy’s effect on the onset of drug use among urban adolescents placed at risk. Journal of Adolescent Health, 40(4), 358.e9-358.e17
26. Heyneman, S. P., & Loxley, W., (1983). The effect of primary school quality on academic achievement across twenty-nine high- and low-income countries. Am. J. Sociol. 88(6), 1162–1194.
27. Heyneman, S.P., & Lee, B., (2012). Impact of international studies of academic achievement on policy and research. In: Ann Rutkowski, L., von Davier, M., Rutkowski, D. (Eds.), Handbook of International Large Scale Assessment: Background, Technical Issues, and Methods of Data Analysis (pp. 37–74). Chapman and Hall Publishers.
28. Hiebert, J., & Grouws, D. A. (2007). The effects of classroom mathematics teaching on students’ learning. In F. Lester (Ed), Second Handbook of Research on Mathematics Teaching and Learning (pp. 371–404). Information Age Publishing.
29. Ho, S., & Willms, J. D. (1996). Effects of parental involvement on eighth-grade achievement. Sociology of Education, 69(2), 126-141.
30. Huang, F. L. (2010). The role of socioeconomic status and school quality in the Philippines: Revisiting the Heyneman–Loxley effect. International Journal of Educational Development, 30(3), 288-296.
31. Juvonen, J., Espinoza, G., & Knifsend, C. (2012). The role of peer relationships in student academic and extracurricular engagement. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 387-401). Boston.
32. Karabay, E. (2012). Examination of the predictive powers of sociocultural variables for PISA science literacy by years (Unpublished master’s thesis). Ankara University, Ankara.
33. Karabay, E. (2013). Investigation of the predictive power of family and school characteristics for PISA reading skills, mathematics and science literacy by years (Unpublished master’s thesis). Gazi University, Ankara.
34. Klem, A. M., & Connell, J. P. (2004). Relationships matter: Linking teacher support to student engagement and achievement. Journal of School Health, 74(7), 262-273.
35. Lee, V. E. (2000). Using hierarchical linear modeling to study social contexts: The case of school effects. Educational Psychologist, 35(2), 125-141.
36. Lee, V. E., & Bryk, A. S. (1989). A multilevel model of the social distribution of high school achievement. Sociology of Education, 62(3), 172-192.
37. Lin, H. S., Hong, Z. R., & Huang, T. C. (2012). The role of emotional factors in building public scientific literacy and engagement with science. International Journal of Science Education, 34(1), 25-42.
38. Ma, X., & Willms, J.D. (2004). School disciplinary climate: Characteristics and effects on eight grade achievement. The Alberta Journal of Education Research, 50(2), 169-188.
39. Martin, M. O., Mullis, I. V. S., Foy, P., & Stanco, G. M. (2012). TIMSS 2011 international results in science. Chestnut Hill, MA: Boston College.
40. Newman, B. M., Myers, M. C., Newman, P. R., Lohman, B. J., & Smith, V. L. (2000). The transition to high school for academically promising, urban, low-income African American youth. Adolescence, 35(137), 45-66.
41. Nunnally, J. C., & Bernstein, I. H. (1991). Psychometric Theory. McGraw-Hill.
42. OECD, (2004). Learning for tomorrow’s world: First results from PISA 2003. OECD Publishing.
43. OECD, (2009). PISA Data Analysis Manual SPSS Second Edition. PISA. OECD Publishing.
44. OECD, (2016). PISA 2015 Results (Volume I): Excellence and Equity in Education, PISA, OECD Publishing.
45. OECD, (2016b). Indicator B1 how much is spent per student?, in Education at a Glance 2016: OECD Indicators, pp. 180-197, OECD Publishing. http://dx.doi.org/10.1787/eag-2016-16-en
46. OECD, (2016c). PISA 2015 Results (Volume II): Policies and Practıces for Successful Schools. PISA, OECD Publishing.
47. OECD, (2017). PISA 2015 Results (Volume V): Collaborative Problem Solving, PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264285521-en .
48. Perry, L. (2010). Does the SES of the school matter? Teachers College Record, 112(4), 1137–1162.
49. Perry, L. B., & McConney, A. (2010). Does the SES of the school matter? An examination of socioeconomic status and student achievement using PISA 2003. Teachers College Record, 112(4), 1137-1162.
50. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.
51. Ricard, N. C., & Pelletier, L. G. (2016). Dropping out of high school: The role of parent and teacher self-determination support, reciprocal friendships and academic motivation. Contemporary Educational Psychology, 44, 32-40.
52. Riddell, A.R., (1989a). An alternative approach to the study of school effectiveness in third world countries. Comp. Educ. Rev. 33(4), 481–497.
53. Riddell, A.R., (1989b). Response to Heyneman. Comp. Educ. Rev. 33(4), 505–506.
54. Rumberger, R. W., & Palardy, G. J. (2005). Does segregation still matter? The impact of student composition on academic achievement in high school. Teachers College Record, 107, 1999–2045.
55. Rutter, M., & Maughan, B. (2002). School effectiveness findings 1979–2002. Journal of School Psychology, 40(6), 451-475.
56. Saed, S., & Hammouri, H. (2010). Does subject matter matter? Estimating the impact of instructional practices and resources on student achievement in science and mathematics: Findings from TIMSS 2007. Evaluation & Research in Education, 23(4), 287-299.
57. Schleicher, A. (2009). Securing quality and equity in education: Lessons from PISA. Prospects, 39(3), 251-263.
58. Schmidt, W. H., McKnight, C. C., Valverde, G. A., Houang, R. T., & Wiley, D. E. (1997). Many visions, many aims: A cross-national investigation of curricular intentions in school mathematics. Kluwer Academic Publishers
59. Schwartz, W. (1995). Opportunity to learn standards: Their impact on urban students. Digest No. 110. New York City: ERIC Clearinghouse on Urban Education, ERIC No. ED389816.
60. Skinner, E. A., Pitzer, J. R., & Steele, J. S. (2016). Can student engagement serve as a motivational resource for academic coping, persistence, and learning during late elementary and early middle school? Developmental psychology, 52(12), 2099-2117.
61. Stacey, K. (2010). Mathematical and scientific literacy around the world. Journal of Science and Mathematics Education in Southeast Asia, 33(1), 1-16.
62. Sun, L., Bradley, K. D., & Akers, K. (2012). A multilevel modelling approach to investigating factors impacting science achievement for secondary school students: PISA Hong Kong sample. International Journal of Science Education, 34(14), 2107-2125.
63. Şirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75, 417–453.
64. Valeski, T. N., & Stipek, D. J. (2001). Young children's feelings about school. Child Development, 72(4), 1198-1213.
65. Valverde, G. A., Bianchi, L. J., Wolfe, R. G., Schmid, W. H., & Houang, R. T. (2002). According to the book. Using TIMSS to investigate the translation of policy into practice through the world of textbooks. Kluwer Academic Publishers.
66. Van Ewijk, R., & Sleegers, P. (2010). The effect of peer socioeconomic status on student achievement: A meta-analysis. Educational Research Review, 5(2), 134-150.
67. Wijaya, A. (2017). The relationships between Indonesian fourth graders’ difficulties in fractions and the opportunity to learn fractions: A snapshot of TIMMS results. International Journal of Instruction, 10(4), 221-236.
68. Willms, J. D. (1999). Quality and inequality in children’s literacy: The effects on families, schools, and communities. In D. P. Keating & C. Hertzman (Eds.), Developmental health and the wealth of nations: Social, biological, and educational dynamics (pp. 72–93). Guilford Press.
69. Woltman, H., Feldstain, A., MacKay, J. C., & Rocchi, M. (2012). An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology, 8(1), 52-69.
70. Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2-3), 114-128.
71. Yıldırım, K. (2012). The main determinants of the quality of Education in Turkey in accordance to PISA 2006 data. The Journal of Turkish Educational Sciences, 10(2), 229-255.