‘SEEING THE TRUTH IN A CROOKED MIRROR’: EXAMINING PISA 2018 STUDENT ACHIEVEMENT WITHIN A NON-LINEAR FRAMEWORK

Abstract

The aim of this study is to examine 15-year-old students’ metacognitive strategies (i.e., assessing credibility, summarizing, understanding and remembering), general fear of failure, self-efficacy beliefs, subjective well-being, and growth mindset as predictors of their graded performances regarding science, mathematics, and reading literacy. Based on the data derived from the PISA Student Questionnaire 2018 (Turkey; N = 5938), a predictive correlational design was adopted in the present study. The data were analyzed through the multivariate adaptive regression splines (MARSplines) and path analyses. The results of the MARSplines analysis showed that, with one exception (i.e., assessing credibility), metacognitive strategies, self-efficacy beliefs, subjective well-being, general fear of failure, and growth mindset predicted graded performances regarding science, mathematics, and reading literacy in a non-linear manner. The results of the MARSplines analysis also demonstrated that the mentioned variables could be ranked according to their levels of importance in predicting science, mathematics, and reading literacy performances. The results of the path analysis, through which the relationships between the research variables were examined as linear, revealed that the path model had been misspecified. However, the results of the MARSplines analysis indicated that this was due to examining the non-linear relationships within a linear framework.

Keywords

• Student achievement
• Path analysis
• MARSplines
• Metacognitive strategies
• Growth mindset
• General fear of failure
• Subjective well-being
• General self-efficacy

‘EĞRİ BİR AYNADA DOĞRUYU GÖRMEK’: PISA 2018 ÖĞRENCİ BAŞARISININ DOĞRUSAL OLMAYAN BİR ÇERÇEVEDE İNCELENMESİ

Abstract

Bu araştırmanın amacı, 15 yaş grubu öğrencilerinin meta-biliş stratejileri (güvenilirliği değerlendirme, özetleme, anlama ve hatırlama), genel başarısızlık korkuları, öz-yeterlik inançları, öznel iyi oluşları ve yeteneğin geliştirilebilir olduğuna ilişkin inançlarının fen, matematik ve okuma başarılarının yordayıcıları olarak incelenmesidir. Araştırmada, PISA 2018 öğrenci anketi (Türkiye) verilerinden hareketle (N = 5938), yordayıcı ilişkisel desen kullanılmıştır. Veriler, çok değişkenli uyarlanabilir regresyon eğrileri (MARSplines) ve yol analizleri aracılığıyla çözümlenmiştir. MARSplines analizi sonuçları; biri dışında (güvenilirliği değerlendirme) meta-biliş stratejilerinin, öz-yeterlik inançlarının, öznel iyi oluşun, genel başarısızlık korkusunun ve yeteneğin geliştirilebilir olduğuna yönelik inançların fen, matematik ve okuma başarısını doğrusal olmayan biçimde yordadıklarını göstermiştir. MARSplines analizi sonuçları; söz konusu değişkenlerin fen, matematik ve okuma başarısının yordanmasındaki önem düzeylerine göre sıralanabildiğini de göstermiştir. Araştırma değişkenleri arasındaki ilişkilerin doğrusal olarak incelendiği yol analizi aracılığıyla elde edilen bulgular, yol modelinin hatalı tanımlandığını göstermiştir. Ancak MARSplines analizi sonuçları, bu durumun, doğrusal olmayan ilişkilerin doğrusal bir çerçevede incelenmesine bağlı olarak ortaya çıktığına işaret etmiştir.

Keywords

• Öğrenci başarısı
• Yol analizi
• MARSplines
• Meta-biliş stratejileri
• Yeteneğin geliştirilebilir olduğuna ilişkin inanç
• Genel başarısızlık korkusu
• Öznel iyi oluş
• Genel öz-yeterlik

References

1. American Psychological Association. (t.y.). Fear of failure. In APA dictionary of psychology. https://dictionary.apa.org/fear-of-failure. Erişim tarihi: 01.04.2021
2. Amholt, T. T., Dammeyer, J., Carter, R., & Niclasen, J. (2020). Psychological well-being and academic achievement among school-aged children: A systematic review. Child Indicators Research, 13, 1523-1548. https://doi.org/10.1007/s12187-020-09725-9
3. Atkinson, J. W. (1964). An introduction to motivation. Van Nostrand.
4. Bai, B., & Wang, J. (2020). The role of growth mindset, self-efficacy and intrinsic value in self-regulated learning and English language learning achievements. Language Teaching Research. https://doi.org/10.1177%2F1362168820933190
5. Bailey, T. H., & Phillips, L. J. (2015). The influence of motivation and adaptation on students’ subjective well-being, meaning in life and academic performance. Higher Education Research & Development, 35(2), 201-216. http://dx.doi.org/10.1080/07294360.2015.1087474
6. Baker, L. (2013). Metacognitive strategies. In J. Hattie, & E. M. Anderman, International guide to student achievement (pp. 419-421). Routledge.
7. Bandura, A. (1997). Self-Efficacy: The exercise of control. W H Freeman/Times Books/Henry Holt & Co.
8. Barbeau, K., Boileau, K., Sarr, F., & Smith, K. (2019). Path analysis in Mplus: A tutorial using a conceptual model of psychological and behavioral antecedents of bulimic symptoms in young adults. The Quantitative Methods for Psychology, 15(1), 38–53. https://doi.org/10.20982/tqmp.15.1.p038

9. Bennett, D. A. (2001). How can i deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25(5), 464-469. https://doi.org/10.1111/j.1467-842X.2001.tb00294.x
10. Bentler, P. M. (2006). EQS 6 structural equations program manual. Multivariate Software, Inc.
11. Bernardo, A. B. I., Cai, Y., & King, R. B. (2021). Society‐level social axiom moderates the association between growth mindset and achievement across cultures. British Journal of Educational Psychology, e12411. https://doi.org/10.1111/bjep.12411
12. Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78(1), 246-263. https://doi.org/10.1111/j.1467-8624.2007.00995.x
13. Boehmke, B., & Greenwell, B. (2020). Hands-On machine learning with R. Taylor & Francis.
14. Bonne, L., & Johnston, M. (2016). Students’ beliefs about themselves as mathematics learners. Thinking Skills and Creativity, 20, 17-28. https://doi.org/10.1016/j.tsc.2016.02.001
15. Booth, M. Z., Abercrombie, S., & Frey, C. J. (2017). Contradictions of adolescent self-construal: Examining the interaction of ethnic identity, self-efficacy and academic achievement. Mid-Western Educational Researcher, 29(1), 3-19.
16. Borgonovi, F., & Han, S. W. (2021). Gender disparities in fear of failure among 15-year-old students: The role of gender inequality, the organisation of schooling and economic conditions. Journal of Adolescence, 86, 28-39. https://doi.org/10.1016/j.adolescence.2020.11.009
17. Burnette, J. L., O'Boyle, E. H., VanEpps, E., Pollack, J. M., & Finkel, E. J. (2013). Mindsets matter: A meta-analytic review of the effects of implicit theories on self-regulation. Psychological Bulletin, 139(3), 655–701. https://doi.org/10.1037/a0029531
18. Bücker, S., Nuraydin, S., Simonsmeier, B. A., Schneider, M., & Luhmann, M. (2018). Subjective well-being and academic achievement: A meta-analysis. Journal of Research in Personality, 74, 83-94. https://doi.org/10.1016/j.jrp.2018.02.007
19. Cabarse, J. R., Cabusa, C. M., & Baran, J. A. (2018). Math and science performance on reading comprehension: A symbolic regression analysis. International Journal of English and Education, 7(4), 91-101.
20. Callan, G. L., Marchant, G. J., Finch, W. H., & German, R. L. (2016). Metacognition, strategies, achievement, and demographics: Relationships across countries. Educational Sciences: Theory & Practice, 16(5), 1485-1502. https://doi.org/10.12738/estp.2016.5.0137
21. Chen, G., Gully, S. M., & Eden, D. (2004). General self-efficacy and self-esteem: Toward theoretical and empirical distinction between correlated self-evaluations. Journal of Organizational Behavior, 25(3), 375-395. https://doi.org/10.1002/job.251
22. Chen, J. H., Björkman, A., Zou, J. H., & Engström, M. (2019). Self–regulated learning ability, metacognitive ability, and general self-efficacy in a sample of nursing students: A cross-sectional and correlational study. Nurse Education in Practice, 37, 15-21. https://doi.org/10.1016/j.nepr.2019.04.014
23. Choi, N. (2005). Self‐efficacy and self‐concept as predictors of college students' academic performance. Psychology in the Schools, 42(2), 197-205. https://doi.org/10.1002/pits.20048
24. Chumney, F. L. (2012). Comparison of maximum likelihood, bayesian, partial least squares, and generalized structured component analysis methods for estimation of structural equation models with small samples: An eploratory study [Master’s thesis, University of Nebraska – Lincoln]. https://digitalcommons.unl.edu/
25. Cohen, J. (1988). Statistical power analysis (2nd ed.). Erlbaum.
26. Costa, A., & Faria, L. (2018). Implicit theories of intelligence and academic achievement: A meta-analytic review. Frontiers in Psychology, 9, 829. https://doi.org/10.3389/fpsyg.2018.00829
27. Cromley, J. G. (2009). Reading achievement and science proficiency: International comparisons from the programme on international student assessment. Reading Psychology, 30(2), 89-118. https://doi.org/10.1080/02702710802274903
28. D'agostino, A., Schirripa Spagnolo, F., & Salvati, N. (2020). Studying the relationship between anxiety and school achievement: Evidence from PISA data. Advance. https://doi.org/10.31124/advance.12459470.v1
29. De Castella, K., Byrne, D., & Covington, M. (2013). Unmotivated or motivated to fail? A cross-cultural study of achievement motivation, fear of failure, and student disengagement. Journal of Educational Psychology, 105(3), 861–880. https://doi.org/10.1037/a0032464
30. Diener, E., Oishi, S., & Lucas, R. E. (2015). National accounts of subjective well-being. American Psychologist, 70(3), 234-242. https://doi.org/10.1037/a0038899
31. Ding, H., & Homer, M. (2020). Interpreting mathematics performance in PISA: Taking account of reading performance. International Journal of Educational Research, 102, 101566. https://doi.org/10.1016/j.ijer.2020.101566
32. Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development. Psychology Press.
33. Dweck, C. S. (2006). Mindset: The new psychology of success. Random House.
34. Ehrenreich, B. (2009). Smile or die: How positive thinking fooled America and the world. Granta.
35. Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231-235). Lawrence Erlbaum.
36. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill.
37. Fredrickson, B. L. (2004). The broaden-and-build theory of positive emotions. Philosophical Transactions of the Royal Society B, Biological Sciences, 359(1449), 1367–1378. https://doi.org/10.1098/rstb.2004.1512
38. Friedman, J. (1991). Multivariate adaptive regression splines. The Annals of Statistics, 19(1), 1-67.
39. Govorova, E., Benítez, I., & Muñiz, J. (2020). Predicting student well-being: Network analysis based on PISA 2018. International Journal of Environmental Research and Public Health, 17(11), 4014. https://doi.org/10.3390/ijerph17114014
40. Grether, T., Sowislo, J. F., & Wiese, B. S. (2018). Top-down or bottom-up? Prospective relations between general and domain-specific self-efficacy beliefs during a work-family transition. Personality and Individual Differences, 121, 131-139. https://doi.org/10.1016/j.paid.2017.09.021
41. Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference and prediction (2nd ed.). Springer. https://doi.org/10.1007/978-0-387-84858-7
42. Hatlevik, O. E., Throndsen, I., Loi, M., & Gudmundsdottir, G. B. (2018). Students’ ICT self-efficacy and computer and information literacy: Determinants and relationships. Computers & Education, 118, 107-119. https://doi.org/10.1016/j.compedu.2017.11.011
43. Heinze, A., Reiss, K., & Franziska, R. (2005). Mathematics achievement and interest in mathematics from a differential perspective. Zentralblatt füur Didaktik der Mathematik 37, 212–220. https://doi.org/10.1007/s11858-005-0011-7
44. Hill, T., & Lewicki, P. (2006). Statistics: Methods and applications: A comprehensive reference for science, industry, and data mining. StatSoft.
45. Keeley, J., Zayac, R., & Correia, C. (2008). Curvilinear relationships between statistics anxiety and performance among undergraduate students: Evidence for optimal anxiety. Statistics Education Research Journal, 7(1), 4–15.
46. Keller, L., Preckel, F., & Brunner, M. (2020). Nonlinear relations between achievement and academic self-concepts in elementary and secondary school: An integrative data analysis across 13 countries. Journal of Educational Psychology. Advance online publication. https://doi.org/10.1037/edu0000533
47. Klassen, R. M. (2007). Using predictions to learn about the self-efficacy of early adolescents with and without learning disabilities. Contemporary Educational Psychology, 32(2), 173-187. https://doi.org/10.1016/j.cedpsych.2006.10.001
48. Kline, R. B. (2016). Principle and practice of structural equation modeling (4th ed.). The Guilford.
49. Koyuncu, İ., & Fırat, T. (2020). Investigating reading literacy in PISA 2018 assessment. International Electronic Journal of Elementary Education, 13(2), 263-275.
50. Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134. https://doi.org/10.1037/0022-3514.77.6.1121
51. Lee, J. (2020). Non-cognitive characteristics and academic achievement in Southeast Asian countries based on PISA 2009, 2012, and 2015. OECD Education Working Papers (No. 233). OECD Publishing. https://doi.org/10.1787/c3626e2f-en
52. Lent, R. W., Brown, S. D., & Larkin, K. C. (1986). Self-efficacy in the prediction of academic performance and perceived career options. Journal of Counseling Psychology, 33(3), 265-269. https://doi.org/10.1037/0022-0167.33.3.265
53. Lin, M., Lucas, H. C. Jr., & Shmueli, G. (2013). Too big to fail: Large samples and the p-value problem. Information Systems Research, 24(4), 906–917. https://doi.org/10.1287/isre.2013.0480
54. Luszczynska, A., Gutiérrez‐Doña, B., & Schwarzer, R. (2005). General self‐efficacy in various domains of human functioning: Evidence from five countries. International Journal of Psychology, 40(2), 80-89. https://doi.org/10.1080/00207590444000041
55. Lyubomirsky, S., King, L., & Diener, E. (2005). The benefits of frequent positive affect: Does happiness lead to success? Psychological Bulletin, 131(6), 803-855. https://doi.org/10.1037/0033-2909.131.6.803
56. Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3), 519-530. https://doi.org/10.2307/2334770
57. MacCallum, R. C. (1995). Model specification: Procedures, strategies, and related issues. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 16-36). Sage.
58. Milborrow, S. (2020). earth: Multivariate adaptive regression splines (Version 5.3.0) [Computer software]. R Foundation for Statistical Computing. https://www.r-project.org/
59. Millî Eğitim Bakanlığı (MEB). (2019). PISA 2018 Türkiye ön raporu (Rapor no. 10). Eğitim Analiz ve Değerlendirme Raporları Serisi. MEB.
60. Muthén, L. K., & Muthén, B. O. (2012). Mplus statistical modeling software: Release 7.0 [Computer software]. Muthén & Muthén.
61. Ng, Z. J., Huebner, S. E., & Hills, K. J. (2015). Life satisfaction and academic performance in early adolescents: Evidence for reciprocal association. Journal of School Psychology, 53(6), 479-491. https://doi.org/10.1016/j.jsp.2015.09.004
62. Nimon, K. F. (2012). Statistical assumptions of substantive analyses across the general linear model: A mini-review. Frontiers in Psychology, 3, 322. https://doi.org/10.3389/fpsyg.2012.00322
63. Nisbet, R., Miner, G., & Yale, K. (2018). Handbook of statistical analysis and data mining applications (2nd ed.). Academic Press.
64. OECD. (baskıda). PISA 2018 technical report. OECD Publishing.
65. OECD. (2019a). PISA 2018 assessment and analytical framework. OECD Publishing. https://doi.org/10.1787/b25efab8-en
66. OECD. (2019b). PISA 2018 results (volume I): What students know and can do. OECD Publishing. https://doi.org/10.1787/19963777
67. OECD. (2019c). PISA 2018 results (volume II): Where all students can succeed. OECD Publishing. https://doi.org/10.1787/b5fd1b8f-en
68. OECD. (2019d). PISA 2018 results (Volume III): What school life means for students’ lives. OECD Publishing. https://doi.org/10.1787/acd78851-en
69. Ohtani, K., & Hisasaka, T. (2018). Beyond intelligence: A meta-analytic review of the relationship among metacognition, intelligence, and academic performance. Metacognition Learning, 13(2), 179–212. https://doi.org/10.1007/s11409-018-9183-8
70. Osborne, J. W. (2012). Best practices in data cleaning: A complete guide to everything you need to do before and after collecting your data. Sage.
71. Özberk, E. H., Atalay-Kabasakal, K., & Boztunç-Öztürk, N. (2017). Investigating the factors affecting Turkish students’ PISA 2012 mathematics achievement using hierarchical linear modeling. Hacettepe University Journal of Education, 32(3), 544-559. https://doi.org/10.16986/HUJE.2017026950
72. Özkan, U. B. (2020). PISA-2015 verilerine göre öğrencilerin ders dışı etkinliklere katılımlarının akademik başarılarına etkisi. İnönü Üniversitesi Eğitim Fakültesi Dergisi, 21(1), 254-269. https://doi.org/10.17679/inuefd.504780
73. Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543–578. https://doi.org/10.3102/00346543066004543
74. Pearl, J. & Mackenzie, D. (2020). Neden sorusunun kitabı: Neden sonuç ilişkisinin yeni bilimi (Çev. M. Havzalı). Ginko Kitap.
75. Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18, 315–341. https://doi.org/10.1007/s10648-006-9029-9
76. Pressley, M. (2002). Metacognition and self‐regulated comprehension. In A. Farstrup, & S. Samuels (Eds.), What research has to say about reading instruction (pp. 291– 309). International Reading Association. Programme for International Student Assessment (PISA). (2009). PISA data analysis manual SPSS (2nd ed.). OECD.
77. R Core Team. (2020). R: A language and environment for statistical computing (Version 4.0.3) [Computer software]. R Foundation for Statistical Computing. https://www.r-project.org/
78. Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs: General and Applied, 80(1), 1–28. https://doi.org/10.1037/h0092976
79. Schacht, S., & Stewart, B. (1992). Interactive/user-friendly gimmicks for teaching statistics. Teaching Sociology, 20(4), 329-332. https://doi.org/10.2307/1318981
80. Schafer, J. L. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research, 8(1), 3-15. https://doi.org/10.1177%2F096228029900800102
81. Schulze, A. (2020). Examinging the relationship among self-efficacy, fear of failure, and impostor phenomenon at a HBCU (Publication No. 28025928) [Doctoral dissertation, University of Louisiana Monroe]. ProQuest Dissertations and Theses Global.
82. Serra, M. J., & DeMarree, K. G. (2016). Unskilled and unaware in the classroom: College students’ desired grades predict their biased grade predictions. Memory & Cognition, 44, 1127–1137. https://doi.org/10.3758/s13421-016-0624-9
83. Shkullaku, R. (2013). The relationship between self – efficacy and academic performance in the context of gender among Albanian students. European Academic Research, 1(4), 467-478.
84. Soysa, C. K., & Wilcomb, C. J. (2015). Mindfulness, self-compassion, self-efficacy, and gender as predictors of depression, anxiety, stress, and well-being. Mindfulness, 6, 217–226. https://doi.org/10.1007/s12671-013-0247-1
85. Sriram, R. (2014). Rethinking intelligence: The role of mindset in promoting success for academically high-risk students. Journal of College Student Retention: Research, Theory & Practice, 15(4), 515–536. https://doi.org/10.2190/CS.15.4.c
86. Steffe, L. P., & Gale, J. E. (Eds.). (1995). Constructivism in education. Lawrence Erlbaum.
87. Streiner, D. L. (2005). Finding our way: An introduction to path analysis. The Canadian Journal of Psychiatry, 50(2), 115–122. https://doi.org/10.1177/070674370505000207
88. Suhr, D. (2008). Step your way through path analysis. Western Users of SAS Software Conference Proceedings. http://lexjansen.com/wuss/2008/pos/pos04.pdf
89. Suldo, S. M., Riley, K. N., & Shaffer, E. J. (2006). Academic correlates of children and adolescents’ life satisfaction. School Psychology International, 27(5), 567–582. https://doi.org/10.1177/0143034306073411 Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson
90. Tan, E. W. S., Lim, S. W. H., & Manalo, E. (2016). Global-local processing impacts academic risk taking. The Quarterly Journal of Experimental Psychology, 70(12), 2434-2444. http://dx.doi.org/10.1080/17470218.2016.1240815
91. Thakkar, J. J. (2020). Structural equation modelling: Application for research and practice (with AMOS and R). Springer. https://doi.org/10.1007/978-981-15-3793-6
92. Wu, J-Y. (2014). Gender differences in online reading engagement, metacognitive strategies, navigation skills and reading literacy. Journal of Computer Assisted Learning, 20(3), 252-271. https://doi.org/10.1111/jcal.12054
93. Yavuz, H. Ç., İlgün-Dibek, M., & Yalçın, S. (2017). Türk ve Vietnamlı öğrencilerin PISA 2012 matematik okuryazarlığı ile dürtü ve güdülenme özellikleri arasındaki ilişkiler. İlköğretim Online, 16(1), 178-196. http://dx.doi.org/10.17051/io.2017.45107
94. Yu, C. H. (2010). A model must be wrong to be useful: The role of linear modeling and false assumptions in theoretical explanation. The Open Statistics & Probability Journal, 2, 1-8. https://doi.org/10.2174/1876527001002010001
95. Zhang, W., & Goh, A. T. C. (2016). Multivariate adaptive regression splines and neural network models for prediction of pile drivability. Geoscience Frontiers, 7(1), 45-52. http://dx.doi.org/10.1016/j.gsf.2014.10.003