Tahminden Açıklamaya: Okuma Okuryazarlığının Araştırılmasında Topluluk Makine Öğrenmesi ile Yapısal Eşitlik Modellemesinin Birleştirilmesi

Abstract

Bu çalışmada, (1) PIRLS 2021 döngüsünde okuma okuryazarlığı becerisini tahmin etmede torbalama, rastgele ormanlar ve gradyan artırma algoritmalarının performansının ve değişken önem sıralamalarının incelenmesi ve (2) bu algoritmalar tarafından belirlenen etkili yordayıcı değişkenlerin yapısal eşitlik modellemesi (YEM) kullanılarak aralarındaki nedensel ilişkilerin araştırılması amaçlanmıştır. Farklı okuma başarı düzeylerindeki Finlandiya, Portekiz ve Türkiye'den veriler analiz edilmiştir. Sonuçlar, tüm algoritmaların ülkeler arasında benzer ve orta düzeyde tahmin performansı gösterdiğini ortaya koymuştur. Değişken önem sıralamaları, tüm ülkelerde algoritmalar arasında tutarlıdır ve temel yordayıcı değişkenler büyük ölçüde ülkeler arasında benzerlik göstermiştir. Finlandiya'da en etkili yordayıcılar okumaya yönelik özgüven, ilkokul öncesi erken okuryazarlık görevlerindeki performans ve evdeki öğrenme kaynakları olmuştur. Portekiz'de okumaya yönelik özgüven ve evdeki öğrenme kaynakları en etkili iken, Türkiye'de evdeki öğrenme kaynakları ve okumaya yönelik özgüvenin etkisi baskındır. Kısmi bağımlılık grafikleri, bu değişkenlerin tüm ülkelerde pozitif etkileri olduğunu göstermiştir. Özellikle, okumaya yönelik özgüven değişkeni daha yüksek okuma başarısına sahip olan Finlandiya ve Portekiz'de baskın bir etkiye sahipken, evdeki öğrenmeye yönelik kaynaklar değişkeni, nispeten daha düşük okuma başarısına sahip, Türkiye'de daha etkili bulunmuştur. YEM sonuçları bu bulguları desteklemiştir. YEM modelleri Türkiye ve Portekiz için kabul edilebilir uyum, Finlandiya için ise iyi uyum göstermiştir. Modeller, Türkiye'de bağlamsal ve ailesel faktörlerin daha belirgin olduğunu, Portekiz'de bireysel faktörlerin etkili olduğunu ve Finlandiya'da ise erken okuryazarlık görevlerinin dikkate değer bir etkiye sahip olduğunu ortaya koymuştur. Bu bulgular, okuma okuryazarlığını araştırmada makine öğrenmesi algoritmaları ile YEM'in birleştirilmesinin değerini ortaya koymaktadır. Tahmine dayalı modelleme ve nedensel analizlerin entegrasyonu, farklı eğitim bağlamlarında okuma okuryazarlığı performansını ve temel etkenleri kapsamlı bir şekilde anlamayı sağlayabilir.

Keywords

• topluluk makine öğrenmesi algoritmaları, yapısal eşitlik modellemesi, okuma okuryazarlığı, okumaya yönelik özgüven, PIRLS

Ethical Statement

Çalışmada ikincil veriler kullanıldığı için etik kurul iznine gerek yoktur.

From Prediction to Explanation: Integrating Ensemble Machine Learning and Structural Equation Modeling in Reading Literacy

Abstract

This study aimed to (1) examine the performance and variable importance rankings of bagging, random forests, and gradient boosting algorithms in predicting reading comprehension skills in the PIRLS 2021 cycle, and (2) investigate the causal relationships of influential predictor variables identified by these algorithms using Structural Equation Modeling (SEM). Data from Finland, Portugal, and Türkiye, representing different reading achievement levels, were analyzed. Results showed that all algorithms exhibited similar and moderate predictive performance across countries. Variable importance rankings were consistent across algorithms within each country, and the key predictors were largely similar across countries. In Finland, the most influential predictors were students’ reading confidence, early literacy task performance before primary school, and home learning resources. In Portugal, reading confidence and home resources were most influential, whereas in Türkiye, home resources and reading confidence were dominant. Partial dependence analyses indicated that these variables had positive effects across all countries. Notably, reading confidence had a dominant effect in Finland and Portugal, which have higher reading achievement, while home resources were particularly influential in Türkiye, with comparatively lower reading achievement. SEM results supported these findings. The SEM models showed acceptable fit for Türkiye and Portugal, and good fit for Finland. According to the models, contextual and familial factors were more prominent in Türkiye, individual factors were influential in Portugal, and early literacy tasks had a noteworthy impact in Finland. These findings highlight the value of combining machine learning algorithms and SEM to investigate reading literacy. The integration of predictive modeling and causal analysis can provide a comprehensive understanding of reading literacy performance and its underlying factors across different educational contexts.

Keywords

• ensemble machine learning algorithms
• structural equation modelling
• reading comprehension
• students confident in reading
• PIRLS

Ethical Statement

Since secondary data were used in this study, ethical committee approval was not required.

References

1. Austin, P. C., Lee, D. S., Steyerberg, E. W., & Tu, J. V. (2012). Regression trees for predicting mortality in patients with cardiovascular disease: What improvement is achieved by using ensemble‐based methods?. Biometrical journal, 54(5), 657-673. https://doi.org/10.1002/bimj.201100251
2. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.
3. Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123–140. https://doi.org/10.1007/BF00058655
4. Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32.https://doi.org/10.1023/A:1010933404324https://doi.org/10.1023/A:1010933404324
5. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Harvard University Press.
6. Cho, E., Kim, E. H., Ju, U., & Lee, G. A. (2021). Motivational predictors of reading comprehension in middle school: Role of self-efficacy and growth mindsets. Reading and Writing: An Interdisciplinary Journal, 34(9), 2337–2355. https://doi.org/10.1007/s11145-021-10146-5
7. Claes, R., Laga, J., Denies, K., Bleukx, N., Dockx, J., Van Keer, H., & Aesaert, K. (2024). The interplay between students’ home literacy environment, reading attitudes and comprehension: a serial mediation analysis using PIRLS 2021-data. Large-scale Assessments in Education, 12(1). https://doi.org/10.1186/s40536-024-00233-8
8. Duncan, G. J., & Magnuson, K. (2013). Investing in preschool programs. Journal of economic perspectives, 27(2), 109-132. https://dio.org/10.1257/jep.27.2.109

10. Fiorentini, N., Maboudi, M., Leandri, P., Losa, M., & Gerke, M. (2020). Surface motion prediction and mapping for road infrastructures management by PS-InSAR measurements and machine learning algorithms. Remote Sensing, 12(23), 3976. https://doi.org/10.3390/rs12233976
11. Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189–1232. https://doi.org/10.1214/aos/1013203451
12. Friedman, J.H. (2002). Stochastic gradient boosting. Computational Statistics and Data Analysis, 38(4), 367–378. https://doi.org/10.1016/S0167-9473(01)00065-2
13. Geske, A., Kampmane, K., & Ozola, A. (2021). The influence of school factors on students’ self-concept: Findings from PIRLS 2016. Human, Technologies and Quality of Education, 227-241. https://doi.org/10.22364/htqe.2021.16
14. Guthrie, J. T., & Wigfield, A. (2000). Engagement and motivation in reading. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3, pp. 403–422). Erlbaum.
15. Harris, L., Fondrie, W. E., Oh, S., & Noble, W. S. (2023). Evaluating proteomics imputation methods with improved criteria. Journal of proteome research, 22(11), 3427-3438. https://doi.org/10.1021/acs.jproteome.3c00205
16. Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction. Springer. https://doi.org/10.1007/978-0-387-84858-7
17. Hatlevik, O. E., Throndsen, I., Loi, M., & Gudmundsdottir, G. B. (2018). Students’ ICT self-efficacy and computer and information literacy: Determinants and relationships. Computers & Education, 118, 107–119. https://doi.org/10.1016/j.compedu.2017.11.011
18. Hew, K. F., Hu, X., Qiao, C., & Tang, Y. (2020). What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Computers & Education, 145, 103724. https://doi.org/10.1016/j.compedu.2019.103724
19. Hothorn, T., Hornik, K., & Zeileis, A. (2006). Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical Statistics, 15(3), 651–674. https://doi.org/10.1198/106186006X133933
20. Jäger, S., Allhorn, A., & Bießmann, F. (2021). A benchmark for data imputation methods. Frontiers in Big Data, 4, Article No: 693674. https://doi.org/10.3389/fdata.2021.693674
21. James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning with applications in R. Springer. https://doi.org/10.1007/978-1-0716-1418-1
22. Khoudi, Z., Nachaoui, M., & Lyaqini, S. (2024). Identifying the contextual factors related to the reading performance of Moroccan fourth-grade students from a machine learning-based approach. Education and Information Technologies, 29(3), 3047-3073. https://doi.org/10.1007/s10639-023-11881-8
23. Kim, C., & Park, T. (2022). Predicting determinants of lifelong learning intention using gradient boosting machine (GBM) with grid search. Sustainability, 14(9), 5256. https://doi.org/10.3390/su14095256
24. Kline, R. B. (2023). Principles and practice of structural equation modeling. Guilford publications.
25. Kraus, E. B., Wild, J., & Hilbert, S. (2024). Using interpretable machine learning for differential item functioning detection in psychometric tests. Applied Psychological Measurement, 48(4-5), 167-186. https://doi.org/10.1177/01466216241238744
27. Leino, K., Nissinen, K., & Sirén, M. (2022). Associations between teacher quality, instructional quality, and student reading outcomes in Nordic PIRLS 2016 data. Large-Scale Assessments in Education, 10(1), 25. https://doi.org/10.1186/s40536-022-00146-4
28. Liaw, A., & Wiener, M. (2002). Classification and regression by randomforest. R News, 2(3), 18–22.
29. Ma, L., & Zhao, Z. (2025). Reading motivation and reading comprehension achievement among English majors in China: A descriptive correlational study. Heliyon, 11(3). https://doi.org/10.1016/j.heliyon.2025.e42427
30. Man, K. (2024). Multimodal data fusion to detect preknowledge test-taking behavior using machine learning. Educational and Psychological Measurement, 84(4), 753-779. https://doi.org/10.1177/001316442311936
31. McMahon, S. D., Wernsman, J., & Rose, D. S. (2009). The relation of classroom environment and school belonging to academic self-efficacy among urban fourth-and fifth-grade students. The Elementary School Journal, 109(3), 267-281. https://doi.org/10.1086/592307
32. Mertler, C. A., & Vannatta, R. (2017). Advanced and multivariate statistical methods: Practical application and interpretation. Routledge.
33. Mullis, I. V. S., & Martin, M. O. (2019). PIRLS 2021 reading assessment framework. In I. V. S. Mullis & M. O. Martin (Eds.), PIRLS 2021 assessment frameworks (pp. 5-25). Boston College, TIMSS & PIRLS International Study Center. https://timssandpirls.bc.edu/pirls2021/frameworks/
34. Mullis, I. V. S., Martin, M. O., & Liu, J. (2019). PIRLS 2021 context questionnaire framework. In I. V. S. Mullis & M. O. Martin (Eds.), PIRLS 2021 assessment frameworks (pp. 27-56). Boston College, TIMSS & PIRLS International Study Center. https://timssandpirls.bc.edu/pirls2021/frameworks/
35. Mullis, I. V. S., von Davier, M., Foy, P., Fishbein, B., Reynolds, K. A., & Wry, E. (2023). PIRLS 2021 international results in reading. Boston College, TIMSS & PIRLS International Study Center. https://doi.org/10.6017/lse.tpisc.tr2103.kb5342
36. Norudin, M., Aminah, W., Abdul, M., & Anis, M. N. (2024). The mediating effect of early literacy activities on primary school learners’ reading comprehension. Perspectives in Education, 42(4), 216–231. https://doi.org/10.18820/2519593X/pie.v42.i4.15
37. Olweus, D., & Limber, S. P. (2010). Bullying in school: Evaluation and dissemination of the Olweus Bullying Prevention Program. American Journal of Orthopsychiatry, 80(1), 124–134. https://doi.org/10.1111/j.1939-0025.2010.01015.x
38. Papadopoulos, S., Azar, E., Woon, W. L., & Kontokosta, C. E. (2017). Evaluation of tree-based ensemble learning algorithms for building energy performance estimation. Journal of Building Performance Simulation, 11(3), 322–332. https://doi.org/10.1080/19401493.2017.1354919
39. Pongsophon, P. (2024). Deciphering the layers of reading achievement: A multi‐level analysis of student and school‐level predictors in top‐performing PIRLS 2021 countries. British Educational Research Journal, 6, 2777-2798. https://doi.org/10.1002/berj.4053
40. Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments & Computers, 36(4), 717–731. https://doi.org/10.3758/bf03206553
41. Redelmeier, A., Jullum, M., Aas, K. (2020). Explaining predictive models with mixed features using shapley values and conditional ınference trees. In A. Holzinger, P. Kieseberg, A. Tjoa & E. Weippl (Eds.), Machine Learning and Knowledge Extraction. CD-MAKE 2020. Lecture Notes in Computer Science, vol 12279. Springer, Cham. https://doi.org/10.1007/978-3-030-57321-8_7
42. Saidani, O., Umer, M., Alshardan, Alshardan, A., Alturki, N., Nappi, M. & Ashraf, I. (2024). Student academic success prediction in multimedia-supported virtual learning system using ensemble learning approach. Multimedia Tools and Application, 83, 87553–87578. https://doi.org/10.1007/s11042-024-18669-z
43. Schunk, D. H. (2003). Self-efficacy for reading and writing: Influence of modeling, goal setting, and self-evaluation. Reading & Writing Quarterly: Overcoming Learning Difficulties, 19(2), 159–172. https://doi.org/10.1080/10573560308219
44. Sembiring, L. T. A. B., Rukmini, D., Mujiyanto, J., & Yuliasri, I. (2018). The impact of comprehension instruction on students' reading comprehension with different ability grouping and self-efficacy. TESOL International Journal, 13(4), 156–171.
45. Sinharay, S., Zhang, M., & Deane, P. (2019). Prediction of essay scores from writing process and product features using data mining methods. Applied Measurement in Education, 32(2), 116–137. https://doi.org/10.1080/08957347.2019.1577245
46. Stekhoven, D. J., & Bühlmann, P. (2012). MissForest—non-parametric missing value imputation for mixed-type data. Bioinformatics, 28(1), 112–118. https://doi.org/10.1093/bioinformatics/btr597
47. Stevens, J. P. (2009). Applied multivariate statistics for the social sciences. Routledge.
48. Strand, O., & Schwippert, K. (2019). The impact of home language and home resources on reading achievement in ten-year-olds in Norway; PIRLS 2016. Nordic Journal of Literacy Research, 5(1), 1-17. https://doi.org/10.23865/njlr.v5.1260
49. Strobl, C. (2013). Data mining. In T. D. Little (Ed.), The Oxford handbook of quantitative methods in psychology: Vol. 2: Statistical analysis (pp. 678–700). OUP USA
50. Strobl, C., Boulesteix, A. L., Zeileis, A., & Hothorn, T. (2007). Bias in random forest variable importance measures: Illustrations, sources, and a solution. BMC Bioinformatics, 8, Article Number: 25. https://doi.org/10.1186/1471-2105-8-25
51. Thomure, H. T., Oo, C. Z., & Khine, M. S. (2025). Reading achievements among students in the United Arab Emirates: A multilevel analysis of PIRLS 2021. International Journal of Educational Research Open, 8, 100421. https://doi.org/10.1016/j.ijedro.2024.100421
52. Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: A meta-analysis. Psychological Bulletin, 140(4), 1174–1204. https://doi.org/10.1037/a0036620
53. Walker, B. J. (2003). The cultivation of student self-efficacy in reading and writing. Reading & Writing Quarterly: Overcoming Learning Difficulties, 19(2), 173–187. https://doi.org/10.1080/10573560308217
54. Weigel, D. J., Martin, S. S., & Bennett, K. K. (2006). Contributions of the home literacy environment to preschool-aged children’s emerging literacy and language skills. Early Child Development and Care, 176(3–4), 357–378. https://doi.org/10.1080/03004430500063747
55. Yu, J. (2021). Academic performance prediction method of online education using random forest algorithm and artificial intelligence methods. International Journal of Emerging Technologies in Learning, 16(5), 45–57. https://doi.org/10.3991/ijet.v16i05.20297
56. Zhu, L., You, H., Hong, M., & Fang, Z. (2025). Predictive insights into US students’ mathematics performance on PISA 2022 using ensemble tree-based machine learning models. International Journal of Educational Research, 130, 102537. https://doi.org/10.1016/j.ijer.2025.102537