Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy

Eda Akdoğdu Yıldız; Kübra Atalay Kabasakal

doi:10.17152/gefad.1700937

EN TR

Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy

Öz

In this research, it is aimed to compare hierarchical linear modelling and artificial neural network estimation methods in predicting students' reading comprehension success in the Program for International Student Assessment (PISA) 2018 application. In accordance with this purpose, it is planned to determine how students' PISA success status is estimated at student and school level, the data mining method used in estimation and the explained variance and error values of multilevel modelling. The type of study is, in a way, relational research because of the establishment of models in which there are relationships between dependent and independent variables. On the other hand, it is descriptive research in terms of performing analyses with two methods for each country sampled in the study and comparing the results obtained in terms of explained variance and error values. In this research, the performance of data mining techniques (artificial neural networks – ANN) and multilevel analysis methods (hierarchical linear modeling – HLM) in the field of education is evaluated. It has been determined that HLM carries out the estimation process with lower error and higher R^2 than ANN in the analysis of multi-level data. In addition, HLM provides more information about the predictive level of the variables and the variance that is not explained by the variables in the model compared to ANN. For this reason, HLM analysis was used to examine the variables that affect reading comprehension success in the study. As a result, it was seen that the student level and school level variables added to the model had a statistically significant effect on reading comprehension achievement. While teacher-directed instruction and lack of educational material at school cause negative effects on reading comprehension success, it has been determined that economic-social-cultural situation, metacognitive strategies, disciplinary climate in the classroom, teacher support, and staff shortage variables have positive effects. The results obtained are generally in agreement with similar studies in the literature.

Anahtar Kelimeler

Hierarchical linear modelling, Data mining, Artificial neural networks, Reading comprehension, PISA

Hiyerarşik Doğrusal Modellemenin ve Yapay Sinir Ağları Yöntemlerinin PISA 2018 Okuma Okuryazarlığı Tahmin Doğruluğunun Araştırılması

Öz

Bu araştırmada Uluslararası Öğrenci Değerlendirme Programı (PISA) 2018 uygulamasında öğrencilerin okuduğunu anlama başarısını tahmin etmede hiyerarşik lineer modelleme ve yapay sinir ağları tahmin yöntemlerinin karşılaştırılması amaçlanmaktadır. Bu amaç doğrultusunda; öğrencilerin PISA başarı durumlarının birey ve okul düzeyinde nasıl tahmin edildiği, tahmin etmede kullanılan veri madenciliği yöntemi ve çok düzeyli modellemenin açıklanan varyans ve hata değerlerinin belirlenmesi planlanmaktadır. Çalışmanın türü, bağımlı ve bağımsız değişkenler arasında ilişkilerin bulunduğu modellerin kurulmasından dolayı bir yönüyle ilişkisel araştırmadır. Diğer bir yönüyle ise çalışmada örnekleme alınan her ülke için iki yöntemle analizler gerçekleştirilip elde edilen sonuçların açıklanan varyans ve hata değerleri açısından karşılaştırılması bakımından betimsel araştırma niteliğindedir. Bu araştırmada eğitim alanında da kullanılmaya başlanan veri madenciliği (yapay sinir ağları-YSA) ve çok düzeyli analiz yöntemlerinin (hiyerarşik lineer model-HLM) nasıl performans gösterdiğine ilişkin bulgular elde edilmiştir. HLM’in çok düzeyleri verilerin analizinde YSA’ya göre daha düşük hata ve daha yüksek R^2 ile tahminleme sürecini yürüttüğü belirlenmiştir. Ayrıca HLM değişkenlerin yordama düzeyi ve modelde yer alan değişkenler tarafından açıklanmayan varyans hakkında YSA’ya göre daha fazla bilgi sunmaktadır. Bu sebeple çalışmada okuduğunu anlama başarısını etkileyen değişkenleri incelemek için HLM analizi kullanılmıştır. Sonuç olarak modele eklenen birey düzeyi (düzey 1) ve okul düzeyi (düzey 2) değişkenlerinin okuduğunu anlama başarısı üzerinde istatiksel olarak anlamlı etkisi olduğu görülmüştür. Okuduğunu anlama başarısı üzerinde öğretmen yönlendirmeli öğretim ve okuldaki eğitimsel materyal eksikliği negatif yönlü etkiye sebep olurken ekonomik-sosyal-kültürel durum, üstbiliş stratejileri, sınıftaki disiplin iklimi, öğretmen desteği, personel eksikliği değişkenleri pozitif yönlü etkisi olduğu tespit edilmiştir. Elde edilen sonuçlar genel olarak literatürde yer alan benzer çalışmalarla uyum göstermektedir.

Anahtar Kelimeler

Hiyerarşik lineer modelleme, Veri madenciliği, Yapay sinir ağları, Okuduğunu anlama, PISA

Kaynakça

Abiodun, O. I., Jantan, A., Omolara, A. E., Dada, K. V., Mohamed, N. A., & Arshad, H. (2018). State-of-the-art in artificial neural network applications: A survey. Heliyon, 4(11), e00938. https://doi.org/10.1016/j.heliyon.2018.e00938
Akkoyunlu, B. (2008, Mayıs). Bilgi okuryazarlığı ve yaşam boyu öğrenme. 8th International Educational Technology Conference (IECT) toplantısında sunulan bildiri, Anadolu Üniversitesi, Eskişehir.
Anderson, R.C., Hiebert, E.H., Scott, J.A. & Wilkinson, I.A.G. (1988). Becoming a nation of readers: the report of the commission on reading. Education and Treatment of Children, 11(4), 389-396.
Anderson, D., & McNeill, G. (1992). Artificial neural networks technology. Kaman Sciences Corporation
Ali, O., Murray, P. A., Momin, M., Dwivedi, Y. K., & Malik, T. (2023). The effects of artificial intelligence applications in educational settings: Challenges and strategies. Technological Forecasting and Social Change, 199, 123076. https://doi.org/10.1016/j.techfore.2023.123076
Baba, A. (2024). Neural networks from biological to artificial and vice versa. Biosystems, 235, 105110. https://doi.org/10.1016/j.biosystems.2023.105110
Bennett, D. A. (2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25(5), 464-469.
Botchkarev, A. (2019). A new typology design of performance metrics to measure errors in machine learning regression algorithms. Interdisciplinary Journal of Information Knowledge and Management, 14, 045–076. https://doi.org/10.28945/4184
Chai, T., & Draxler, R. R. (2014). Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature. Geoscientific Model Development, 7(3), 1247–1250. https://doi.org/10.5194/gmd-7-1247-2014
Chavez, H., Chavez-Arias, B., Contreras-Rosas, S., Alvarez-Rodríguez, J. M., & Raymundo, C. (2023). Artificial neural network model to predict student performance using nonpersonal information. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1106679

Chen, J., Lin, C. H., & Chen, G. (2021). A cross-cultural perspective on the relationships among social media use, self-regulated learning and adolescents’ digital reading literacy. Computers & Education, 175, 104322.
Chen, Q., Lei, Y., Wen, Z., Li, S., Li, J., & Kong, Y. (2019). Teacher support, reading strategy and reading literacy: A two-level mediation model. Best Evid Chin Edu, 2(1), 157-170. https://doi.org/10.15354/bece.19.ar1036
Courtney, M., Karakus, M., Ersozlu, Z., & Nurumov, K. (2022). The influence of ICT use and related attitudes on students’ math and science performance: Multilevel analyses of the last decade’s PISA surveys. Large-Scale Assessments in Education, 10(1), 1-26. https://doi.org/10.1186/s40536-022-00128-6
Çelenk, S. (2003). Okul aile iş birliği ile okuduğunu anlama başarısı arasındaki ilişki. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 24, 33-39.
Dong, X., & Hu, J. (2019). An exploration of impact factors influencing students’ reading literacy in Singapore with machine learning approaches. International Journal of English Linguistics, 9(5), 52-65. https://doi.org/10.5539/ijel.v9n5p52
Ercan, U. (2021). Ev dışı gıda tüketim sınıflarının yapay sinir ağları ile tahmin edilmesi. İşletme Araştırmaları Dergisi, 13(4), 3265-3277.
Feng, Y., & Jones, K. (2015, July 8-10). Comparing multilevel modelling and artificial neural networks in house price prediction [Paper presentation]. 2015 2nd IEEE International Conference on Spatial Data Mining and Geographical Knowledge Services, Fuzhou, China.
Fritsch, S., Guenther, F., & Wright, M. N. (2019). neuralnet: Training of Neural Networks. R package version 1.44.2.
Gamazo, A., & Martínez-Abad, F. (2020). An exploration of factors linked to academic performance in PISA 2018 through data mining techniques. Frontiers in Psychology, 11, 575167. https://doi.org/10.3389/fpsyg.2020.575167
Guthrie, J. T., & Wigfield, A. (2000). Engagement and motivation in reading. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3, pp. 403–422). Lawrence Erlbaum.
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Springer Science & Business Media.
Hawkins, D. M. (2003). The problem of overfitting. Journal of Chemical Information and Computer Sciences, 44(1), 1–12. https://doi.org/10.1021/ci0342472
Haykin, S. (2008). Neural networks: a comprehensive foundation. New Jersey: Prentice-Hall.
Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2. ed): New York: Routledge.
Hu, J., & Wang, Y. (2022). Influence of students’ perceptions of instruction quality on their digital reading performance in 29 OECD countries: A multilevel analysis. Computers & Education, 189, 104591. https://doi.org/10.1016/j.compedu.2022.104591
Kumova M., S. & Kışla, T. (2020). Yapay sinir ağları. In T. Güyer, H. Yurdugül & S. Yıldırım (Ed.), Eğitsel veri madenciliği ve öğrenme algoritmaları (pp. 127-146). Anı Publishing.
Koyuncu, İ., & Fırat, T. (2021). Investigating Reading Literacy in PISA 2018 assessment. Lnternational Electronic Journal of Elementary Education, 13(2), 263–275. https://doi.org/10.26822/iejee.2021.189
Kuhn, M. (2022). caret: classification and regression training. R package version 6.0-91.
Kursa, M. B., & Rudnicki, W. R. (2010). Feature selection with the boruta package. Journal of Statistical Software, 36(11), 1-13.
Leisch, F., & Dimitriadou, E. (2021). mlbench: Machine learning benchmark problems. R package version 2.1-3.
Levy, J., Mussack, D., Brunner, M., Keller, U., Cardoso-Leite, P., & Fischbach, A. (2020). Contrasting classical and machine learning approaches in the estimation of Value-Added scores in Large-Scale Educational data. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.02190
Liaw, A., & Wiener, M. (2002). Classification and Regression by randomForest. R News 2(3), 18-22.
Lee, Y. H., & Wu, J. Y. (2013). The indirect effects of online social entertainment and information seeking activities on reading literacy. Computers & Education, 67, 168-177. https://doi.org/10.1016/j.compedu.2013.03.001
Lin, M. B., Groves, W. A., Freivalds, A., Lee, E. G., & Harper, M. (2011). Comparison of artificial neural network (ANN) and partial least squares (PLS) regression models for predicting respiratory ventilation: an exploratory study. European Journal of Applied Physiology, 112(5), 1603–1611. https://doi.org/10.1007/s00421-011-2118-6
Lim, H. J., & Jung, H. (2019). Factors related to digital reading achievement: A multi-level analysis using international large scale data. Computers & Education, 133, 82–93. https://doi.org/10.1016/j.compedu.2019.01.007
Ma, L., Xiao, L., & Hau, K. T. (2022). Teacher feedback, disciplinary climate, student self-concept, and reading achievement: A multilevel moderated mediation model. Learning and Instruction, 79, 101602.
Maas, C. J. M., & Hox, J. J. (2005). Sufficient Sample Sizes for Multilevel Modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1(3), 86–92. https://doi.org/10.1027/1614-2241.1.3.86
McNeish, D. M., & Stapleton, L. M. (2016). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28(2), 295–314. https://doi.org/10.1007/s10648-014-9287-x
Michela, J. L. (2006). Software review: HLM 6. Organizational Research Methods, 9(1), 119-122. https://doi.org/10.1177/10944281052812
Munson, M.A., Caruana, R. (2009). On Feature Selection, Bias-Variance, and Bagging. Buntine. In W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (Eds.), Machine Learning and Knowledge Discovery in Databases. ECML PKDD Lecture Notes in Computer Science, vol 5782. Springer, Berlin, Heidelberg.
Muratkyzy, A. (2020). Equity and excellence in the Kazakhstani Education System: A multilevel analysis of the personal and contextual factors contributing to students’ reading literacy performance on PISA 2018 [Unpublished master’s thesis]. Nazarbayev University Institute of Educational Sciences, Astana.
Ni, X. (2008). Research of data mining based on neural networks. World Academy of Science, Engineering and Technology, 39, 381-384.
OECD. (2019). PISA 2018 results volume I: What students know and can do. Paris: OECD Publishing. https://doi.org/10.1787/5f07c754-en.
Oreta, A. W. C. (2004). Simulating size effect on shear strength of RC beams without stirrups using neural networks. Engineering Structures, 26(5), 681-691. https://doi.org/10.1016/j.engstruct.2004.01.009
Radoyevic, N. (2006). Exploring the use of effective learning strategies to increase students’ reading comprehension and test taking skills (Yüksek lisans tezi). Brock Üniversitesi Eğitim Fakültesi, St. Catharines, Ontario.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models. Applications and Data Analysis Methods (2nd ed.). Thousand Oaks, CA: Sage Publications.
Rose, D. S., Parks, M., Androes, K., & McMahon, S. D. (2000). Imagery-based learning: Improving elementary students' reading comprehension with drama techniques. The Journal of Educational Research, 94(1), 55-63.
Shahini, A. (2021). Inequalities in Albanian education: Evidence from large-scale assessment studies. Kultura i Edukacja, 4(134), 40-70.
Snow, C. E. (2002). Reading for Understanding: Toward an R&D Program in Reading Comprehension. RAND Corporation.
Tang, F., & Ishwaran, H. (2017). Random forest missing data algorithms. Statistical Analysis and Data Mining the ASA Data Science Journal, 10(6), 363–377. https://doi.org/10.1002/sam.11348
Weng, W., & Luo, W. (2023). A comparative analysis of data mining methods and hierarchical linear modeling using PISA 2018 data. International Journal of Database Management Systems, 15(2/3), 1–16. https://doi.org/10.5121/ijdms.2023.15301
Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York.
Willmott, C., & Matsuura, K. (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research, 30, 79–82. https://doi.org/10.3354/cr030079
Yıldız Aybek, H. S., & Okur, M. R. (2018). Predicting achievement with artificial neural networks: The case of Anadolu University Open Education System. International Journal of Assessment Tools in Education, 5(3), 474-490.
Yousafzai, B. K., Khan, S. A., Rahman, T., Khan, I., Ullah, I., Ur Rehman, A., Baz, M., Hamam, H., & Cheikhrouhou, O. (2021). Student-Performulator: Student Academic Performance Using Hybrid Deep Neural Network. Sustainability, 13(17), 9775. https://doi.org/10.3390/su13179775

Ayrıntılar

Birincil Dil

İngilizce

Konular

Ulusal ve Uluslararası Başarı Karşılaştırmaları, Eğitimde Ölçme ve Değerlendirme (Diğer)

Bölüm

Araştırma Makalesi

Yazarlar

Eda Akdoğdu Yıldız ^*
0000-0003-4374-4379
Australia

Kübra Atalay Kabasakal
0000-0002-3580-5568
Türkiye

Yayımlanma Tarihi

30 Ağustos 2025

Gönderilme Tarihi

17 Mayıs 2025

Kabul Tarihi

8 Ağustos 2025

Yayımlandığı Sayı

Yıl 2025 Cilt: 45 Sayı: 2

DOI

https://doi.org/10.17152/gefad.1700937

IZ

https://izlik.org/JA56CF25ZB

APA

Akdoğdu Yıldız, E., & Atalay Kabasakal, K. (2025). Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy. Gazi Eğitim Fakültesi Dergisi, 45(2), 543-568. https://doi.org/10.17152/gefad.1700937

AMA

1.Akdoğdu Yıldız E, Atalay Kabasakal K. Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy. GEFAD. 2025;45(2):543-568. doi:10.17152/gefad.1700937

Chicago

Akdoğdu Yıldız, Eda, ve Kübra Atalay Kabasakal. 2025. “Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy”. Gazi Eğitim Fakültesi Dergisi 45 (2): 543-68. https://doi.org/10.17152/gefad.1700937.

EndNote

Akdoğdu Yıldız E, Atalay Kabasakal K (01 Ağustos 2025) Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy. Gazi Eğitim Fakültesi Dergisi 45 2 543–568.

IEEE

[1]E. Akdoğdu Yıldız ve K. Atalay Kabasakal, “Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy”, GEFAD, c. 45, sy 2, ss. 543–568, Ağu. 2025, doi: 10.17152/gefad.1700937.

ISNAD

Akdoğdu Yıldız, Eda - Atalay Kabasakal, Kübra. “Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy”. Gazi Eğitim Fakültesi Dergisi 45/2 (01 Ağustos 2025): 543-568. https://doi.org/10.17152/gefad.1700937.

JAMA

1.Akdoğdu Yıldız E, Atalay Kabasakal K. Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy. GEFAD. 2025;45:543–568.

MLA

Akdoğdu Yıldız, Eda, ve Kübra Atalay Kabasakal. “Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy”. Gazi Eğitim Fakültesi Dergisi, c. 45, sy 2, Ağustos 2025, ss. 543-68, doi:10.17152/gefad.1700937.

Vancouver

1.Eda Akdoğdu Yıldız, Kübra Atalay Kabasakal. Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy. GEFAD. 01 Ağustos 2025;45(2):543-68. doi:10.17152/gefad.1700937

Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy

Öz

Anahtar Kelimeler

Hiyerarşik Doğrusal Modellemenin ve Yapay Sinir Ağları Yöntemlerinin PISA 2018 Okuma Okuryazarlığı Tahmin Doğruluğunun Araştırılması

Öz

Anahtar Kelimeler

Kaynakça

Ayrıntılar

Birincil Dil

Konular

Bölüm

Yazarlar

Yayımlanma Tarihi

Gönderilme Tarihi

Kabul Tarihi

Yayımlandığı Sayı

DOI

IZ

Kaynak Göster