TY  - JOUR
T1  - Investigation of Prediction Accuracy of Hierarchical Linear Modelling and Artificial Neural Networks Methods on PISA 2018 Reading Literacy
TT  - Hiyerarşik Doğrusal Modellemenin ve Yapay Sinir Ağları Yöntemlerinin PISA 2018 Okuma Okuryazarlığı Tahmin Doğruluğunun Araştırılması
AU  - Akdoğdu Yıldız, Eda
AU  - Atalay Kabasakal, Kübra
PY  - 2025
DA  - August
Y2  - 2025
DO  - 10.17152/gefad.1700937
JF  - Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi
JO  - GUJGEF
PB  - Gazi University
WT  - DergiPark
SN  - 1301-9058
SP  - 543
EP  - 568
VL  - 45
IS  - 2
LA  - en
AB  - In this research, it is aimed to compare hierarchical linear modelling and artificial neural network estimation methods in predicting students' reading comprehension success in the Program for International Student Assessment (PISA) 2018 application. In accordance with this purpose, it is planned to determine how students' PISA success status is estimated at student and school level, the data mining method used in estimation and the explained variance and error values of multilevel modelling. The type of study is, in a way, relational research because of the establishment of models in which there are relationships between dependent and independent variables. On the other hand, it is descriptive research in terms of performing analyses with two methods for each country sampled in the study and comparing the results obtained in terms of explained variance and error values. In this research, the performance of data mining techniques (artificial neural networks – ANN) and multilevel analysis methods (hierarchical linear modeling – HLM) in the field of education is evaluated. It has been determined that HLM carries out the estimation process with lower error and higher R^2 than ANN in the analysis of multi-level data. In addition, HLM provides more information about the predictive level of the variables and the variance that is not explained by the variables in the model compared to ANN. For this reason, HLM analysis was used to examine the variables that affect reading comprehension success in the study. As a result, it was seen that the student level and school level variables added to the model had a statistically significant effect on reading comprehension achievement. While teacher-directed instruction and lack of educational material at school cause negative effects on reading comprehension success, it has been determined that economic-social-cultural situation, metacognitive strategies, disciplinary climate in the classroom, teacher support, and staff shortage variables have positive effects. The results obtained are generally in agreement with similar studies in the literature.
KW  - Hierarchical linear modelling
KW  - Data mining
KW  - Artificial neural networks
KW  - Reading comprehension
KW  - PISA
N2  - Bu araştırmada Uluslararası Öğrenci Değerlendirme Programı (PISA) 2018 uygulamasında öğrencilerin okuduğunu anlama başarısını tahmin etmede hiyerarşik lineer modelleme ve yapay sinir ağları tahmin yöntemlerinin karşılaştırılması amaçlanmaktadır. Bu amaç doğrultusunda; öğrencilerin PISA başarı durumlarının birey ve okul düzeyinde nasıl tahmin edildiği, tahmin etmede kullanılan veri madenciliği yöntemi ve çok düzeyli modellemenin açıklanan varyans ve hata değerlerinin belirlenmesi planlanmaktadır. Çalışmanın türü, bağımlı ve bağımsız değişkenler arasında ilişkilerin bulunduğu modellerin kurulmasından dolayı bir yönüyle ilişkisel araştırmadır. Diğer bir yönüyle ise çalışmada örnekleme alınan her ülke için iki yöntemle analizler gerçekleştirilip elde edilen sonuçların açıklanan varyans ve hata değerleri açısından karşılaştırılması bakımından betimsel araştırma niteliğindedir. Bu araştırmada eğitim alanında da kullanılmaya başlanan veri madenciliği (yapay sinir ağları-YSA) ve çok düzeyli analiz yöntemlerinin (hiyerarşik lineer model-HLM) nasıl performans gösterdiğine ilişkin bulgular elde edilmiştir. HLM’in çok düzeyleri verilerin analizinde YSA’ya göre daha düşük hata ve daha yüksek R^2 ile tahminleme sürecini yürüttüğü belirlenmiştir. Ayrıca HLM değişkenlerin yordama düzeyi ve modelde yer alan değişkenler tarafından açıklanmayan varyans hakkında YSA’ya göre daha fazla bilgi sunmaktadır. Bu sebeple çalışmada okuduğunu anlama başarısını etkileyen değişkenleri incelemek için HLM analizi kullanılmıştır. Sonuç olarak modele eklenen birey düzeyi (düzey 1) ve okul düzeyi (düzey 2) değişkenlerinin okuduğunu anlama başarısı üzerinde istatiksel olarak anlamlı etkisi olduğu görülmüştür. Okuduğunu anlama başarısı üzerinde öğretmen yönlendirmeli öğretim ve okuldaki eğitimsel materyal eksikliği negatif yönlü etkiye sebep olurken ekonomik-sosyal-kültürel durum, üstbiliş stratejileri, sınıftaki disiplin iklimi, öğretmen desteği, personel eksikliği değişkenleri pozitif yönlü etkisi olduğu tespit edilmiştir.  Elde edilen sonuçlar genel olarak literatürde yer alan benzer çalışmalarla uyum göstermektedir.
CR  - Abiodun, O. I., Jantan, A., Omolara, A. E., Dada, K. V., Mohamed, N. A., & Arshad, H. (2018). State-of-the-art in artificial neural network applications: A survey. Heliyon, 4(11), e00938. https://doi.org/10.1016/j.heliyon.2018.e00938
CR  - Akkoyunlu, B. (2008, Mayıs). Bilgi okuryazarlığı ve yaşam boyu öğrenme. 8th International Educational Technology Conference (IECT) toplantısında sunulan bildiri, Anadolu Üniversitesi, Eskişehir.
CR  - Anderson, R.C., Hiebert, E.H., Scott, J.A. & Wilkinson, I.A.G. (1988). Becoming a nation of readers: the report of the commission on reading. Education and Treatment of Children, 11(4), 389-396.
CR  - Anderson, D., & McNeill, G. (1992). Artificial neural networks technology. Kaman Sciences Corporation
CR  - Ali, O., Murray, P. A., Momin, M., Dwivedi, Y. K., & Malik, T. (2023). The effects of artificial intelligence applications in educational settings: Challenges and strategies. Technological Forecasting and Social Change, 199, 123076. https://doi.org/10.1016/j.techfore.2023.123076
CR  - Baba, A. (2024). Neural networks from biological to artificial and vice versa. Biosystems, 235, 105110. https://doi.org/10.1016/j.biosystems.2023.105110
CR  - Bennett, D. A. (2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25(5), 464-469.
CR  - Botchkarev, A. (2019). A new typology design of performance metrics to measure errors in machine learning regression algorithms. Interdisciplinary Journal of Information Knowledge and Management, 14, 045–076. https://doi.org/10.28945/4184
CR  - Chai, T., & Draxler, R. R. (2014). Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature. Geoscientific Model Development, 7(3), 1247–1250. https://doi.org/10.5194/gmd-7-1247-2014
CR  - Chavez, H., Chavez-Arias, B., Contreras-Rosas, S., Alvarez-Rodríguez, J. M., & Raymundo, C. (2023). Artificial neural network model to predict student performance using nonpersonal information. Frontiers in Education, 8. https://doi.org/10.3389/feduc.2023.1106679
CR  - Chen, J., Lin, C. H., & Chen, G. (2021). A cross-cultural perspective on the relationships among social media use, self-regulated learning and adolescents’ digital reading literacy. Computers & Education, 175, 104322.
CR  - Chen, Q., Lei, Y., Wen, Z., Li, S., Li, J., & Kong, Y. (2019). Teacher support, reading strategy and reading literacy: A two-level mediation model. Best Evid Chin Edu, 2(1), 157-170. https://doi.org/10.15354/bece.19.ar1036
CR  - Courtney, M., Karakus, M., Ersozlu, Z., & Nurumov, K. (2022). The influence of ICT use and related attitudes on students’ math and science performance: Multilevel analyses of the last decade’s PISA surveys. Large-Scale Assessments in Education, 10(1), 1-26. https://doi.org/10.1186/s40536-022-00128-6
CR  - Çelenk, S. (2003). Okul aile iş birliği ile okuduğunu anlama başarısı arasındaki ilişki. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 24, 33-39.
CR  - Dong, X., & Hu, J. (2019). An exploration of impact factors influencing students’ reading literacy in Singapore with machine learning approaches. International Journal of English Linguistics, 9(5), 52-65. https://doi.org/10.5539/ijel.v9n5p52
CR  - Ercan, U. (2021). Ev dışı gıda tüketim sınıflarının yapay sinir ağları ile tahmin edilmesi. İşletme Araştırmaları Dergisi, 13(4), 3265-3277.
CR  - Feng, Y., & Jones, K. (2015, July 8-10). Comparing multilevel modelling and artificial neural networks in house price prediction [Paper presentation]. 2015 2nd IEEE International Conference on Spatial Data Mining and Geographical Knowledge Services, Fuzhou, China.
CR  - Fritsch, S., Guenther, F., & Wright, M. N. (2019). neuralnet: Training of Neural Networks. R package version 1.44.2.
CR  - Gamazo, A., & Martínez-Abad, F. (2020). An exploration of factors linked to academic performance in PISA 2018 through data mining techniques. Frontiers in Psychology, 11, 575167. https://doi.org/10.3389/fpsyg.2020.575167
CR  - Guthrie, J. T., & Wigfield, A. (2000). Engagement and motivation in reading. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3, pp. 403–422). Lawrence Erlbaum.
CR  - Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). Springer Science & Business Media.
CR  - Hawkins, D. M. (2003). The problem of overfitting. Journal of Chemical Information and Computer Sciences, 44(1), 1–12. https://doi.org/10.1021/ci0342472
CR  - Haykin, S. (2008). Neural networks: a comprehensive foundation. New Jersey: Prentice-Hall.
CR  - Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2. ed): New York: Routledge.
CR  - Hu, J., & Wang, Y. (2022). Influence of students’ perceptions of instruction quality on their digital reading performance in 29 OECD countries: A multilevel analysis. Computers & Education, 189, 104591. https://doi.org/10.1016/j.compedu.2022.104591
CR  - Kumova M., S. & Kışla, T. (2020). Yapay sinir ağları. In T. Güyer, H. Yurdugül & S. Yıldırım (Ed.), Eğitsel veri madenciliği ve öğrenme algoritmaları (pp. 127-146). Anı Publishing.
CR  - Koyuncu, İ., & Fırat, T. (2021). Investigating Reading Literacy in PISA 2018 assessment. Lnternational Electronic Journal of Elementary Education, 13(2), 263–275. https://doi.org/10.26822/iejee.2021.189
CR  - Kuhn, M. (2022). caret: classification and regression training. R package version 6.0-91.
CR  - Kursa, M. B., & Rudnicki, W. R. (2010). Feature selection with the boruta package. Journal of Statistical Software, 36(11), 1-13.
CR  - Leisch, F., & Dimitriadou, E. (2021). mlbench: Machine learning benchmark problems. R package version 2.1-3.
CR  - Levy, J., Mussack, D., Brunner, M., Keller, U., Cardoso-Leite, P., & Fischbach, A. (2020). Contrasting classical and machine learning approaches in the estimation of Value-Added scores in Large-Scale Educational data. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.02190
CR  - Liaw, A., & Wiener, M. (2002). Classification and Regression by randomForest. R News 2(3), 18-22.
CR  - Lee, Y. H., & Wu, J. Y. (2013). The indirect effects of online social entertainment and information seeking activities on reading literacy. Computers & Education, 67, 168-177. https://doi.org/10.1016/j.compedu.2013.03.001
CR  - Lin, M. B., Groves, W. A., Freivalds, A., Lee, E. G., & Harper, M. (2011). Comparison of artificial neural network (ANN) and partial least squares (PLS) regression models for predicting respiratory ventilation: an exploratory study. European Journal of Applied Physiology, 112(5), 1603–1611. https://doi.org/10.1007/s00421-011-2118-6
CR  - Lim, H. J., & Jung, H. (2019). Factors related to digital reading achievement: A multi-level analysis using international large scale data. Computers & Education, 133, 82–93. https://doi.org/10.1016/j.compedu.2019.01.007
CR  - Ma, L., Xiao, L., & Hau, K. T. (2022). Teacher feedback, disciplinary climate, student self-concept, and reading achievement: A multilevel moderated mediation model. Learning and Instruction, 79, 101602.
CR  - Maas, C. J. M., & Hox, J. J. (2005). Sufficient Sample Sizes for Multilevel Modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1(3), 86–92. https://doi.org/10.1027/1614-2241.1.3.86
CR  - McNeish, D. M., & Stapleton, L. M. (2016). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28(2), 295–314. https://doi.org/10.1007/s10648-014-9287-x
CR  - Michela, J. L. (2006). Software review: HLM 6. Organizational Research Methods, 9(1), 119-122. https://doi.org/10.1177/10944281052812
CR  - Munson, M.A., Caruana, R. (2009). On Feature Selection, Bias-Variance, and Bagging. Buntine. In W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (Eds.), Machine Learning and Knowledge Discovery in Databases. ECML PKDD Lecture Notes in Computer Science, vol 5782. Springer, Berlin, Heidelberg.
CR  - Muratkyzy, A. (2020). Equity and excellence in the Kazakhstani Education System: A multilevel analysis of the personal and contextual factors contributing to students’ reading literacy performance on PISA 2018 [Unpublished master’s thesis]. Nazarbayev University Institute of Educational Sciences, Astana.
CR  - Ni, X. (2008). Research of data mining based on neural networks. World Academy of Science, Engineering and Technology, 39, 381-384.
CR  - OECD. (2019). PISA 2018 results volume I: What students know and can do. Paris: OECD Publishing. https://doi.org/10.1787/5f07c754-en.
CR  - Oreta, A. W. C. (2004). Simulating size effect on shear strength of RC beams without stirrups using neural networks. Engineering Structures, 26(5), 681-691. https://doi.org/10.1016/j.engstruct.2004.01.009
CR  - Radoyevic, N. (2006). Exploring the use of effective learning strategies to increase students’ reading comprehension and test taking skills (Yüksek lisans tezi). Brock Üniversitesi Eğitim Fakültesi, St. Catharines, Ontario.
CR  - Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models. Applications and Data Analysis Methods (2nd ed.). Thousand Oaks, CA: Sage Publications.
CR  - Rose, D. S., Parks, M., Androes, K., & McMahon, S. D. (2000). Imagery-based learning: Improving elementary students' reading comprehension with drama techniques. The Journal of Educational Research, 94(1), 55-63.
CR  - Shahini, A. (2021). Inequalities in Albanian education: Evidence from large-scale assessment studies. Kultura i Edukacja, 4(134), 40-70.
CR  - Snow, C. E. (2002). Reading for Understanding: Toward an R&D Program in Reading Comprehension. RAND Corporation.
CR  - Tang, F., & Ishwaran, H. (2017). Random forest missing data algorithms. Statistical Analysis and Data Mining the ASA Data Science Journal, 10(6), 363–377. https://doi.org/10.1002/sam.11348
CR  - Weng, W., & Luo, W. (2023). A comparative analysis of data mining methods and hierarchical linear modeling using PISA 2018 data. International Journal of Database Management Systems, 15(2/3), 1–16. https://doi.org/10.5121/ijdms.2023.15301
CR  - Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York.
CR  - Willmott, C., & Matsuura, K. (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research, 30, 79–82. https://doi.org/10.3354/cr030079
CR  - Yıldız Aybek, H. S., & Okur, M. R. (2018). Predicting achievement with artificial neural networks: The case of Anadolu University Open Education System. International Journal of Assessment Tools in Education, 5(3), 474-490.
CR  - Yousafzai, B. K., Khan, S. A., Rahman, T., Khan, I., Ullah, I., Ur Rehman, A., Baz, M., Hamam, H., & Cheikhrouhou, O. (2021). Student-Performulator: Student Academic Performance Using Hybrid Deep Neural Network. Sustainability, 13(17), 9775. https://doi.org/10.3390/su13179775
UR  - https://doi.org/10.17152/gefad.1700937
L1  - https://dergipark.org.tr/en/download/article-file/4877457
ER  -