How does the ICT Access and Usage Influence Student Achievement in PISA 2009 and 2012?

Abstract

The purpose of this study is to investigate the effects of access and usage of information and communication technologies (ICT) on Turkish students’ mathematics achievement implemented in PISA 2009 and PISA 2012. A correlational research model was used in this study. In this study, the data which were obtained from the PISA 2009 and PISA 2012 mathematics achievement tests and from the information and communications technologies familiarity questionnaire (ICTFQ) in Turkey were used. In this study, three student level variables and two school variables of ICTFQ which are common indexes both in PISA 2009 and PISA 2012 were selected to compare the effect of ICT variables on PISA mathematics achievement implemented in different years. Two-level Hierarchical Linear Modeling (HLM) analysis was performed in the analysis of the data. As a result, the student level variables had a small or a trivial effect on mathematics achievement. The effect size value of the ENTUSE variable was similar in the PISA 2009 and the PISA 2012 implementation, but the effect size value of the HOMSCH variable and the ICTHOME variable on mathematics achievement in PISA 2012 was lower than in PISA 2009. The ICTSCH and the USESCH variables at the school level had a large effect on mathematics achievement in two implementations of PISA 2009 and PISA 2012. The effect size value of the ICTSCH variable on mathematics achievement in PISA 2012 was higher than in PISA 2009. The effect size value of the ICTSCH variable, having a negative relationship with mathematics achievement in PISA 2012, was lower than in PISA 2009. In this study, the explained variance ratio of mathematics achievement by the school ICT variables level was greater than by the student ICT variables level.

Keywords

• Information and Communication Technologies (ICT)
• mathematics achievement
• PISA 2009
• PISA 2012
• two-level hierarchical linear models

References

1. Acar, T. (2012). 2009 yılı uluslararası öğrenci başarılarını değerlendirme programı’nda Türk öğrencilerin başarılarını etkileyen faktörler. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 3(2), 309-314. http://dergipark.org.tr/tr/pub/epod/issue/5802/77221 adresinden edinilmiştir.
2. Akın, Y., & Cancan, M. (2007). Matematik öğretiminde problem çözümüne yönelik öğrenci görüşleri analizi. Atatürk Üniversitesi Kazım Karabekir Eğitim Fakültesi Dergisi, (16), 374-390. http://dergipark.org.tr/tr/pub/ataunikkefd/issue/2777/37247 adresinden edinilmiştir.
3. Alakoç, Z. (2003). Matematik öğretiminde teknolojik modern öğretim yaklaşımları. TOJET: The Turkish Online Journal of Educational Technology, 2(1), 43-49. http://tojet.net/articles/v2i1/217.pdf adresinden edinilmiştir.
4. Anderson, R. E. (2008). Implications of the information and knowledge society for education. In J. Voogt & G. Knezek (Eds.), International handbook of information technology in primary and secondary education (pp. 5-22). Boston, MA: Springer.
5. Aypay, A. (2010). Information and communication technology (ICT) usage and achievement of Turkish students in PISA 2006. Turkish Online Journal of Educational Technology (TOJET), 9(2), 116-124. Retrieved from https://files.eric.ed.gov/fulltext/EJ898009.pdf
6. Baki, A., Yalçınkaya, H. A., Özpınar, İ., & Uzun, S. Ç. (2009). İlköğretim matematik öğretmenleri ve öğretmen adaylarının öğretim teknolojilerine bakış açılarının karşılaştırılması. Turkish Journal of Computer and Mathematics Education, 1(1), 67-85. http://dergipark.org.tr/tr/pub/turkbilmat/issue/21560/231419 adresinden edinilmiştir.
7. Balanskat, A., Bannister, D., Hertz, B., Sigillò, E., & Vuorikari, R. (2013). Overview and analysis of 1: 1 learning initiatives in Europe. In S. Bocconi, A. Balanskat, P. Kampylis, & Y. Punie (Eds.), Overview and analysis of 1: 1 learning initiatives in Europe (pp. 1-166). Spain, Luxembourg: European Commission. doi: 10.2791/20333.
8. Balanskat, A., Blamire, R., & Kefala, S. (2006). The ICT impact report: A review of studies of ICT impact on schools in Europe. European Schoolnet, 1, 1-71. Retrieved from http://colccti.colfinder.org/sites/default/files/ict_impact_report_0.pdf

9. Barkatsas, A. T., Kasimatis, K., & Gialamas, V. (2009). Learning secondary mathematics with technology: Exploring the complex interrelationship between students’ attitudes, engagement, gender and achievement. Computers & Education, 52(3), 562-570. doi: 10.1016/j.compedu.2008.11.001
10. Becker, H. J. (2000). Pedagogical motivations for student computer use that lead to student engagement. Educational Technology, 40(5), 5-17. Retrieved from https://www.jstor.org/stable/pdf/44428608.pdf
11. Biagi, F., & Loi, M. (2013). Measuring ICT use and learning outcomes: Evidence from recent econometric studies. European Journal of Education, 48(1), 28-42. doi: 10.1111/ejed.12016
12. Bilican-Demir, S., & Yıldırım, Ö. (2016). Okulda ve okul dışında bilgi ve iletişim teknolojilerinin kullanımının öğrencilerin PISA 2012 performansıyla ilişkisinin incelenmesi. Kastamonu Eğitim Dergisi - Kastamonu Education Journal, 24(1), 251-262. http://dergipark.org.tr/tr/pub/kefdergi/issue/22606/241619 adresinden edinilmiştir.
13. Bingimlas, K. A. (2009). Barriers to the successful integration of ICT in teaching and learning environments: A review of the literature. Eurasia Journal of Mathematics, Science & Technology Education, 5(3), 235-245. doi: 10.12973/ejmste/75275
14. Cheema, J. R., & Hang, B. (2013). Quantity and quality of computer use and academic achievement: Evidence from a large-scale international test program. International Journal of Education and Development using Information and Communication Technology, 9(2), 95-106. Retrieved from http://ijedict.dec.uwi.edu/viewissue.php?id=35
15. Cox, M. J., & Marshall, G. M. (2007). Effects of ICT: Do we know what we should know? Education and Information Technologies, 12(2), 59-70. doi: 10.1007/s10639-007-9032-x De Witte, K., & Rogge, N. (2014). Does ICT matter for effectiveness and efficiency in mathematics education? Computers & Education, 75, 173-184. doi: 10.1016/j.compedu.2014.02.012
16. Delen, E., & Bulut, O. (2011). The relationship between students’ exposure to technology and their achievement in science and math. TOJET, 10(3), 311-317. Retrieved from http://www.tojet.net/articles/v10i3/10336.pdf
17. Demir, İ., & Kılıç, S. (2009). Effects of computer use on students’ mathematics achievement in Turkey. Procedia Social and Behavioral Sciences, 1(1), 1802-1804. doi: 10.1016/j.sbspro.2009.01.319
18. Demir, İ., Kılıç, S., & Ünal, H. (2010). Effects of students’ and schools’ characteristics on mathematics achievement: findings from PISA 2006. Procedia-Social and Behavioral Sciences, 2(2), 3099-3103. doi: 10.1016/j.sbspro.2010.03.472
19. Demiraslan, Y., & Usluel, Y. K. (2005). Bilgi ve iletişim teknolojilerinin öğrenme öğretme sürecine entegrasyonunda öğretmenlerin durumu. The Turkish Online Journal of Educational Technology, 4(3), 109-114. http://www.tojet.net/articles/v4i3/4315.pdf adresinden edinilmiştir.
20. Dumais, S. A. (2009). Cohort and gender differences in extracurricular participation: The relationship between activities, math achievement, and college expectations. Sociological Spectrum, 29(1), 72-100. doi: 10.1080/02732170802480543
21. Erdoğdu, F., & Erdoğdu, E. (2015). The impact of access to ICT, student background and school/home environment on academic success of students in Turkey: An international comparative analysis. Computers & Education, 82, 26-49. doi: 10.1016/j.compedu.2014.10.023
22. Erstad, O. (2009). Addressing the complexity of impact. A multilevel approach towards ICT in education. In F. Scheuermann & F. Pedró (Eds.), Assessing the effects of ICT in education Indicators, criteria and benchmarks for international comparisons (pp. 21-40). Luxembourg: Publications Office of the European Union.
23. Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255-284. doi: 10.1080/15391523.2010.10782551
24. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2011). How to design and evaluate research in education (8th ed.). New York, NY: McGraw-Hill Humanities/Social Sciences/Languages.
25. Grüsser, S. M., Thalemann, R., & Griffiths, M. D. (2006). Excessive computer game playing: Evidence for addiction and aggression? CyberPsychology and Behavior, 10(2), 290-292. doi: 10.1089/cpb.2006.9956
26. Güven, S., & Demirçelik, D. A. (2013). 6. 7. ve 8. sınıf öğrencilerin performans ödevleri hakkındaki görüşleri ve bu ödevi hazırlamaya yönelik etik algıları. Uluslararası Avrasya Sosyal Bilimler Dergisi-International Journal of Eurasia Social Sciences, 4(13), 83-104. http://www.ijoess.com/Makaleler/271634930_sevim%20g%C3%BCven-performan%20%C3%B6devlerine%20y%C3%B6nelik.pdf adresinden edinilmiştir.
27. Güzeller, C. O., & Akın, A. (2014). Relationship between ICT variables and mathematics achievement based on PISA 2006 database: International evıdence. TOJET, 13(1), 184-192. Retrieved from https://files.eric.ed.gov/fulltext/EJ1018171.pdf
28. Hamlen, K. R. (2011). Children’s choices and strategies in video games. Computers in Human Behavior, 27(1), 532-539. doi: 10.1016/j.chb.2010.10.001
29. Heinz, J. (2016). Digital skills and the influence of students’ socio-economic background: An exploratory study in German elementary schools. Italian Journal of Sociology of Education, 8(2), 186-212. doi: 10.14658/pupj-ijse-2016-2-9
30. Hofmann, D. A., & Gavin, M. B. (1998). Centering decisions in hierarchical linear models: Implications for research in organizations. Journal of Management, 24(5), 623-641. doi: 10.1177/014920639802400504
31. Hu, X., Gong, Y., Lai, C., & Leung, F. K. (2018). The relationship between ICT and student literacy in mathematics, reading, and science across 44 countries: A multilevel analysis. Computers & Education, 125, 1-13. doi: 10.1016/j.compedu.2018.05.021
32. İlgün-Dibek, M., Yalçın, S., & Yavuz, H. Ç. (2016). Matematik okuryazarlığı ile bilgi ve iletişim teknolojileri kullanım becerileri arasındaki ilişki: PISA 2012. Ahi Evran Üniversitesi Kırşehir Eğitim Fakültesi Dergisi (KEFAD)-Ahi Evran University Journal of Kırşehir Education Faculty, 17(3), 39-58. http://kefad.ahievran.edu.tr/Kefad/ArchiveIssues/PDF/ad952109-a151-e711-80ef-00224d68272d adresinden edinilmiştir.
33. Jang, S. J. (2009). Exploration of secondary students’ creativity by integrating web-based technology into an innovative science curriculum. Computers & Education, 52(1), 247-255. doi: 10.1016/j.compedu.2008.08.002
34. Karabay, E., Yıldırım, A., & Güler, G. (2015). Yıllara göre PISA matematik okuryazarlığının öğrenci ve okul özellikleri ile ilişkisinin aşamalı doğrusal modeller ile analizi. Mehmet Akif Ersoy Üniversitesi Eğitim Fakültesi Dergisi, 1(36), 137-151. http://dergipark.org.tr/tr/pub/maeuefd/issue/19409/206317 adresinden edinilmiştir.
35. Kayaduman, H., Sırakaya, M., & Seferoğlu, S. S. (2011, Şubat). Eğitimde FATİH projesinin öğretmenlerin yeterlik durumları açısından incelenmesi. XIII. Akademik Bilişim Konferansı- XIII. Conference of Academic Informatics, Malatya, Türkiye.
36. Kim, H. S., Kil, H. J., & Shin, A. (2014). An analysis of variables affecting the ICT literacy level of Korean elementary school students. Computers & Education, 77, 29-38. doi: 10.1016/j.compedu.2014.04.009
37. Kubiatko, M., & Vlckova, K. (2010). The relationship between ICT use and science knowledge for Czech students: A secondary analysıs of PISA 2006. International Journal of Science and Mathematics Education, 8(3), 523-543. doi: 10.1007/s10763-010-9195-6
38. Lazakidou, G., & Retalis, S. (2010). Using computer supported collaborative learning strategies for helping students acquire self-regulated problem-solving skills in mathematics. Computers & Education, 54(1), 3-13. doi: 10.1016/j.compedu.2009.02.020
39. Lei, J., & Zhao, Y. (2007). Technology uses and student achievement: A longitudinal study. Computers & Education, 49(2), 284-296. doi: 10.1016/j.compedu.2005.06.013
40. Li, X., & Atkins, M. S. (2004). Early childhood computer experience and cognitive and motor development. Pediatrics, 113(6), 1715-1722. doi: 10.1542/peds.113.6.1715
41. Livingstone, S. (2012). Critical reflections on the benefits of ICT in education. Oxford Review of Education, 38(1), 9-24. doi: 10.1080/03054985.2011.577938
42. Luu, K., & Freeman, J. G. (2011). An analysis of the relationship between information and communication technology (ICT) and scientific literacy in Canada and Australia. Computers & Education, 56(4), 1072-1082. doi: 10.1016/j.compedu.2010.11.008
43. McMahon, G. (2009). Critical thinking and ICT integration in a Western Australian secondary school. Educational Technology & Society, 12(4), 269-281. Retrieved from https://www.j-ets.net/collection/published-issues/12_4
44. Milli Eğitim Bakanlığı. (2010). PISA 2009 Ulusal ön raporu. Ankara: Eğitimi Araştırma ve Geliştirme Dairesi Başkanlığı.
45. Milli Eğitim Bakanlığı. (2011). İlköğretim okullarındaki (1-5. Sınıf) ödev uygulamalarının değerlendirilmesi araştırması. Ankara: Eğitim Araştırma ve Geliştirme Dairesi Yayınları.
46. Milli Eğitim Bakanlığı. (2013a). Ortaokul öğretim matematik dersi (5, 6, 7 ve 8. sınıflar) öğretim programı. Ankara: MEB Yayınları.
47. Milli Eğitim Bakanlığı. (2013b). PISA 2012 Ulusal ön raporu. Ankara: Yenilik ve Eğitim Teknolojileri Genel Müdürlüğü.
48. Murphy, D. (2016). A literature review: The effect of implementing technology in a high school mathematics classroom. International Journal of Research in Education and Science (IJRES), 2(2), 295-299. Retrieved from https://www.ijres.net/index.php/ijres/article/view/109/73
49. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: National Council of Teachers of Mathematics
50. National Council of Teachers of Mathematics. (2014). Principles to actions: Ensuring mathematical success for all. Reston, VA: National Council of Teachers of Mathematics
51. O’Neil, H. F., Wainess, R., & Baker, E. L. (2005). Classification of learning outcomes: Evidence from the computer games literature. The Curriculum Journal, 16(5), 455-474. doi: 10.1080/09585170500384529
52. OECD.Stat (2018). Student-teacher ratio and average class size. Retrieved from: https://stats.oecd.org/Index.aspx?DataSetCode=EAG_PERS_RATIO#.
53. Olkun, S., & Altun, A. (2003). İlköğretim öğrencilerinin bilgisayar deneyimleri ile uzamsal düşünme ve geometri başarıları arasındaki ilişki. The Turkish Online Journal of Educational Technology, 2(4), 86-91. http://www.tojet.net/volumes/v2i4.pdf#page=86 adresinden edinilmiştir.
54. Organisation for Economic Co-operation and Development. (2012). PISA 2009 technical report. Paris: OECD Publishing. doi: 10.1787/9789264167872-en
55. Organisation for Economic Co-operation and Development. (2013). PISA 2012 assessment and analytical framework. Mathematics, reading, science, problem solving and financial literacy. Paris: OECD Publishing.
56. Organisation for Economic Co-operation and Development. (2014a). PISA 2012 results: What students know and can do (Volume I, Revised Edition, February 2014): Student performance in mathematics, reading and science. Paris: OECD Publishing. Retrieved from https://www.oecd.org/pisa/keyfindings/pisa-2012-results-volume-I.pdf
57. Organisation for Economic Co-operation and Development. (2014b). PISA 2012 technical report. Paris: OECD Publishing. Retrieved from https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf
58. Organisation for Economic Co-operation and Development. (2014c). Scaling procedures and construct validation of context questionnaire data. Paris: OECD Publishing. Retrieved from https://www.oecd.org/pisa/pisaproducts/PISA%202012%20Technical%20Report_Chapter%2016.pdf
59. Organisation for Economic Co-operation and Development. (2017). PISA 2015 assessment and analytical framework. Science, reading, mathematic, financial literacy and collaborative problem solving, revised edition. Paris: OECD Publishing. doi: 10.1787/9789264281820-en
60. Organisation for Economic Co-operation and Development. (2018a). PISA database. Retrieved from http://www.oecd.org/pisa/data/pisa2009database-downloadabledata.htm
61. Organisation for Economic Co-operation and Development. (2018b). PISA database. Retrieved from http://www.oecd.org/pisa/data/pisa2012database-downloadabledata.htm
62. Özer, Y., & Anıl, D. (2011). Öğrencilerin fen ve matematik başarılarını etkileyen faktörlerin yapısal eşitlik modeli ile incelenmesi. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi- H. U. Journal of Education, (41), 313-324. http://www.efdergi.hacettepe.edu.tr/shw_artcl-702.html adresinden edinilmiştir.
63. Pamuk, S., Çakır, R., Ergun, M., Yılmaz, H. B., & Ayas, C. (2013). Öğretmen ve öğrenci bakış açısıyla tablet pc ve etkileşimli tahta kullanımı: FATİH projesi değerlendirmesi. Kuram ve Uygulamada Eğitim Bilimleri - Educational Sciences: Theory & Practice, 13(3), 1799-1822.
64. Pandolfini, V. (2016). Exploring the impact of ICTs in education: Controversies and challenges. Italian Journal of Sociology of Education, 8(2), 28-53. doi: 10.14658/pupj-ijse-2016-2-3
65. Petko, D., Cantieni, A., & Prasse, D. (2017). Perceived quality of educational technology matters: A secondary analysis of students’ ICT use, ICT-related attitudes, and PISA 2012 test scores. Journal of Educational Computing Research, 54(8), 1070-1091. doi: 10.1177/0735633116649373
66. Raudenbush, S. W., & Bryk, A. (2002). Hierarchical linear models: Applications and data analysis methods (2nd edition). London: Sage.
67. Rosenthal, R., & Rosnow, R. L. (2008). Essentials of behavioral research: Methods and data analysis (3rd ed.). New York, NY: McGraw-Hill
68. Scheuermann, F., & Pedró, F. (Eds.). (2009). Assessing the effects of ICT in education: Indicators, criteria and benchmarks for international comparisons. Luxembourg: Publications Office of the European Union.
69. Seferoğlu, S. S. (2015). Okullarda teknoloji kullanımı ve uygulamalar: Gözlemler, sorunlar ve çözüm önerileri. Artı Eğitim, 123, 90-91. http://www.egitimtercihi.com/okulgazetesi/17207-okullarda-teknoloji-kullan-m-ve-uygulamalar.html adresinden edinilmiştir.
70. Shaikh, Z., A., & Khoja, S. A. (2011). Role of ICT in shaping the future of Pakistani higher education system. The Turkish Online Journal of Educational Technology, 10(1), 149-161.
71. Skryabin, M., Zhang, J., Liu, L., & Zhang, D. (2015). How the ICT development level and usage influence student achievement in reading, mathematics, and science? Computers & Education, 85, 49-58. doi: 10.1016/j.compedu.2015.02.004
72. Subrahmanyam, K., Greenfield, P., Kraut, R., & Gross, E. (2001). The impact of computer use on children’s and adolescents’ development. Journal of Applied Developmental Psychology, 22(1), 7-30. doi: 10.1016/S0193-3973(00)00063-0
73. Sutherland, R., Robertson, S., & John, P. (2009). Improving classroom learning with ICT. London: Routledge.
74. Şengül, M., & Demir, E. (2018). Farklı ülkelerdeki öğrencilerin bilgi-iletişim teknolojilerine aşinalıklarının çeşitli değişkenlere göre sınıflama doğruluklarının incelenmesi. Electronic Journal of Social Sciences, 17(68), 386-1409. doi: 10.17755/esosder.345757
75. Trucano, M. (2005). Knowledge maps: ICT in education. Washington, DC: infoDev/World Bank.
76. Ural, A. (2015). Ortaokul matematik öğretmenlerinin bilgi iletişim teknolojisi ve psikomotor beceri kullanımlarının incelenmesi. Turkish Journal of Computer and Mathematics Education, 6(1), 93-116. doi: 10.16949/turcomat.18249
77. von Secker, C. E., & Lissitz, R. W. (1999). Estimating the impact of instructional practices on student achievement in science. Journal of Research in Science Teaching, 36(10), 1110-1126. doi: 10.1002/(SICI)1098-2736(199912)36:10<1110::AID-TEA4>3.0.CO;2-T
78. Wittwer, J., & Senkbeil, M. (2008). Is students’ computer use at home related to their mathematical performance at school? Computers & Education, 50(4), 1558-2571. doi: 10.1016/j.compedu.2007.03.001
79. Yenilmez, K., & Dereli, A. (2009). İlköğretim okullarında matematiğe karşı olumsuz önyargı oluşturan etkenler. e-Journal of New World Sciences Academy Education Sciences, 4(1), 25-33. http://dergipark.org.tr/tr/pub/nwsaedu/issue/19829/212470 adresinden edinilmiştir.
80. Yorgancı, S., & Terzioğlu, Ö. (2013). Matematik öğretiminde akıllı tahta kullanımının başarıya ve matematiğe karşı tutuma etkisi. Kastamonu Eğitim Dergisi - Kastamonu Education Journal, 21(3), 919-930. http://dergipark.org.tr/tr/pub/kefdergi/issue/22605/241582 adresinden edinilmiştir.
81. Youssef A. B., & Dahmani M. (2008). The impact of ICT on student performance in higher education: Direct effects, indirect effects, and organizational change. Revista de Universidad y Sociedad del Conocimiento, 5(1), 45-56. doi: 10.7238/rusc.v5i1.321
82. Yusuf, M. O., & Afolabi, A. O. (2010). Effects of computer assisted instruction (CAI) on secondary school students’ performance in biology. The Turkish Online Journal of Educational Technology, 9(1), 62-69. Retrieved from https://files.eric.ed.gov/fulltext/EJ875764.pdf
83. Zengin, Y., Kağızmanlı, T. B., Tatar, E., & İşleyen, T. (2013). Bilgisayar destekli matematik öğretimi dersinde dinamik matematik yazılımının kullanımı. Mustafa Kemal Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 10(23), 167-180. http://dergipark.org.tr/tr/pub/mkusbed/issue/19550/208256 adresinden edinilmiştir.
84. Zhang, D., & Liu, L. (2016). How does ICT use influence students’ achievements in math and science over time? Evidence from PISA 2000 to 2012. Eurasia Journal of Mathematics, Science & Technology Education, 12(9), 2431-2449. doi: 10.12973/eurasia.2016.1297a
85. Ziya, E., Doğan, N., & Kelecioğlu, H. (2010). What is the predict level of which computer using skills measured in PISA for achievement in math. The Turkish Online Journal of Educational Technology, 9(4), 185-191. Retrieved from https://files.eric.ed.gov/fulltext/EJ908084.pdf