Data Literacy at School: A Scale Development Study

Abstract

The purpose of this study is to develop a valid and reliable scale to determine and evaluate the different dimensions of data literacy at school. The study is a quantitative descriptive survey model. The sampling for exploratory factor analysis was formed of 307 and confirmatory factor analysis 338 teachers and school administrators who are on active duty in 2023-2024 educational year in Kastamonu. Data was collected through a five item likert data collection tool. A three-dimension structure was formed and it was confirmed by CFA. The dimensions of data culture at school are; “data identification”, “data use” and “data management”. Internal reliability and validity was verified through Cronbach Alpha (Cronbach’s α=.882), split half method (r=.837), Spearman-Brown correlation coefficient (R=.911) and Guttman’s lambda (λ=.904). The external reliability and validity was verified by test-retest technique (first application n=44, second application n=39, r=.800, p≤.05, R=.961, p≤.05, and Kendal’s tau-b is τb=.904, p≤.05). The findings confirmed the validity and reliability of the scale.

Keywords

• data literacy
• data identification
• data culture
• data management

References

1. Abrams, L. M., Varier, D., & Mehdi, T. (2021). The intersection of school context and teachers’ data use practice: Implications for an integrated approach to capacity building. Studies in Educational Evaluation, 69(2021), 1-13. https://doi.org/10.1016/j.stueduc.2020.100868
2. Anderson, C. (2015). Creating a data driven organization: Practical advice from the trenches. O’Reilly Media, Inc.
3. Anderson, S., Leithwood, K., & Strauss, T. (2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9(3), 292-327. https://doi.org/10.1080/15700761003731492
4. Aydoğdu, F. (2023). AVE ve CR hesaplama. Retrieved from https://www.fuataydogdu.com/avecr
5. Bai, J., & Ng, S. (2005). Tests for skewness, kurtosis, and normality for time series data. Journal of Business & Economic Statistics, 23(1), 49-60. http://www.columbia.edu/~sn2294/pub/jbes-05.pdf
6. Bartlett, M. S. (1950). Tests of significance in factor analysis. British Journal of Psychology, 5(2), 109-133.
7. Bennett, C., Graham, I. D., Kristjansson, E., Kearing, S. A., Clay, K. F., & O’Connor, A. M. (2010). Validation of a preparation for decision making scale. Patient Education and Counseling, 78(1), 130-133. https://doi.org/10.1016/j.pec.2009.05.012
8. Breiter, A., & Light, D. (2006). Data for school improvement: Factors for designing effective information systems to support decision-making in schools. Educational Technology & Society, 9(3), 206-217. https://www.ifib-consult.de/publikationsdateien/Breiter_Light-2006.pdf

9. Brown, T. A. (2006). Confirmatory factor analysis for applied research. In D. A. Kenny (Edt.). Methodology in the social sciences (pp. 1-100). The Guilford Press. https://epdf.tips/confirmatory-factor-analysis-for-applied-research.html
10. Büyüköztürk, Ş., Çokluk, Ö., & Köklü, N. (2013). Sosyal bilimler için istatistik (13th ed.). Pegem Akademi.
11. Büyüköztürk, Ş., Çakmak, E. K., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2008). Bilimsel araştırma yöntemleri (2nd ed). Pegem Akademi.
12. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5
13. Chigwada, J., Chiparausha, B., & Kasiroori, J. (2017). Research data management in research institutions in Zimbabwe. Data Science Journal, 16(31), 1-9. https://doi.org/10.5334/dsj-2017-031
14. Coburn, C. E., & Turner, E. O. (2012). The practice of data use: An introduction. American Journal of Education, 118(2), 99-111. https://www.journals.uchicago.edu/doi/abs/10.1086/663272
15. Coburn, C. E., & Turner, E. O. (2011). Research on data use: A framework and analysis. Measurement: Interdisciplinary Research & Perspective, 9(4), 173–206. https://doi.org/10.1080/15366367.2011.626729
16. Cohen, L., Manion, L. & Morrison, K. (2018). Research methods in education (8th ed.). Routledge.
17. Custer, S., King, E. M., Atinc, T. M., Read, L., & Sethi, T. (2018). Toward data-driven education systems: Insights into using information to measure results and manage change. Center for Universal Education at Brookings. https://www.brookings.edu/wp-content/uploads/2018/02/toward-data-driven-education-systems.pdf
18. Çokluk, Ö., Şekercioğlu, G. & Büyüköztürk, Ş. (2018). Sosyal bilimler için çok değişkenli istatistik: SPSS ve LISREL uygulamaları (5. bs). Pegem Akademi.
19. Datnow, A., & Park, V. (2014). Data-driven leadership. John Wiley & Sons.
20. Dogan, E., & Ottekin Demirbolat, A. (2021). Data-driven decision-making in schools scale: A study of validity and reliability. International Journal of Curriculum and Instruction, 13(1), 507-523. https://files.eric.ed.gov/fulltext/EJ1285547.pdf
21. Duygulu, A. (2023). Veri yönetimi. Kriter Yayınevi.
22. Earl, L., & Fullan, M. (2003). Using data in leadership for learning. Cambridge Journal of Education, 33(3), 383-394. https://doi.org/10.1080/0305764032000122023
23. Engmann, S., & Cousineau, D. (2011). Comparing distributions: The two-sample Anderson-Darling test as an alternative to the Kolmogorov-Smirnoff test. Journal of Applied Quantitative Methods, 6(3), 1-38.
24. Field, A. (2009). Discovering statistics using SPSS (3rd ed). Sage Publications Ltd.
25. Filderman, M. J., Toste, J. R., Didion, L., & Peng, P. (2022). Data literacy training for K–12 teachers: A meta-analysis of the effects on teacher outcomes. Remedial and Special Education, 43(5), 328-343. https://files.eric.ed.gov/fulltext/EJ1350487.pdf
26. Fontichiaro, K., & Oehrli, J. A. (2016). Why data literacy matters. Knowledge Quest, 44(5), 21-27. https://files.eric.ed.gov/fulltext/EJ1099487.pdf
27. French, B. F., & Finch, W. H. (2006). Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling, 13(3), 378-402.
28. Goldring, E., & Berends, M. (2009). Leading with data: Pathways to improve your school. Corwin Press, A Sage Company.
29. Gordon, K. (2007). Principles of data management: Facilitating information sharing. Swindon: The British Computer Society Publishing and Information Products.
30. Hair, J. F., Black, W. C., Babin, B. J. & Anderson, R. E. (2014). Multivariate data analysis (7th ed). Pearson Education Limited.
31. Harrington, D. (2009). Confirmatory factor analysis. In T. Tripodi (Ed.), Pocket guides to social work reseacrh methods (pp. 3-121). Oxford University Press. https://l24.im/s4Y6den
32. Hatchavanich, D. (2014). A comparison of type I error and power of Bartlett’s test, Levene’s test and O’Brien’s test for homogeneity of variance tests. Southeast Asian Journal of Sciences, 3(2), 181-194. https://sajs.ntt.edu.vn/index.php/jst/article/view/106
33. Heiman, G. W. (2011). Basic statistics for the behavioral sciences (6th ed.). Wadsworth, Cengage Learning.
34. Henderson, J., & Corry, M. (2021). Data literacy training and use for educational professionals. Journal of Research in Innovative Teaching & Learning, 14(2), 232-244. https://doi.org/10.1108/JRIT-11-2019-0074
35. Howitt, D., & Cramer, D. (2017). Research methods in psychology (5th ed). Pearson Education Limited.
36. Ikemoto, G. S., & Marsh, J. A. (2007). Cutting through the “data-driven” mantra: Different conceptions of data-driven decision making. Teachers College Record, 109(13), 105-131. https://doi.org/10.1177/016146810710901310
37. Kippers, W. B., Poortman, C. L., Schildkamp, K., & Visscher, A. J. (2018). Data literacy: What do educators learn and struggle with during a data use intervention? Studies in Educational Evaluation, 56(2018), 21-31. https://doi.org/10.1016/j.stueduc.2017.11.001
38. Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed). Guilford Press.
39. Knapp, M. S., Copland, M A., & Swinnerton, J. A. (2007). Understanding the promise and dynamics of data-informed leadership. Yearbook of the National Society for the Study of Education, 106(1), 74-104. https://doi.org/10.1111/j.1744-7984.2007.00098.x.
40. Li, L., & Bentler, P. M. (2011). Quantified choice of root-mean-square errors of approximation for evaluation and power analysis of small differences between structural equation models. Psychological Methods, 16(2), 116–126. https://doi.org.10.1037/a0022657
41. Little, J. W. (2012). Understanding data use practice among teachers: The contribution of micro-process studies. American Journal of Education, 118(2), 143-166. https://www.jstor.org/stable/10.1086/663271?origin=JSTOR-pdf
42. Mandinach, E. B., & Gummer, E. S. (2013). A systemic view of implementing data literacy in educator preparation. Educational Researcher, 42(1), 30-37. https://doi.org/10.3102/0013189x12459803
43. Martin, W. E., & Bridgmon, K. D. (2012). Quantitative and statistical research methods: From hypothesis to results. Jossey-Bass.
44. Matthews, P. (2016). Data literacy conceptions, community capabilities. The Journal of Community Informatics, 12(3), 47-56. https://openjournals.uwaterloo.ca/index.php/JoCI/article/download/3277/4300/ McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods 23(3), 412-434. http://dx.doi.org/10.1037/met0000144
45. Naillioğlu Kaymak, M., & Doğan, E. (2023). Veri Okuryazarliği Ölçeği’nin Türk kültürüne uyarlanmasi. Trakya Journal of Education, 13(2). https://dergipark.org.tr/tr/download/article-file/2521835
46. Nordstokke, D. W., & Colp, S. M. (2014). Investigating the robustness of the nonparametric Levene test with more than two groups. Psicológica, 35(2), 361-383. https://www.redalyc.org/pdf/169/16931314010.pdf
47. Obst, P. L., & White, K. M. (2004). Revisiting the sense of community index: A confirmatory factor analysis. Journal of Community Psychology, 32(6), 691-705. https://eprints.qut.edu.au/604/2/604.pdf
48. Pangrazio, L., & Sefton-Green, J. (2020). The social utility of ‘data literacy’. Learning, Media and Technology, 45(2), 208-220. https://sci-hub.se/10.1080/17439884.2020.1707223
49. Pangrazio, L., & Selwyn, N. (2019). ‘Personal data literacies’: A critical literacies approach to enhancing understandings of personal digital data. New Media & Society, 21(2), 419-437. https://doi.org/10.1177/1461444818799523
50. Prudon, P. (2015). Confirmatory factor analysis as a tool in research using questionnaires: A critique. Comprehensive Psychology, 4(10), 1-17. https://journals.sagepub.com/doi/pdf/10.2466/03.CP.4.10
51. Qin, J., & D'Ignazio, J. (2010). Lessons learned from a two-year experience in science data literacy education. Proceedings of the International Association of Scientific and Technological University Libraries, 31st Annual Conference, 5(2010), 1-11. http://docs.lib.purdue.edu/iatul2010/conf/day2/5
52. Ridsdale, C., Rothwell, J., Smit, M., Ali-Hassan, H., Bliemel, M., Irvine, D., Kelley, D., Matwin, S., & Wuetherick, B. (2015). Strategies and best practices for data literacy education: Knowledge synthesis report. Dalhousie University.
53. Schildkamp, K., & Poortman, C. (2015), Factors influencing the functioning of data teams. Teachers College Record, 117(4), 1-42. https://doi.org/10.1177/016146811511700403
54. Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323-338. https://doi.org/10.3200/JOER.99.6.323-338
55. Shek, D. T., & Yu, L. (2014). Confirmatory factor analysis using AMOS: A demonstration. International Journal on Disability and Human Development, 13(2), 191-204. https://www.academia.edu/download/54409775/cfa-amos.pdf
56. Shields, M. (2005). Information literacy, statistical literacy, data literacy. IASSIST Quarterly, 28(2-3), 6-11. https://iassistquarterly.com/index.php/iassist/article/view/790/782
57. Singh, K. (2007). Quantitative social research methods. Sage Publications Inc.
58. Sivo, S. A., Fan, X., Witta, E. L., & Willse, J. T. (2006). The search for “optimal" cutoff properties: Fit index criteria in structural equation modeling. The Journal of Experimental Education, 74(3), 267-288. https://doi.org/10.3200/JEXE.74.3.267-288
59. Stephenson, E., & Schifter Caravello, P. (2007). Incorporating data literacy into undergraduate information literacy programs in the social sciences: A pilot project. Reference Services Review, 35(4), 525-540. https://doi.org/10.1108/00907320710838354
60. Stockemer, D. (2019). Quantitative methods for the social sciences: A practical introduction with examples in SPSS and stata. Springer International Publishing.
61. Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed). Pearson Education, Inc.
62. Thompson, B. L., Green, S. B., & Yang, Y. (2010). Assessment of the maximal split-half coefficient to estimate reliability. Educational and Psychological Measurement, 70(2), 232-251.
63. Van Audenhove, L., Van den Broeck, W., & Mariën, I. (2020). Data literacy and education: Introduction and the challenges for our field. Journal of Media Literacy Education, 12(3), 1-5. https://doi.org/10.23860/JMLE-2020-12-3-1
64. van Geel, M., Keuning, T., Visscher, A., & Fox, J. P. (2017). Changes in educators' data literacy during a data-based decision making intervention. Teaching and Teacher Education, 64(2017), 187-198. http://dx.doi.org/10.1016/j.tate.2017.02.015
65. Vanhoof, J., Verhaeghe, G., Petegem, P., & Valcke, M. (2013). Improving data literacy in schools: Lessons from the school feedback project. In, K. Schildkamp, M. Lai, & L. Earl (Eds.), Data-based decision making in education: Studies in educational leadership, volume 17 (pp. 113-134). Springer. https://doi.org/10.1007/978-94-007-4816-3_7
66. Wolff, A., Gooch, D., Montaner, J. J. C., Rashid, U., & Kortuem, G. (2016). Creating an understanding of data literacy for a data-driven society. The Journal of Community Informatics, 12(3), 1-26. https://openjournals.uwaterloo.ca/index.php/JoCI/article/view/3275
67. Yılmaz, E., & Jafarova, G. (2022). Development of data driven decision making scale: a validity and reliability study. Research on Education and Psychology, 6(Special Issue), 69-91. https://dergipark.org.tr/en/download/article-file/2376933