Development of Data Driven Decision Making Scale: A Validity and Reliability Study

Abstract

In this study, it was aimed at developing a valid and reliable evaluation tool with the purpose of evaluating the Data Driven Decision Making Skills of teachers who work in primary school, middle-school and high-school levels. 534 teachers were included in the study (256 for EFA and 278 for CFA) (63 % female and 37 % male). For the scale development process, 730 teachers constituted the whole study group. In order to determine the structural validity of the scale, exploratory factor analysis and confirmatory factor analysis were used. As a result of the exploratory factor analysis, it was determined that the scale consisted of 10 items and 2 sub-dimensions. In the light of the literature, these dimensions were titled “Data literacy” and “Decision making”. The 2 sub-dimensional structure of the scale was subjected to the confirmatory factor analysis and as a result of the CFA, 1 item was excluded from the scale. The 2 sub-dimensional model created as a result of the EFA of DDDMS was tested with CFA and the adaptive values are at an acceptable level. In addition, the t values related to the high and low group difference of the scale showed that DDDMS is able to assess the structure in a distinctive manner. In order to determine the reliability of the scale, the Cronbach alpha internal consistency coefficients were calculated. When the reliability analyses results were viewed in the light of Data Driven Decision Making Scale’s factors, 0,782 value was obtained for the “Data Literacy” sub-dimension and 0,672 value was obtained for the “Decision Making” sub-dimension. The inner consistency coefficient of DDDMS is 0,790. As a result of the findings, it was determined that Data Driven Decision Making Scale is a valid and reliable assessment tool to evaluate the DDDM skills of teachers.

Keywords

• Data • Data driven decision making • Data literacy
• Data
• Data driven decision making

References

1. Acuna, E., & Rodriguez, C. (2004). A meta analysis study of outlier detection methods in classification. Technical paper, Department of Mathematics, University of Puerto Rico at Mayaguez, 1-25.
2. Alexander, C.S. & Becker, H. J. (1978). The use of vignettes in survey research. Public Opinion Quarterly, 42(1), 93–. https://doi:10.1086/268432
3. Alpar, R. (2013). Çok Değişkenli İstatistiksel Yöntemler (Multivariate Statistical Methods). Ankara: Detay Yayıncılık.
4. Altun, N. & Karasu, N. (2021). Risk grubu öğrenciler için gönderme öncesi süreçte veriye dayalı karar verme. [Data-Driven Decision Making in the Pre-Referral Process for Risk Group Students] TEBD, 19(1), 593-612. https://doi.org/10.37217/tebd.906636
5. Anderson, C. (2015). Creating a Data-Driven Organization. USA: O’Reilly Media.
6. Anderson, D.L. (2015). Teacher perceptions of data driven decision making for school improvement . (Unpublished doctoral dissertation). Marian University. USA: Fond du Lac, Wisconsin.
7. Arici, İ. (2007). İlköğretim din kültürü ve ahlak bilgisi dersinde öğrenci başarısını etkileyen faktörler (Ankara örneği). (Doktora tezi). [The Effective factors on the success of the students in the religious culture and ethics course (Ankara example)]. (Unpublished doctoral dissertation). Ankara University Institute of Social Sciences, Ankara.
8. Bahar, H.H. (2019). Temel kavramlar. Kağan, M.& Yalçın, S.(Ed.), Eğitime Giriş içinde, (s.1-36). [Basic consepts]. Kagan, M & Yalcin, S. (Ed.), in İntroduction to Education, (p.1-36 ). Ankara: Pegem Academy.

9. Bernhardt, V.L. (2001). Intersections. Journal of Staff Development, 21(1), ss. 33-36.
10. Bouchard, H. (2018). Using data-driven decision-making to drive ınstructional decisions with high school mathematics teachers (Unpublished doctoral dissertation). Concordia University. USA:Portland.
11. Bryman, A. & Cramer, D. (2001). Quantitative data analysis with SPSS release 10 for Windows. London: Routledge.
12. Buyukozturk, S. (2015). Deneysel desenler: Öntest-sontest, kontrol grubu, desen ve veri analizi. [Experimental designs: Pretest-posttest, control group, design and data analysis]. Ankara: Pegem A.
13. Calık, T. & Arslan, M.M. (2019). Eğitime Giriş. [İntroduction to Education] Ankara: Pegem Academy.
14. Cambridge Dictionary (2022). https://dictionary.cambridge.org/dictionary/english/data
15. Can, A. (2018). SPSS ile bilimsel araştırma sürecinde nicel veri analizi [Quantitative data analysis in scientific research process with SPSS. ]. Ankara: Pegem Yayıncılık.
16. Cemaloglu, N. (2019). Veriye dayalı yönetim [Data driven management]. Ankara: Pegem Academy.
17. Comrey, A. L & Lee, H. L. (1992). A first course in factor analysis. New Jersey: Erlbaum, Hillsdale.
18. Considine, G. & Zappala, G. (2002). The influence of social and economic disadvantage in the academic performance of school students in Australia. Journal of Sociology, 38(29), 129-148.
19. Corey, M.M. (2016). School leadership and data-driven ınstruction: a small city school case study (Unpublished doctoral dissertation). University at Buffalo. USA: New York.
20. Corrigan, M. W., Grove, D., & Vincent, P. F. (2011). Multi-dimensional education: A common sense approach to data-driven thinking. Thousand Oaks, CA: Corwin Publishing.
21. Cubuk, A. (2019). Ortaokul öğrencilerinin internet bağımlılığı, fiziksel aktivite düzeyleri ve akademik başarı arasında ilişkinin incelenmesi. (Yüksek lisans tezi). [Internet addiction, physical activity level and academic achievement status of middle school students (master thesis)], Marmara University Institute of Educational Sciences, Istanbul.
22. Demir, K. (2009). İlköğretim okullarında verilere dayalı karar verme. [Data-Driven Decision-Making in Primary School.] Educational Administration: Theory and Practice 15 (59), 367-398. Retrieved from https://dergipark.org.tr/en/pub/kuey/issue/10338/126675
23. Dilekci, U., Nartgun, S. & Nartgun, Z. (2020). Okullarda Veriye Dayalı Yönetim [Data-Driven Management in Schools]. International Pegem Conference on Education (IPCEDU-2020) . pp. 232-243.
24. Dogan, E. (2021). Okul yönetiminde veriye dayalı karar verme sürecinin yönetici görüşlerine göre değerlendirilmesi. [Evaluating the data-driven decision making process in school management according to the views of the administrators] (Doctoral dissertation) Gazi Unıversity Graduate School Of Educatıonal Sciences. Ankara.
25. Durmaz, M., Huseyinli, T. & Güclü, C. (2016). Zaman yönetimi becerileri ile akademik başarı arasındaki ilişki. [The relation between time management and academic success]. Journal of Humanities and Social Sciences Research,, 5(7). pp. 2291-2303.
26. Ediger, M. (2010). Data Based Instruction in Reading. Reading Improvement, 47(4), 175-178.
27. Erdoğdu M. Y. & Kenarlı Ö (2008). Duygusal zeka ile akademik başarı arasındaki ilişki. [Relations between emotional quotient with academic success.] Milli Eğitim, 37(178), 297 - 310.
28. Eskiocak, S.(2005). Sınıf öğretmenlerinin öğretimi planlama aşamasında karar verme sürecine etki eden etmenlerin analizi (Yüksek Lisans Tezi). [The analyzes of the factors which affected the primary school teachers’ making decision process in their plannıing for education. (Master Thesis)] Cukurova University Department of Educational Sciences, Adana.
29. Gujjar, A.A. & Naoreen, B (2009). Role of teacher as classroom manager. İ-manager’s Journal on Educational Psychology, 2(4), 65-73. Erişim adresi: https://files.eric.ed.gov/fulltext/EJ1097655.pdf
30. Gunduz, S., Atas, H. & Elkovan, C.G. (2020). Çok kriterli karar verme [Multi-criteria decision making]. Ankara: Gazi Kitabevi.
31. Guris, S., & Astar, M. (2015). Bilimsel araştırmalarda SPSS ile istatistik [Statistics with SPSS in scientific research]. Istanbul: Der Publications.
32. Halverson, R., Grigg, J., Prichett, R. & Thomas, C. (2006). The new instruction leadership: Creating data-driven instructional systems in schools. Journal of School Leadership 25(3). ss.3-58. https://doi:10.1177/105268461502500305.
33. Harris, N. (2011). The Impact of professional development in data based decision making on the teaching practices of educators (Unpublished doctoral dissertation). Walden University. USA: Minnesota.
34. Hughes, R. & Huby, M. (2002). The application of vignettes in social and nursing research. Methodological İssues in Nursing Research, 37(4), 382–386. https://doi:10.1046/j.1365-2648.2002.02100.x
35. Ikemoto, G. S., & Marsh, J. A. (2007). Cutting through the "data-driven" mantra: Different conceptions of data-driven decision making. In the Yearbook of the National Society for the Study of Education (Vol. 106, pp. 105-131). https://doi: 10.1111/j.1744-7984.2007.00099.x
36. Jr.S.C. (2016). An investigation of primary and secondary teachers’ beliefs in the usefulness of data-drıven decision-making (Unpublished doctoral dissertation). Pace University. USA: New York.
37. Karabacak, M.E. (2019). Data driven decision making in public administration: an assessment based on business intelligence maturity model. (Master Thesis). Ankara Yildirim Beyazit University The Institute Of Social Sciences. Ankara.
38. Kline, R. B., (2005). Principles and practice of structural equation modeling. Guilford Press, New York.
39. Kaufman, T., Graham, C. R., Picciano, A. G., Wiley, D., & Popham, J. A. (2014). Data-driven decision making in the k12 classroom. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 337–346). New York, NY: Springer.
40. Kline, R. B. (2011). Principles and practice of structural equation modeling. Guilford Publications.
41. Luo, M.N. (2005). AN empirical study of high school principals’ data-driven decision-making practices and their relationships to contextual variables (Unpublished doctoral dissertation). University of Nebraska. USA: Omaha.
42. Mandinach, E.B. (2012). A Perfect time for data use: using data-driven decision making to inform practice. Educational Psychologist, 47(2), ss.71-85. http://dx.doi.org/10.1080/00461520.2012.667064
43. Markarian, B.S.(2009). Holding on and holding out: why some teachers resist the move toward data-driven decision making (Unpublished doctoral dissertation). University Of Southern California. USA: California
44. Marsh, J.A.,, Pane, J.F. & Hamilton, L.S. (2006). Making sense of data-driven decision making in education. https://www.rand.org/content/dam/rand/pubs/occasional_papers/2006/RAND_OP170.pdf
45. Mazlumoglu, M. (2019). Sınıf öğretmenlerinin karar verme becerileri ile öz düzenleme becerileri arasındaki ilişkinin incelenmesi (Yüksek lisans tezi). [Investigation of the relationship between decision making skills and self-regulation skills of classroom teachers (Master Thesis)] Ataturk University İnstitute Of Educational Sciences , Erzurum.
46. McLeod, S., & Seashore, K. (2006, Spring). Data driven decision-making readiness survey: Teachers. Minneapolis, MN: University of Minnesota.
47. Mertler, C.A.(2014). The Data-Driven Classroom: How do I use student data to improve my instruction? (ASCD Arias). USA: ASCD.
48. Morgan, G. A., Leech, N. L. Gloeckner, G.W. ve Barrett, K. C. (2004). SPSS for introductory statistics: Use and interpretation. Psychology Press.
49. Meydan, C. H. & Sesen, H. (2011). Structural equation modeling AMOS applications. Detay Yayıncılık.
50. Moriarty, T.W. (2013). Data-driven decision making: teachers’ use of data in the classroom (Unpublished doctoral dissertation). University of San Diego. USA: San Diego.
51. Murrell, M.A. (2012). Principals can improve student achievement with Data driven decision making (Unpublished doctoral dissertation). University of Houston. USA: Houston.
52. Olufemioladebinu, T., Adediran, A.A., Oyediran, W.O. (2018). Factors ınfluencing the academic achievement of students’ in colleges of education in Southwest, Nigeria. Journal of Education and Human Development, 7(3). 109-115. https://doi: 10.15640/jehd.v7n3a12
53. Ozkan, M.& Arslantas.(2013). Etkili öğretmen özellikleri üzerine sıralama yöntemiyle bir ölçekleme çalışması. [A study of scalıng wıth rankıng judgment method on characterıstıc of effectıve teacher]. Trakya University Journal of Social Sciences , 15(1), 311-330.
54. Perger, M. ve Takacs, İ. (2016). Factors contributing to students’ academic success based on the students’ opinion at bme faculty of economic and social sciences. Periodica Polytechnica Social and Management Sciences, 24(2). 119-135.
55. Sahin Kolemen, C. & Erisen, Y. (2017). Mesleki ve Teknik Ortaöğretim öğrencilerinin problem çözme ve eleştirel düşünme becerileri ile akademik başarıları arasındaki ilişkinin incelenmesi. [An İnvestigation on the relationship between problem solving and critical thinking skill, and academic achievement of vocational and technical high school students]. Turkey Education Journal, 2(1), 42-60.
56. Sahin, A. (2011). Öğretmen algılarına göre etkili öğretmen davranışları. [Effective teacher’s attitudes according to teacher’s perceptions]. KEFAD, 12(1), 239-259.
57. Schwartz, W. (2002). Data-driven equity in urban schools. Eric Clearinghouse on Urban Education, 177, 1-7. Seker, H. & Gencdogan, B. (2014). Psikolojide ve eğitimde ölçme aracı geliştirme. (2. Basım). [Developing a measurement tool in psychology and education. (2nd Edition)]. Nobel Yayınevi
58. Sencan, H. (2005). Sosyal ve Davranışsal Ölçümlerde Güvenirlik ve Geçerlik. [Reliability and Validity in Social and Behavioral Measurements. Ankara: Seçkin Publishing.] Ankara: Seçkin Yayıncılık.
59. Seyis, S. (2011). Ortaöğretim öğrencilerinin motivasyonları ve duygusal zekaları ile akademik başarıları arasındaki ilişki (Yüksek Lisans Tezi). [Relationship between high school students' motivation, emotional intelligence and academic achievement) (Master Thesis).] Karadeniz Technical University, Institute of Educational Sciences, Trabzon.
60. Simpson, G.H. (2011). School leaders’ use of data-driven decision-making for school improvement: a study of promising practices in two California charter schools (Unpublished doctoral dissertation). University Of Southern California. USA: California
61. Starks, C.V. (2014). Data driven decision-making in midwest charter schools: teachers’ experiences in the classroom. (Unpublished doctoral dissertation). University of Missouri Kansas City. USA: Missouri.
62. Tabachnick, B. G. & Fidell, L. S. (2007). Using multivariate statistics. Boston: Allyn and Bacon.
63. Tabak, H. , Sahin, F. & Yavuz Tabak, B. (2020). Okul yöneticilerinin karar alma yaklaşımları: veriye dayalı karar almaya geçiş [School administrators decision making approaches: transitıon to data-based decision making]. The Journal of Turkish Social Research, 24 (3), pp. 713-725. Retrieved from https://dergipark.org.tr/en/pub/tsadergisi/issue/58254/727254
64. Tavsancıl, E. (2005). Tutumların ölçülmesi ve spss ile veri analizi [Measuring attitudes and data analysis with spss. Nobel Publishing]. Nobel Yayıncılık.
65. Teigen, B.N. (2009). A Systematic Examination of Data-Driven Decision-making within a School Division: The Relationships among Principal Beliefs, School Characteristics, and Accreditation Status (Unpublished doctoral dissertation). Virginia Commonwealth University. USA: Richmond.
66. Tezbasaran, A. A. (1996). Likert tipi ölçek geliştirme kılavuzu. (Likert type scale development guide). Ankara: Publications of the Association of Psychologists.
67. Thompson, B. (2008). Foundations of behavioral statistics: An insight-based approach. New York, NY: Guilford Press
68. Turan, H. (2019). Data-Driven decision making. N.Cemaloglu in. Data-Driven Management (p.2-17). Ankara: Pegem Academy.
69. Wagaman, R.M. (2015). Understandıng the prıvate school: how do prıvate school teachers understand and descrıbe data-drıven decısıon makıng? (Unpublished doctoral dissertation). Capella University. USA: Minnesota.
70. White, V.C. (2008). Relationships among principals’ beliefs about data-driven decision making, principal and school characteristics, and student achievement in elementary schools (Unpublished doctoral dissertation). University Of Florida. USA: Florida.
71. Yao, J.M. (2009). Examining the applications of data-driven decision making on classroom instruction (Unpublished doctoral dissertation). University Of Southern California. USA: California
72. Yildiz, B. (2016). İlkokul öğrencilerinin akademik başarılarının arttırılmasında öğretmen, okul yönetimi ve öğrenci veli görüşlerinin incelenmesi (Yüksek lisans tezi). [The review of teacher, school management and students' parents to improve the success of elementary students' academic successes (Master thesis).] Mersin University Institute of Educational Sciences. Mersin.
73. Yilmaz, E., Sahin, Y.L & Akbulut, Y. (2015). Dijital veri güvenliği farkındalığı ölçeğinin geliştirilmesi. [Development of the digital data security awareness scale] Online Academic Journal of Information Technology, 6(21). 23-40. https://doi: 10.5824/1309-1581.2015.4.002.x