TY  - JOUR
T1  - Eğitim Araştırmalarında Çok Düzeyli Meta-Analiz Modelleri: Örnek Uygulamalı Bir Rehber
TT  - Multilevel Meta-Analysis Models in Educational Research: A Practical Guide
AU  - Karaca, Dilek
AU  - Aydın, Burak
AU  - Atılgan, Hakan
PY  - 2024
DA  - September
Y2  - 2024
DO  - 10.53444/deubefd.1476011
JF  - Dokuz Eylül Üniversitesi Buca Eğitim Fakültesi Dergisi
JO  - DEU BEF Dergi
PB  - Dokuz Eylül Üniversitesi
WT  - DergiPark
SN  - 2602-2850
SP  - 2502
EP  - 2530
IS  - 61
LA  - tr
AB  - Meta-analiz belirli alanda yapılmış çalışmalardan sistematik şekilde elde edilen nicel verileri kullanarak o alanla ilgili genel durumu ortaya koymaya çalışan istatistiksel bir yöntemdir. Geleneksel meta-analiz yöntemleri, etki büyüklükleri arasında bağımlılık olmadığı varsayımına dayanmasına rağmen özellikle sosyal bilimlerde etki büyüklüğü bağımlılığına neden olabilecek çok sayıda durum söz konusudur. Araştırmacının elde ettiği etki büyüklüklerindeki bağımlılık veri setinde kümeli bir yapı oluşturur. Geleneksel meta-analiz uygulamalarındaki etki büyüklüğü bağımlılığı sorunu ile başa çıkmak ve kümeli veri yapısını dikkate almak için önerilen yöntemlerden biri çok düzeyli meta-analitik modellerin kullanılmasıdır. Çok düzeyli modeller diğer istatistiksel çerçevelerle birleştirilebilir ve kümeli veri yapılarının daha savunulabilir şekilde çözümlenebilmesini sağlayabilir. Bu sebeple çok düzeyli modellerin sosyal bilimlerde kullanım sıklığı her geçen gün artmaktadır.  Bu çalışmanın amacı örnek bir veri seti üzerinden çok düzeyli meta-analitik modellerin nasıl uygulanabileceğini göstermektir. R yazılımı kullanılarak gerçekleştirilen analizlerde metafor paketinin rma.mv fonksiyonu kullanılmıştır. Bu uygulama okuyuculara veri dosyasının düzenlenmesi, R yazılımının hazırlanması, genel etkinin hesaplanması, çalışma içi ve çalışmalar arası varyans heterojenliğinin incelenmesi ile kategorik ve sürekli değişkenlere ait moderatör analizlerinin nasıl yapılacağını adım adım anlatan bir kılavuz niteliği taşımaktadır. Çalışmada kullanılan veri dosyası ve R betiği okuyucuların kullanımı için ekler kısmında sunulmuştur.
KW  - Meta-analiz
KW  - çok düzeyli meta-analiz
KW  - çok düzeyli modeller
KW  - metafor
KW  - R yazılımı
N2  - Meta-analysis is a statistical method that tries to reveal the general situation about that field by using quantitative data obtained systematically from studies conducted in a certain field. However, traditional meta-analysis methods assume effect size independence, which can be a significant and misleading limitation in social sciences research. The dependence in the effect sizes obtained by the researcher creates a clustered data structure, a problem that needs attention. One of the methods to address this problem is the use of multilevel meta-analytic models. Multilevel models can be combined with other statistical frameworks, providing a more defensible analysis of clustered data structures. For this reason, the frequency of multilevel models in social sciences is steadily increasing. This study demonstrates how multilevel meta-analytic models can be applied to a sample data set. In the R environment, the rma.mv function of the metafor package was utilized for the analyses. This application is a step-by-step guide for readers on organizing the data file, preparing the R software, calculating the overall effect, examining the heterogeneity of variance within and between studies, and conducting moderator analyses with categorical and continuous variables. The data file and R script used in the study are shared as supplementary material for the readers.
CR  - APA. (2008). Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist, 63(9), 839-851. https://doi.org/10.1037/0003-066X.63.9.839
CR  - Assink, M., & Wibbelink, C. J. M. (2016). Fitting three-level meta-analytic models in R: A step-by-step tutorial. The Quantitative Methods for Psychology, 12(3), 154-174. https://doi.org/10.20982/tqmp.12.3.p154
CR  - Becker, B. J. (2000). Multivariate meta-analysis. Handbook of applied multivariate statistics and mathematical modeling, 499-525. https://doi.org/10.1016/B978-012691360-6/50018-5
CR  - Begg, C. B., & Mazumdar, M. (1994). Operating Characteristics of a Rank Correlation Test for Publication Bias. Biometrics, 50(4), 1088–1101. https://doi.org/10.2307/2533446
CR  - Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley & Sons. http://doi.org/10.1002/9780470743386
CR  - Cheung, M. W. L. (2014). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods, 19, 211-229. https://doi.org/10.1037/a0032968
CR  - Cheung, M. W. L. (2015). Meta-Analysis: A Structural Equation Modeling Approach. John Wiley & Sons.  https://doi.org/10.1080/10705510802561295
CR  - Cheung, M. W. L. (2019). A Guide to Conducting a Meta-Analysis with Non-Independent Effect Sizes. Neuropsychology Review, 29(4), 387-396. https://doi.org/10.1007/s11065-019-09415-6
CR  - Cheung, S. F., & Chan, D. K. S. (2004). Dependent Effect Sizes in Meta-Analysis: Incorporating the Degree of Interdependence. Journal of Applied Psychology, 89(5), 780-791. https://doi.org/10.1037/0021-9010.89.5.780
CR  - Cheung, S. F., & Chan, D. K. S. (2014). Meta-analyzing dependent correlations: An SPSS macro and an R script. Behavior Research Methods, 46(2), 331-345. https://doi.org/10.3758/s13428-013-0386-2
CR  - Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2. edition). Lawrence Erlbaum Associates, Publishers. https://doi.org/10.4324/9780203771587
CR  - Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2019). The handbook of research synthesis and meta-analysis (3.edition). Russell Sage Foundation.
CR  - Duval, S., & Tweedie, R. (2000). Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455-463. https://doi.org/10.1111/j.0006-341X.2000.00455.x
CR  - Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. bmj, 315(7109), 629-634. https://doi.org/10.1136/bmj.315.7109.629
CR  - Fisher, R. A. (1933). Statistical Methods for Research Workers. Nature, 131(3307), Article 3307. https://doi.org/10.1038/131383b0
CR  - Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational researcher, 5(10), 3-8. https://doi.org/10.3102/0013189X005010003
CR  - Gooty, J., Banks, G. C., Loignon, A. C., Tonidandel, S., & Williams, C. E. (2021). Meta-Analyses as a Multi-Level Model. Organizational Research Methods, 24(2), 389-411. https://doi.org/10.1177/1094428119857471
CR  - Harrer, M., Cuijpers, P., Furukawa, T. A., & Ebert, D. D. (2021). Doing Meta-Analysis with R: A Hands-On Guide (1.st edition). Chapman and Hall/CRC. https://doi.org/10.1201/9781003107347
CR  - Hedges, L. V. (1982). Fitting Categorical Models to Effect Sizes from a Series of Experiments. Journal of Educational Statistics, 7(2), 119-137. https://doi.org/10.3102/10769986007002119
CR  - Hedges, L. V., & Olkin, I. (1988). Statistical Methods for Meta-Analysis. Journal of Educational Statistics, 13(1), 75. https://doi.org/10.2307/1164953
CR  - Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6(3), 203-217. https://doi.org/10.1037/1082-989X.6.3.203
CR  - Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine, 21(11), 1539-1558. https://doi.org/10.1002/sim.1186
CR  - Hox, J. J., & De Leeuw, E. D. (2003). Multilevel models for meta-analysis. In Multilevel modeling (pp. 87-104). Psychology Press.
CR  - Hox, J. J., Moerbeek, M., & Schoot, R. van de. (2010). Multilevel Analysis: Techniques and Applications, Second Edition. Routledge. https://doi.org/10.4324/9780203852279
CR  - Hunter, J. E., & Schmidt, F. L. (2004). Methods of Meta-Analysis: Correcting Error and Bias in Research Findings, Second Edition. SAGE.
CR  - Konstantopoulos, S. (2011). Fixed effects and variance components estimation in three-level meta-analysis. Research Synthesis Methods, 2(1), 61-76. https://doi.org/10.1002/jrsm.35
CR  - Lipsey, M. W., & Wilson, D. B. (2001). Practical Meta-Analysis. SAGE Publications, Inc.
CR  - Marín-Martínez, F., & Sánchez-Meca, J. (1999). Averaging Dependent Effect Sizes in Meta-Analysis: A Cautionary Note about Procedures. The Spanish Journal of Psychology, 2, 32-38. https://doi.org/10.1017/S1138741600005436
CR  - Marsh, H. W., Bornmann, L., Mutz, R., Daniel, H.-D., & O’Mara, A. (2009). Gender Effects in the Peer Reviews of Grant Proposals: A Comprehensive Meta-Analysis Comparing Traditional and Multilevel Approaches. Review of Educational Research, 79(3), 1290-1326. https://doi.org/10.3102/0034654309334143
CR  - Midway, S. (2022). Data analysis in R. Retrieved April, 8, 2023. https://bookdown.org/steve_midway/DAR/
CR  - Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. The Lancet, 354(9193), 1896-1900. https://doi.org/10.1016/S0140-6736(99)04149-5
CR  - Morris, S. B. (2008). Estimating Effect Sizes From Pretest-Posttest-Control Group Designs. Organizational Research Methods, 11(2), 364-386. https://doi.org/10.1177/1094428106291059
CR  - Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., ... & Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.  Journal of clinical epidemiology, 134, 103-112. https://doi.org/10.1136/bmj.n71
CR  - Pastor, D. A., & Lazowski, R. A. (2018). On the Multilevel Nature of Meta-Analysis: A Tutorial, Comparison of Software Programs, and Discussion of Analytic Choices. Multivariate Behavioral Research, 53(1), 74-89. https://doi.org/10.1080/00273171.2017.1365684
CR  - Pearson. (1904). Report On Certain Enteric Fever Inoculation Statistics. 1243-1246.
CR  - Pustejovsky, J. (2023). clubSandwich: Cluster-Robust (Sandwich) Variance Estimators with Small-Sample Corrections (R package version 0.5.10). https://cran.r-project.org/web/packages/clubSandwich/index.html
CR  - Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological bulletin, 86(3), 638. https://doi.org/10.1037/0033-2909.86.3.638
CR  - Rosenthal, R. (1991). Meta-Analytic Procedures for Social Research. SAGE Publications, Inc. https://doi.org/10.4135/9781412984997
CR  - Rosenthal, R., & Rubin, D. B. (1986). Meta-analytic procedures for combining studies with multiple effect sizes. Psychological Bulletin, 99(3), 400-406. https://doi.org/10.1037/0033-2909.99.3.400
CR  - RStudio Team (2020). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA URL http://www.rstudio.com/
CR  - Scammacca, N., Roberts, G., & Stuebing, K. K. (2014). Meta-Analysis With Complex Research Designs: Dealing With Dependence From Multiple Measures and Multiple Group Comparisons. Review of Educational Research, 84(3), 328-364. https://doi.org/10.3102/0034654313500826
CR  - Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., Moher, D., Becker, B. J., Sipe, T. A., & Thacker, S. B. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. Jama, 283(15), 2008-2012. https://doi.org/10.1001/jama.283.15.2008
CR  - Şen, S., & Akbaş, N. (2016). Çok Düzeyli Meta-Analiz Yöntemleri Üzerine Bir Çalışma. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 7(1). https://doi.org/10.21031/epod.29995
CR  - Şen, S., & Yıldırım, İ. (2020). CMA ile meta-analiz uygulamaları. Ankara: ANI Yayıncılık.
CR  - Van Den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45(2), 576-594. https://doi.org/10.3758/s13428-012-0261-6
CR  - Viechtbauer, W. (2010). Conducting Meta-Analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1-48. https://doi.org/10.18637/jss.v036.i03
UR  - https://doi.org/10.53444/deubefd.1476011
L1  - https://dergipark.org.tr/tr/download/article-file/3895609
ER  -