Sosyal Bilimlerde ve Eğitim Bilimlerinde Sistematik Derleme, Meta Değerlendirme ve Bibliyometrik Analizler

Öz

Sistematik derlemeler, bir alanda benzer yöntemler ile yapılmış olan çalışmaların kapsamlı ve detaylı bir biçimde taranması; derlemeye girecek çalışmaların çeşitli seçme ölçütleri kullanılarak belirlenmesi, belirlenen çalışmaların yapılandırılmış ve kapsamlı bir kalite değerlendirmesinin ve sentezinin yapılması esasına dayanan araştırma yaklaşımıdır. Bu anlamda özellikle son yıllarda sosyal bilimlerin ve eğitim bilimlerinin çeşitli disiplinlerinde sistematik derlemeler sıklıkla yapılmaktadır. Bu kapsamda çeşitli içerik analizleri, meta değerlendirmeler, meta sentezler, bibliyometrik analizler, bilim haritalama çalışmaları, atıf analizleri, bibliyografik eşleştirmeler, sosyal ağ analizleri, ortak kelime analizleri vb. farklı çalışmalar yapılmaktadır. Bu çalışmada sistematik derleme, meta değerlendirme ve bibliyometrik çalışmaların tanıtılması ve sosyal bilimlerde ve eğitim bilimlerinde kullanımı hakkında bilgi verilmesi amaçlanmıştır. Bu kapsamda “İçerik Analizi, Geleneksel Derleme, Sistematik Derleme, Meta Sentez, Meta Değerlendirme, Meta Çalışma, Bibliyometrik Analiz, Bilim Haritalama, Atıf Analizi, Ortak Atıf Analizi, Bibliyometrik Eşleştirme, Ortak Yazar Analizi, Sosyal Ağ Analizi, Ortak Kelime Analizi” kavramları tanıtılmaya çalışılmıştır.

Anahtar Kelimeler

• Sistematik derleme
• Bibliyometri
• Bilim haritalama
• Meta sentez
• Meta değerlendirme

Systematic Review, Meta Evaluation, and Bibliometric Analysis in Social Sciences and Educational Sciences

Öz

The purpose of this study is to introduce systematic reviews, meta-evaluation, and bibliometric studies and give information about their use in social sciences and educational sciences. In this context, “Traditional Review, Systematic Review, Meta-Synthesis, Meta-Evaluation, Meta-Studies, Bibliometric Analysis, Science Mapping, Citation Analysis, Co-Citation Analysis, Bibliographic Coupling, Co-Author Analysis, Social Network Analysis, Co-Word Analysis” concepts were investigated. Systematic reviews are a research approach based on a comprehensive and detailed screening of studies conducted with similar methods in a given field, identifying studies to be included in the review using different selection criteria, and conducting a structured and comprehensive assessment of the quality and synthesis of specific studies. In this sense, systematic reviews are frequently made in various disciplines of social sciences and educational sciences, especially in recent years. In this context, various content analysis studies, meta-evaluations, meta-synthesis, bibliometric analysis, science mapping studies, citation analysis, bibliographic coupling, social network analysis, co-word analysis, etc. different studies are carried out.

Anahtar Kelimeler

• Systematic review
• Meta-synthesis
• Meta-evaluation
• Bibliometri
• Science mapping

Kaynakça

1. Acedo, F. J., Barroso, C., Casanueva, C. ve Galan, J. L. (2006). Co-authorship in management and organizational studies: An empirical and network analysis. Journal of Management Studies, 43(5), 957-983.
2. Akarçay-Ulutaş, D. ve Boz, A. N. (2019). Sistematik derleme ve meta-analiz. Sosyal bilimlerde araştırma yöntemleri: nicel, nitel ve karma tasarımlar için bir rehber (Edt: Ş. Aslan). Konya: Eğitim Yayınevi. ss. 455-468.
3. Al, U. (2008). Türkiye’nin bilimsel yayın politikası: Atıf dizinlerine dayalı bibliyometrik bir yaklaşım (Doktora Tezi). Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
4. Al, U. ve Coştur, R. (2007). Türk Psikoloji Dergisi’nin bibliyometrik profili. Türk Kütüphaneciliği, 21(2), 142-163.
5. Al, U. ve Tonta, Y. (2004). Atıf analizi: Hacettepe Üniversitesi Kütüphanecilik Bölümü tezlerinde atıf yapılan kaynaklar. Bilgi Dünyası, 5(1), 19-47.
6. Almind, T. C. ve Ingwersen, P. (1997). Informetric analysis on the World Wide Web: Methodological approaches to ‘webometrics’. Journal of Documentation, 53(4), 404-426.
7. Amir-Behghadami, M. ve Janati, A. (2020). Population, intervention, comparison, outcomes and study (PICOS) design as a framework to formulate eligibility criteria in systematic reviews. Emergency Medicine Journal, 37(6), 387.
8. Arık, R. S. ve Türkmen, M. (2009). Eğitim bilimleri alanında yayınlanan bilimsel dergilerde yer alan makalelerin incelenmesi. I. Uluslararası Türkiye Eğitim Araştırmaları Kongre Kitabı. http://www.eab.org.tr/eab/oc/egtconf/pdfkitap/pdf/488.pdf. adresinden 10 Ocak 2012 tarihinde edinilmiştir.

9. Aspfors, J. ve Fransson, G. (2015). Research on mentor education for mentors of newly qualified teachers: A qualitative meta-synthesis. Teaching and Teacher Education, 48, 75-86.
10. Ata, B. ve Urman, B. (2008). Sistematik derlemelerin kritik analizi. Journal of Turkish Society of Obstetric and Gynecology, 5(4), 233-240.
11. Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36, 258-267.
12. Badavan, Y. (1985). Üniversitelerde eğitim yönetimi ve denetimi alanlarında yapılmış tez çalışmalar. Yayınlanmamış Yüksek Lisans Tezi. Hacettepe Üniversitesi Sosyal Bilimler Enstitüsü. Ankara.
13. Bağ, H. ve Çalık, M. (2017). İlköğretim düzeyinde yapılan argümantasyon çalışmalarına yönelik tematik içerik analizi. Eğitim ve Bilim, 42(190), 281-303.
14. Bair, C. R (1999). Meta-synthesis. Annual Meeting of the Association for the Study of Higher Education. San Antonio, TX, November 18-21, 1999. ERIC No: 437866.
15. Balcı, A. (1988). Eğitim yönetimi araştırmalarının durumu: EAQ’de 1970-1985 arasında yayınlanan araştırmalar. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 21(1), 421-434.
16. Barnett-Page, E. ve Thomas, J. (2009). Methods for the synthesis of qualitative research: A critical review. BMC Medical Research Methodology, 9(59). https://doi.org/10.1186/1471-2288-9-59.
17. Barroso, J., Gollop, C., Sandelowski, M., Meynell, J., Pearce, P. ve Collins, L. (2003). The challenge of searching for and retrieving qualitative studies. Western Journal of Nursing Research, 25(2), 153-178.
18. Bellibaş, M. Ş. (2018). Sistematik derleme çalışmalarında betimsel içerik analizi. Eğitim Yönetiminde Araştırma (Edt: K. Beycioğlu, N. Özer, Y. Kondakçı). Ankara: Pegem Akademi. ss. 511-532.
19. Bellibaş, M. Ş. ve Gümüş, S. (2018). Eğitim yönetiminde sistematik derleme çalışmaları. Eğitim Yönetiminde Araştırma (Edt: K. Beycioğlu, N. Özer, Y. Kondakçı). Ankara: Pegem Akademi. ss. 507-508.
20. Bettany-Saltikov, J. (2010a). Learning how to undertake a systematic review: Part 1. Nursing Standard, 24(50), 47-55.
21. Bettany-Saltikov, J. (2010b). Learning how to undertake a systematic review: Part 2. Nursing Standard, 24(51), 47-56.
22. Bettany-Saltikov, J. (2012). How to do a systematic literature review in nursing: A step-by-step guide. New York: Open University Press.
23. Björneborn, L. (2004). Small-world link structures across an academic web space: A library and information science approach. Unpublished PhD Thesis. Royal School of Library and Information Science, Copenhagen.
24. Bobin, K. (2017). Meta-evaluation: A synthesis of evaluation studies 2005-2016. Commonwealth Secretariat. https://thecommonwealth.org/.
25. Booth, A., Papaioannou, D. ve Sutton, A. (2016). Systematic approaches to a successful literature review (2nd Edition). Thousand Oaks, CA, Sage.
26. Borgatti, S. P. ve Ofem, B. (2010). Overview: Social network theory and analysis. Social network theory and educational change (Edt: A. Daly). Cambridge: Harvard Education. pp. 17-30.
27. Campbell Collaboration (2020). Campbell Collaboration Systematic Reviews of Social Science. http://www.campbellcollaboration.org
28. Campbell, R., Pound, P., Morgan, M., Daker-White, G., Britten, N., Pill, R., Yardley, L., Pope, C. ve Donovan, J. (2011). Evaluating meta-ethnography: Systematic analysis and synthesis of qualitative research. Health Technology Assessment, 15(43), 1-164.
29. Campbell, R., Pound, P., Pope, C., Britten, N., Pill, R., Morgan, M. ve Donovan, J. (2003). Evaluating meta-ethnography: A synthesis of qualitative research on lay experiences of diabetes and diabetes care. Social Science and Medicine, 56(4), 671-684.
30. CASP (2018). Critical Appraisal Skills Programme (insert name of checklist i.e. Systematic Review) Checklist. [online] Available at: URL. Accessed: Date Accessed. https://casp-uk.net/wp-content/uploads/2018/01/CASP-Systematic-Review-Checklist_2018.pdf
31. Castillo, F. A. ve Hallinger, P. (2017). Systematic review of research on educational leadership and management in Latin America, 1991-2017. Educational Management Administration & Leadership, 46(2), 207-225.
32. Chalmers, I., Hedges, L. V. ve Cooper, H. (2002). A brief history of research synthesis. Evaluation and the Health Professions, 25, 12-37.
33. Chen, C. (1999). Visualising semantic spaces and author co-citation networks in digital libraries. Information Processing & Management, 35(2), 401-420.
34. Chen, C. (2017). Science mapping: A systematic review of the literature. Journal of Data and Information Science, 2(2), 1-40.
35. Chen, C., Dubin, R., ve Schultz, T. (2014). Science mapping. Encyclopedia of information science and technology (Edt: M. Khosrow-Pour) (3rd Edition). Hershey PA, USA: IGI Global. ss. 4171-4184.
36. Chen, C., Ibekwe, SanJuan, F. ve Hou, J. (2010). The structure and dynamics of cocitation clusters: A multiple‐perspective cocitation analysis. Journal of the American Society for Information Science and Technology, 61(7), 1386-1409.
37. Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E. ve Herrera, F. (2011). Science mapping software tools: Review, analysis, and cooperative study among tools. Journal of the American Society for Information Science and Technology, 62(7), 1382-1402.
38. Cochrane Collaboration (2020). Cochrane Database of Systematic Reviews. http://www.cochrane.org.
39. Cohen, L., Manion, L. ve Morrison, K. (2007). Research methods in education (5th Edition). London and New York: Routledge Falmer.
40. Cooksy, L. J. ve Caracelli, V. J. (2009). Metaevaluation in practice: Selection and application of criteria. Journal of MultiDisciplinary Evaluation, 6(11), 1-15.
41. Cooper, H., Hedges, L. V. ve Valentine, J. C. (2019). Research synthesis as a scientific process. In H. Cooper, L. V. Hedges ve J. C. Valentine (Edt.). The handbook of research synthesis and meta-analysis (3rd Edition). New York: Russell Sage Foundation. ss. 3-18.
42. Crismore, A. (1985). Landscapes: A state-of-the-art assessment of reading comprehension research 1974-1984. A study funded by the U. S. Department of Education. ERIC ED 261350.
43. Çalık, M. ve Sözbilir, M. (2014). İçerik analizinin parametreleri. Eğitim ve Bilim, 39(174), 33-38.
44. Çiftçi, Ş. K., Danışman, Ş., Yalçın, M., Tosuntaş, Ş. B., Ay, Y., Sölpük, N. ve Karadağ, E. (2016). Map of scientific publication in the field of educational sciences and teacher education in Turkey: A bibliometric study. Educational Sciences: Theory & Practice, 16, 1123-1145.
45. Çiltaş, A., Güler, G. ve Sözbilir, M. (2012). Mathematics education research in Turkey: A content analysis study. Educational Sciences: Theory & Practice, 12(1), 574-578.
46. DANIDA - Danish International Development Agency (2004). Meta-evaluation: Private and business sector development interventions. Denmark: Danida. http://www.oecd.org/derec/denmark/36478191.pdf.
47. Davies, P. (2000). The relevance of systematic reviews to educational policy and practice. Oxford Review of Education, 26(3-4), 365-378.
48. Denyer, D. ve Tranfield, D. (2006). Using qualitative research synthesis to build an actionable knowledge base. Management Decision, 44(2), 213-227.
49. DeWitt-Brinks, D. ve Rhodes, S. C. (1992). Listening instruction: A qualitative meta-analysis of twenty-four selected studies. Paper presented at The Annual Meeting of The International Communication Association. Miami, FL. May 20-25, 1992, ERIC Number: ED351721.
50. Dinçer, S. (2018). Eğitim bilimleri araştırmalarında içerik analizi: Meta-analiz, meta-sentez, betimsel içerik analizi. Bartın Üniversitesi Eğitim Fakültesi Dergisi, 7(1), 176-190.
51. Dixon-Woods, M., Bonas, S., Booth, A., Jones, D. R., Miller, T., Shaw, R. L., Smith, J., Sutton, A. ve Young, B. (2006). How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative Research, 6(1), 27-44.
52. Doğan, H. ve Tok, T. N. (2018). Türkiye’de eğitim bilimleri alanında yayınlanan makalelerin incelenmesi: Eğitim ve Bilim Dergisi örneği. Current Research in Education, 4(2), 94-109.
53. Efron, S. E. ve Ravid, R. (2019). Writing the literature review: A practical guide. New York: The Guilford Press.
54. Egger, M., Smith, G. D. ve O’Rourke, K. (2001). Rationale, potentials, and promise of systematic reviews. In M. Egger, G. D. Smith ve D. Altman (Edt.), Systematic reviews in health care: Meta-analysis in context (2nd Edition). London: BMJ Publishing Group. ss. 3-19.
55. EPPI-Centre (2019). What is a systematic review? London: University of London. https://eppi.ioe.ac.uk/cms/.
56. Erdem, D. (2011). Türkiye’de 2005-2006 yılları arasında yayımlanan eğitim bilimleri dergilerindeki makalelerin bazı özellikler açısından incelenmesi: Betimsel bir analiz. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 2(1), 140-147.
57. European Commission (2011). Mid-term meta evaluation of IPA assistance evaluation report. http://aei.pitt.edu/46931/1/IPA.assistance.midterm.meta.eval.pdf.
58. Evans, J. ve Benefield, P. (2001). Systematic reviews of educational research: Does the medical model fit? British Educational Research Journal, 27(5), 527-541.
59. Finfgeld, D. L. (2003). Metasynthesis: The state of the art-so far. Qualitative Health Research, 13(7), 893-904.
60. Fitzpatrick, J. L., Sanders, J. R. ve Worthen B. R. (2004). Program Evaluation: Alternative Approaches and Practical Guidelines (3rd Edition). New York: Pearson and AB.
61. Fraenkel, J. R., Wallen, N. E. ve Hyun, H. H. (2012). How to design and evaluate research in education (7th Edition). New York: McGraw-Hill.
62. France, E. F., Uny, I., Ring, N., Turley, R. L., Maxwell, M., Duncan, E. A. S., Jepson, R. G., Roberts, R. J. ve Noyes, J. (2019). A methodological systematic review of meta-ethnography conduct to articulate the complex analytical phases. BMC Medical Research Methodology, 19(35), 1-18.
63. Gao, X. ve Guan, J. (2009). Networks of scientific journals: An exploration of Chinese patent data. Scientometrics, 80(1), 283-302.
64. Garfield, E. (1980). Bradford's Law and related statistical patterns. Current Contents, 19, 5-12. http://garfield.library.upenn.edu/essays/v4p476y1979-80.pdf.
65. Gilbert, G. N. (1977). Referencing as persuasion. Social Studies of Science, 7(1), 113-122.
66. Glasziou, P., Irwig, L., Bain, C. ve Colditz, G. (2001). Systematic reviews in health care: A practical guide. New York: Cambridge University Press.
67. Gough, D. (2007). Weight of evidence: A framework for the appraisal of the quality and relevance of evidence. Research Papers in Education, 22(2), 213-228.
68. Gough, D. ve Thomas, J. (2016). Systematic reviews of research in education: Aims, myths and multiple methods. Review of Education, 4(1), 84-102.
69. Gough, D., Oliver, S. ve Thomas, J. (2012). Introducing systematic reviews. An introduction to systematic reviews (Edt: D. Gough, S. Oliver ve J. Thomas). London: Sage. ss. 1-16.
70. Gümüş, S. (2018). Nitel araştırmaların sistematik derlemesi: Meta-sentez. Eğitim Yönetiminde Araştırma (Edt: K. Beycioğlu, N. Özer, Y. Kondakçı). Ankara: Pegem Akademi. ss. 533-552.
71. Gümüş, S., Bellibaş, M. Ş., Gümüş, E. ve Hallinger, P. (2019): Science mapping research on educational leadership and management in Turkey: A bibliometric review of international publications. School Leadership & Management, 40(1), 23-44.
72. Hallinger, P. (2013). A conceptual framework for systematic reviews of research in educational leadership and management. Journal of Educational Administration, 51(2), 126-149.
73. Hallinger, P. (2014). Reviewing reviews of research in educational leadership: An empirical assessment. Educational Administration Quarterly, 50(4), 539-576.
74. Hallinger, P. (2017) Surfacing a hidden literature: A systematic review of research on educational leadership and management in Africa. Educational Management Administration & Leadership DOI: 1741143217694895.
75. Hallinger, P. (2019). Science mapping the knowledge base on educational leadership and management from the emerging regions of Asia, Africa and Latin America, 1965-2018. Educational Management Administration & Leadership, 48(2), 209-230.
76. Hallinger, P. ve Hammad, W. (2017). Knowledge production on educational leadership and management in Arab societies: A systematic review of research. Educational Management Administration & Leadership, 47(1), 20-36.
77. Hallinger, P. ve Kovačević, J. (2019). A bibliometric review of research on educational administration: science mapping the literature, 1960 to 2018. Review of Educational Research, 89(3), 335-369.
78. Hallinger, P., Lee, T. H. T. ve Szeto, E. (2013). Review of research on educational leadership and management in Hong Kong, 1995-2012: topographical analysis of an emergent knowledge base. Leadership and Policy in Schools, 12, 256-281.
79. Hertzel, D. H. (2003). Bibliometrics history. Encyclopedia of library and information science (Edt: M. A. Drake). New York: Marcel Dekker. ss. 288-328.
80. Higgins, J. P. T., Lasseron, T., Chandler, J., Tovey, D., Thomas, J., Flemyng, E. ve Churchil, R. (2020). Methodological expectations of cochrane ıntervention reviews (MECIR). London: Cochrane. https://community.cochrane.org/sites/default/files/uploads/Version%20March%202020%20Final%20Online%20version.pdf.
81. Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J. ve Welch, V. A. (Eds.) (2019). Cochrane handbook for systematic reviews of interventions (2nd Edition). Hoboken, NJ : Wiley-Blackwell.
82. Holder, L. B. ve Cook, D. J. (2007). Introduction. Mining graph data (Edt: D. J. Cook ve L. B. Holder). Hoboken, NJ: John Wiley & Sons, Inc. ss. 1-14.
83. Hong, Q. N. vd. (2018). Mixed methods appraisal tool (MMAT) version 2018 user guide. Montréal, Canada: McGill University. Ioannidis, J. P. A. (2018). Meta-research: Why research on research matters. Plos Biology, 16(3), e2005468. https://doi.org/10.1371/journal.pbio.2005468.
84. Ioannidis, J. P., Fanelli, D., Dunne, D. D., Goodman, S. N. (2015). Meta-research: Evaluation and improvement of research methods and practices. Plos Biology, 13(10), e1002264. https://doi.org/10.1371/journal.pbio.
85. Jensen, L. ve Allen, M. (1996). Meta-synthesis of qualitative findings. Qualitative Health Research 6(4), 553-560.
86. Jesson, J. K., Matheson, L. ve Lacey, F. M. (2011). Doing your literature review: Traditional and systematic techniques. London: Sage.
87. Karaçam, Z. (2013). Sistematik derleme metodolojisi: Sistematik derleme hazırlamak için bir rehber. Dokuz Eylül Üniversitesi Hemşirelik Yüksekokulu Elektronik Dergisi, 6(1), 26-33.
88. Karadağ, E. (2009). Türkiye’de eğitim bilimleri alanında yapılmış doktora tezlerinin tematik ve metodolojik açıdan incelemesi: Bir durum çalışması (Doktora Tezi). Marmara Üniversitesi Eğitim Bilimleri Enstitüsü, İstanbul.
89. Kepreotes, E. (2009). The metasynthesis: Reducing the isolation of qualitative research. HNE Handover: For Nurses and Midwives, 2(1), 47-48.
90. Kessler, M. M. (1963). Bibliographic coupling between scientific papers. American Documentation, 14(1), 10-25.
91. Khan, K., Kunz, R. Kleijnen, J. ve Antes, G. (2011). Systematic reviews to support evidence-based medicine (2nd Edition). CRC Press.
92. Koehler, W. (2001). Information science as “little science”: The implications of a bibliometric analysis of the Journal of the American Society for Information Science. Scientometrics, 51(1), 117-132.
93. Lasserson, T. J., Thomas, J., ve Higgins, J. P. T. (2019). Starting a review. In J. P. T. Higgins vd. (Eds), Cochrane handbook for systematic reviews of interventions (2nd Edition). Hoboken: Wiley-Blackwell. ss. 3-12.
94. Last, J. M. (2001). A dictionary of epidemiology. Oxford: Oxford University Press.
95. Lele, U. J. (2004). The CGIAR at 31: An independent meta-evaluation of the Consultative Group on International Agricultural Research. Washington: World Bank Operations Evaluation Department Publications.
96. Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P., Clarke, M., Devereaux, P. J., Kleijnen, J. ve Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. PLoS Medicine, 6(7), 1-28.
97. Lincoln, Y. S. ve Guba, E. G. (1985). Naturalistic Inquiry. Beverly Hills, CA: Sage Publications Inc., http://books.google.com.tr/books?id=2oA9aWlNeooC, [EriĢim Tarihi: 24/10/2011].
98. Littell, J. H., Corcoran, J. ve Pillai, V. (2008). Systematic reviews and meta-analysis. New York: Oxford University Press. Macdonald, G. (2000). Social care: rhetoric and reality. In H. T. O. Davies, S. M. Nutley ve P. C. Smith (Eds.), What works? Evidence-based policy and practice in public services (pp. 117-140). Bristol: Policy Press.
99. MacKenzie, H., Dewey, A., Drahota, A., Kilburn, S., Kalra, P. R., Fogg, C. ve Zachariah, D. (2012). Systematic reviews: What they are, why they are important, and how to get involved. Journal of Clinical and Preventive Cardiology, 1(4), 193-202.
100. Marcus, S. E., Moy, M. ve Coffman, T. (2007). Social network analysis. Mining graph data (Edt: D. J. Cook ve L. B. Holder). Hoboken, NJ: John Wiley & Sons, Inc. ss. 443-468.
101. McBurney, M. K. ve Novak, P. L. (2002). What is bibliometrics and why should you care? Proceedings. IEEE International Professional Communication Conference, Portland, OR, USA, 17-20 September 2002. ss. 108-114. https://ieeexplore.ieee.org/document/1049094.
102. McCain, K. W. (1990). Mapping authors in intellectual space: A technical overview. Journal of the American Society for Information Science, 41(6), 433-443.
103. McCain, K. W. (1991). Mapping economics through the journal literature: An experiment in journal cocitation analysis. Journal of the American Society for Information Science and Technology, 42(4), 290-296.
104. Moher, D., Booth, A. ve Stewart, L. (2014). How to reduce unnecessary duplication: use PROSPERO. BJOG, 121, 784-786.
105. Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A. ve PRISMA-P Group (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1-9.
106. Newman, M. E. J. (2001a). Scientific collaboration networks. Physical Review E, 64, DOI: 10.1103/PhysRevE.64.016131.
107. Newman, M. E. J. (2001b). The structure of scientific collaboration networks. PNAS, 98(2), 404-409.
108. Newman, M. E. J. (2004). Coauthorship networks and patters of scientific collaboration. PNAS, 101(1), 5200-5205.
109. Newman, M. ve Gough, D. (2020). Systematic reviews in educational research: methodology, perspectives and application. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, K. ve Buntins (Eds.), Systematic reviews in educational research: Methodology, perspectives and application. Wiesbaden: Springer VS. ss. 3-22.
110. Noblit, G. W. ve Collins, T. W. (1978). Goals, race and roles: Staff and student patterns in the desegregation process. Paper presented at the American Educational Research Association Annual Meeting. Toronto, Canada, March, 1978. ERIC Number: ED154097.
111. Noblit, G. W. ve Hare, R. D. (1983). Meta-ethnography: Issues in the synthesis and replication of qualitative research. Paper presented at the 67th Annual Meeting of the American Educational Research Association. Montreal, Quebec, April 11-15, 1983. ERIC Number: ED231853.
112. Noblit, G. W. ve Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies (Vol. 11). Newbury Park: Sage. http://books.google.com.
113. Oakley, A. (2002). Social science and evidence-based everything: The case of education. Educational Review, 54(3), 277-286.
114. OECD (2011). The OECD meta-evaluation: overview of evaluations in Strengthening Accountability in Aid for Trade. https://www.oecd-ilibrary.org/
115. Paterson, B. L., Thorne, S. E., Canam, C. ve Jillings, C. (2001). Meta-study of qualitative health research: A practical guide to meta-analysis and meta-synthesis. Thousand Oaks, CA: Sage Publications.
116. Petticrew, M. (2001). Systematic reviews from astronomy to zoology: Myths and misconceptions. British Medical Journal, 322, 98-101.
117. Petticrew, M. ve Roberts, H. (2005). Systematic reviews in the social sciences: A practical guide. Oxford: Blackwell.
118. Polat, S. ve Ay, O. (2016). Meta-sentez: Kavramsal bir çözümleme. Eğitimde Nitel Araştırmalar Dergisi, 4(1), 52-64.
119. PRISMA (2020). Preferred Reporting Items for Systematic Reviews and Meta-Analyses. http://www.prisma-statement.org/.
120. Pritchard, A. (1969). Statistical bibliography or bibliometrics. Journal of Documentation, 25(4), 348-349.
121. PROSPERO (2020). PROSPERO - International prospective register of systematic reviews. https://www.crd.york.ac.uk/prospero/.
122. Rehn, C., Gornitzki, C., Larsson, A. ve Wadskog, D. (2014). Bibliometric handbook for Karolinska Institutet. Karolinska Institutet University Library Publications. https://kib.ki.se/sites/default/files/bibliometric_handbook_2014.pdf.
123. Sağlam, M. ve Yüksel, İ. (2007). Program değerlendirmede meta-analiz ve meta-değerlendirme yöntemleri. Dumlupınar Üniversitesi Sosyal Bilimler Dergisi, 18, 175-188.
124. Sandelowski, M. ve Barroso, J. (2003). Toward a metasynthesis of qualitative findings on motherhood in HIV-positive women. Research in Nursing and Health, 26, 153-170.
125. Sandelowski, M., Docherty, S. ve Emden, C. (1997). Focus on qualitative methods - Qualitative meta synthesis: Issues and techniques. Research in Nursing & Health, 20, 365-371.
126. Scriven, M. (1969). An introduction to meta-evaluation. Educational Product Report, 2(5), 36-38.
127. Scriven, M. (1981). Evaluation thesaurus (3rd Edition). California: Edgepress.
128. Scriven, M. (2009). Meta-evaluation revisited. Journal of Multi Disciplinary Evaluation, 6(11), iii-viii.
129. Scriven, M. (2015a). The Meta-Evaluation Checklist. http://michaelscriven.info/images/META-EVALUATION,Nov.13.2015.docx.
130. Scriven, M. (2015b). Key Evaluation Checklist (KEC). http://michaelscriven.info/images/MS_KEC_8-15-15.doc.
131. Selçuk, Z., Palancı, M., Kandemir, M. ve Dündar, H. (2014). Eğitim ve bilim dergisinde yayınlanan araştırmaların eğilimleri: İçerik analizi. Eğitim ve Bilim, 39(173), 430-453.
132. Sengupta, I. N. (1992). Bibliometrics, informetrics, scientometrics and librametrics: An overview. Libri, 42(2), 75-98.
133. Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A. ve the PRISMA-P Group (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. BMJ, 349, g7647.
134. Silverman, D. (2014). Interpreting qualitative data: Methods for analysing talk, text and interaction. London: SAGE.
135. Skillicorn, D. (2007). Understanding complex datasets: Data mining with matrix decompositions. Chapman & Hall/Crc Data Mining.
136. Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265-269.
137. Small, H. (1978). Cited documents as concept symbols. Social Studies of Science, 8(3), 327-340.
138. Small, H. ve Koenig, M. E. D. (1977). Journal clustering using a bibliographic coupling method. Information Processing and Management, 13(5), 277-288.
139. Smith, L. C. (1981). Citation analysis. Library Trends, 30(1), 83-106.
140. Souza, A. Q. (2017). Meta-Evaluation and Analysis of Project Evaluations 2016. https://www.welthungerhilfe.org/
141. Stern, P. N. ve Harris, C. C. (1985). Women’s health and the self-care paradox: A model to guide self-care readiness - clash between the client and nurse. Health Care for Women International, 6, 151-163.
142. Strobel, J. ve van Barneveld, A. (2009). When is PBL more effective? A meta-synthesis of meta-analyses comparing PBL to conventional classrooms. The Interdisciplinary Journal of Problem-Based Learning, 3(1), 44-58.
143. Stufflebeam, D. L. (1978). Meta evaluation: An overview. Evaluation and the Health Professions, 1(1), 17-43.
144. Stufflebeam, D. L. (2000). The methodology of metaevaluation as reflected in metaevaluations by the Western Michigan University Evaluation Center. Journal of Personnel Evaluation in Education, 14, 95-125.
145. Stufflebeam, D. L. (2001). The metaevaluation imperative. American Journal of Evaluation, 22(2), 183-209.
146. Suri, H. ve Clarke, D. (2009). Advancements in research systhesis methods: From a methodologically inclusive perspective. Review of Educational Research, 79(1), 395-430.
147. Şahiner, M. (2017). Program değerlendirmenin değerlendirmesi: Meta değerlendirme. Türkiye Klinikleri Tıp Eğitimi, 2(1), 51-55.
148. Tavşancıl, E. ve Aslan, E. (2001). Sözel, yazılı ve diğer materyaller için içerik analizi ve uygulama örnekleri. İstanbul: Epsilon Yayınevi.
149. Thelwall, M. (2005). Webometrics. Encyclopedia of library and information sciences (Edt: M. J. Bates ve M. N. Maack). New York: Taylor & Francis. ss. 1-8.
150. Thomas, J. ve Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8(45), 1-10.
151. Thorne, S. E., Jensen, L., Kearney, M. H., Noblit, G. W. ve Sandelowski, M. (2004). Qualitative metasynthesis: Reflections on methodological orientation and ideological agenda. Qualitative Health Research, 14(10), 1342-1365.
152. Tingle, L. R., DeSimone, M. ve Covington B. (2003). A meta evaluation of 11 school-based smoking prevention programs. Journal of School Health, 73(2, 64-67.
153. Torgerson, C. (2003). Systematic reviews. London: Continuum International Publishing Group.
154. TÜBİTAK (2009). Ar-Ge değerlendirme ve etki analizinde kullanılan tanım ve temel kavramlar. TÜBİTAK Bilim, Teknoloji ve Yenilik Politikaları Daire Başkanlığı. Ankara. http://www.tubitak.gov.tr/tubitak_content_files/BTYPD/arsiv/ArGe_degerlendirme_etki_analizi_tanim_kavram.pdf. Erişim Tarihi: 11 Mayıs 2012.
155. Turan, S., Karadağ, E., Bektaş, F. ve Yalçın, M. (2014). Türkiye’de eğitim yönetiminde bilgi üretimi: Kuram ve Uygulamada Eğitim Yönetimi Dergisi 2003-2013 yayınlarının incelenmesi. Kuram ve Uygulamada Eğitim Yönetimi, 20(1), 93-119.
156. Ukşul, Ş. (2016). Türkiye’de eğitimde ölçme ve değerlendirme alanında yapılmış bilimsel yayınların sosyal ağ analizi ile değerlendirilmesi: Bir bibliyometrik çalışma (Yüksek Lisans Tezi). Akdeniz Üniversitesi Eğitim Bilimleri Enstitüsü, Antalya
157. Ültay, E. ve Aydın, M. (2017). Fen bilimleri eğitiminde yapılmış nitel çalışmaların içerik analizi. Trakya Üniversitesi Eğitim Fakültesi Dergisi, 7(2), 701-720.
158. Umut-Zan, B. (2012). Türkiye’de bilim dallarında karşılaştırmalı bibliyometrik analiz çalışması (Doktora Tezi). Ankara Üniversitesi Sosyal Bilimler Enstitüsü, Ankara.
159. UN-Women GERAAS (2018). Global Evaluation Report Assessment and Analysis System: 2017 Meta Evaluation. https://gate.unwomen.org/EvaluationDocument/Download?evaluationDocumentID=9185
160. Van Eck, N. J. ve Waltman, L. (2009). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics, 84, 523-538.
161. Walsh, D. ve Downe, S. (2005). Meta-synthesis method for qualitative research: A literature review. Journal of Advanced Nursing, 50(2), 204-211.
162. Walsh, J., Hochbrueckner, R., Corcoran, J. ve Spence, R. (2016). The lived experience of schizophrenia: A systematic review and meta-synthesis. Social Work in Mental Health, 14(6), 607-624.
163. Wasserman, S. ve Faust, K. (1994). Social network analysis: Methods and applications. Cambridge, UK: Cambridge University Press.
164. Wasserman, S., Scott, J. ve Carrington, P. J. (2005). Introduction. Models and methods in social network analysis. structural analysis in the social sciences (Edt: P. J. Carrington, J. Scott ve S. Wasserman). New York: Cambridge University Press. ss. 1-7.
165. White, H. D. ve McCain, K. W. (1998). Visualizing a discipline: An author co-citation analysis of information science, 1972-1995. Journal of the American Society for Information Science, 49(4), 327-356.
166. Wilder, S. (2014). Effects of parental involvement on academic achievement: A metasynthesis. Educational Review, 66(3), 377-397.
167. Woodside, A. G. ve Sakai, M. Y. (2003). Meta-evaluation: Assessing alternative methods of performance evaluation and audits of planned and implemented marketing strategies. In A. G. Woodside (Edt.) Evaluating Marketing Actions and Outcomes (Advances in Business Marketing and Purchasing, Vol. 12). Bingley: Emerald. pp. 549-663.
168. Yalçın, H. ve Esen, M. (2016). Bilimi ölçümlemek: Bilimin metrisi. Bilim, teknoloji ve inovasyon çağında araştırma üniversitesi olmak (Edt: H. Yalçın, M. Esen, S. Burmaoğlu ve M. F. Sorkun). Ankara: Pegem Akademi. ss. 101-128.
169. Yalçın, S. (2016). Ölçme ve değerlendirme alanındaki dergilerde yayımlanan makalelerin içerik analizi. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 49(1), 65-84.
170. Yıldırım, A. ve Şimşek, H. (2011). Sosyal bilimlerde nitel araştırma yöntemleri (8. Baskı). Ankara: Seçkin.
171. Yıldız, A. (2002). Türkiye’deki yetişkin eğitimi araştırmaları: Amaç, kapsam, yöntem ve eğilimler. Yayınlanmamış Yüksek Lisans Tezi. Ankara Üniversitesi Eğitim Bilimleri Fakültesi, Eğitim Bilimleri Enstitüsü, Ankara.
172. Yılmaz, K. (2018). Türkiye’deki eğitim yönetimi alanı ile ilgili çalışmalara eleştirel bir bakış. Journal of Human Sciences, 15(1), 123-154.
173. Yılmaz, K. (2019). Türkiye’de eğitim yönetimi alanında yapılan örgütsel davranış makalelerindeki yönelimler. Türkiye Eğitim Dergisi, 4(2), 81-103.
174. Yılmaz, K. ve Altınkurt, Y. (2012). An examination of articles published on preschool education in Turkey. Kuram ve Uygulamada Eğitim Bilimleri, 12(4), 3227-3241.
175. Yılmaz, K. ve Çelik, M. (2018). Sosyal ağ kuramı. Yönetim ve Eğitim Yönetimi Kuramları (Edt: K. Demir & K. Yılmaz). Ankara: Pegem Akademi Yayıncılık. ss. 197-219.
176. Yücel-Toy, B. (2015). Türkiye’deki hizmet öncesi öğretmen eğitimi araştırmalarının tematik analizi ve öğretmen eğitimi politikalarının yansımaları. Eğitim ve Bilim, 40(178), 23-60.
177. Zawacki-Richter, O. (2020). Introduction: Systematic reviews in educational research. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, K. ve Buntins (Eds.), Systematic reviews in educational research: Methodology, perspectives and application. Wiesbaden: Springer VS. ss. v-xiv.
178. Zhao, D. ve Strotmann, A. (2008). Evolution of research activities and intellectual influences in information science 1996–2005: Introducing author bibliographic-coupling analysis. Journal of the American Society for Information Science and Technology, 59(13), 2070-2086.
179. Zhao, S. (1991). Metatheory, metamethod, meta-data-analysis: What, why, and how? Sociological Perspectives, 34, 377-390.
180. Zupic, I. ve Cater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods, 18(3), 429-472.