Research Models Used in Doctoral Dissertations in the Area of Education Sciences in Turkey: Quality of Research and Analytical Errors

Abstract

Background Studies examining methodological flaws in educational research since 1960's, consistently demonstrated serious conceptual and methodological problems (Elmore & Woehlke, 1996, p.3). These problems are related to conceptual framework, validity and reliability of research instruments, sample selection procedures and sampling, and appropriateness of research design. Purpose The purpose of this study is to examine quality of research, research designs and types of analytical error in doctoral dissertation in the area of educational sciences in Turkish Universities. Method This study focused on 211 dissertations out of 324 doctoral dissertations completed from 2003 to 2007 in the area of educational sciences. Mainly, due to restrictions on access to dissertations, it was not possible to examine 113 dissertations for the purpose of this study. Educational Research Assessment Form was developed for data collection. The Educational Research Assessment Form consisted of 150 items with ratings from 0 to 10. A varimax rotated factor analysis produced 24 factors with factor loadings greater than or equal to .32 (KMO=.887; X2=1946.109; SD=3825; and Bartlett, p=.000). Internal consistency coefficients, Cronbach's alpha for sub-scales formed as based on factors ranged from .63 to .95. Only four of 24 factors are examined within the scope of this article, including (a) research model, (b) correlational model, (c) experimental model and (d) qualitative model sub-scales. Epistemological document analysis was used as the data collection method. Methodology and errors in dissertations were examined through content analysis by using Educational Research Assessment Form. Findings Findings indicated that nine different research models were used in 211 doctoral dissertation studies. Frequently used models included experimental design (η=85; 37.1%), survey (η=78; 34.0%), correlational study (η=28; 12.2%) and case study (η=26, 11.3%). Each one of causal-comparison, theory building, and research and development designs was used only in one dissertation. In assessing the quality of the designs, the average scores for items varied between 2.68 and 6.37 out of a 10 point scale. Explaining the reason for choosing the design received the lovest score (?=2.68; SD=2.50), while identifying the design received the highest score (?=6.37; SD=2.69). Content analyses demonstrated that research designs were not aproppriate for objectives, research designs named incorrectly and even designs were not defined at all in some studies. Discussion Results demonstrated that survey design or experimental designs were used in 71% dissertations. Similar study by Hsu (2005) examining research published by American Educational Research Journal (AERJ), Journal of Experimental Education (JEE) and Journal of Educational Research (JER) from 1971 to 1998 showed that proportion of studies using survey and experimental designs steadily decreased, while number of case studies and qualitative studies increased. Contrary to findings presented by Hsu, survey design or experimental designs were used in majority of dissertation examined in this study. These findings are consistent with results of studies examining research designs by Balcı (1993) and Akkoyun (1989). Quality of research design consequently diminishes the overall validity and reliability of results. Research model creates the basic structure of the research and the errors in this dimension affect the quality of entire research (Carr, 2007; Neuman, 2007; Thomas & Gorard, 2007).

Keywords

• research in educational sciences , research design , design errors

Eğitim Bilimleri Doktora Tezlerinde Kullanılan Araştırma Modelleri: Nitelik Düzeyleri ve Analitik Hata Tipleri

Öz

Bu araştırmada, Türkiye'de eğitim bilimleri alanında yapılmış doktora tezlerinde kullanılan araştırma modellerinin nitelik düzeyleri ve yapılan analitik hata tipleri belirlenmeye çalışılmıştır. Nitel araştırma yönteminin benimsendiği araştırmada, iç içe geçmiş durum çalışması deseni kullanılmıştır. Araştırmanın kuramsal evreni Türkiye'de eğitim bilimleri alanında yapılmış doktora tezlerinden oluşmaktadır. Araştırmada güncellik ve yöntembilim alanındaki gelişmeler dikkate alınarak belirlenen çalışılabilir evren, Türkiye'de eğitim bilimleri alanında son 5 yılda [2003–2007] yapılan 324 doktora tezinden oluşmaktadır. Araştırmada örnekleme yöntemine başvurulmamış ve çalışma evrenin tamamına ulaşılması hedeflenmiştir. Ancak bazı tezlerin kullanımı ve yayımlanmasın kısıtlı olması nedeniyle örneklem kapsamında ele alınan tez sayısı 211 olarak gerçekleşmiştir. Araştırmada veri toplama aracı olarak, araştırmacı tarafından geliştirilen eğitim araştırmaları değerlendirme ölçütleri formu kullanılmıştır. Araştırmada epistemolojik doküman analizi ile toplanan veriler frekans analizi, betimsel analiz ve tipolojik analiz kullanılarak çözümlenmiştir. Araştırma sonucunda elde edilen bulgulara göre, eğitim bilimleri alanında yapılan doktora tezlerinde kullanılan araştırma modellerinin nitelik düzeylerinin yeterli düzeyinde olmadığı saptanmıştır.

Anahtar Kelimeler

• eğitim bilimleri araştırmaları , araştırma desenleri , araştırmalarda hata tipleri

Kaynakça

1. Akkoyun, F. (1989). Danışma psikolojisindeki araştırmaların gelişimi. II. Ulusal Sosyal Bilimler Kongresi (s.5). Ankara.
2. Balcı, A. (2007). Sosyal bilimlerde araştırma: Yöntem, teknik ve ilkeler. Ankara: Pegema Yayıncılık.
3. Bilgin, N. (2006). Sosyal bilimlerde içerik analizi -Teknikler ve örnek çalışmalar-. Ankara: Siyasal Kitapevi.
4. Carr, W. (2007). Educational research as a practical science International Journal of Research & Method in Education, 30(3), 271-286.
5. Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage Publications.
6. Daniel, L. G. (1998). Use of statistical significance testing in current “general” educational journals: A review of articles with comments for improved practice . Paper presented at the annual meeting of the Association for the Advancement of Educational Research, Ponte Vedra, FL.
7. Dey, I. (2007). Qualitative data analysis: A user-friendly guide for social scientists. London: Routledge Taylor & Francis Group.
8. Dillon, J. T. (1983). The use of questions in educational research. Educational Researcher, 12(9), 19-24.

9. Dunkin, M. J. (1996). Types of errors in synthesizing research in education. Review of Educational Research, 66(2), 87–97.
10. Elmore, P. B., & Woehlke, P. L. (1988). Statistical methods employed in American Educational Research Journal, Educational Researcher and Review of Educational Research from 1978 to 1987. Educational Researcher, 17(9), 9-20.
11. Elmore, P. B., & Woehlke, P. L. (1996). Research methods employed in American Educational Research Journal, Educational Research and Review of Educational Research from 1978 to 1995. (ERIC Document Reproduction Service No. ED397122).
12. Erkuş, A. (2004). Bazı tıp dergilerinin son sayılarındaki makalelerin yöntemsel ve istatistiksel açıdan incelenmesi. Mersin Üniversitesi Tıp Fakültesi Dergisi, 5 (2), 176–181.
13. Forster, N. (1995). The analysis of company documentation. C. Cassell & G. Symon (Eds). Qualitative methods in organizational research: A practical guide. London: Sage Publications.
14. Gall, M. D., Borg, W. R., & Gall, J. P. (2006). Educational research: An introduction. New York: Allyn & Bacon.
15. Gibbs, G. R. (2002). Qualitative data analysis. Buckingman: Open Universty Press.
16. Goodwin, L. D., & Goodwin, W. L. (1985a). An analysis of statistical techniques used in the Journal of Educational Psychology, 1979-1983. Educational Psychologist, 20(1), 13-21.
17. Goodwin, L. D., & Goodwin, W. L. (1985b). Statistical techniques in AERJ articles, 1979-1983: The preparation of graduate students to read educational research literature. Educational Researcher, 14(2), 5-11.
18. Gordon, N. J., Nucci, L. P., West, G. K., Hoerr, W. A., Uğuroglu, M. E., Vukosavich, P., & Tsai, S. (1984). Productivity and citations of educational research: Using educational psychology as the data base. Educational Researcher, 13(7), 14-20.
19. Guba, E. (1990). Subjectivity and objectivity. E. Eisner, & A. Peshkin (Eds.). Qualitative inquiry in education: The continuing debate. (s.74-91). New York: Teachers College Press.
20. Hamburg, M. (1985). Basic statistics: A modern approach. New York: Harcourt, Brace Jovanovich.
21. Hsu, T. (2005). Research methods and data analysis procedures used by educational researchers. International Journal of Research & Method in Education, 28(2), 109–133
22. Jackson, R. L., Drummond, D. K., & Camara, S. (2007). What is qualitative research?. Qualitative Research Reports in Communication, 8(1), 21-28.
23. Karadağ, E. (2009). Türkiye'de eğitim bilimleri alanında yapılmış doktora tezlerinin tematik ve metodolojik açıdan incelenmesi: Bir durum çalışması. Yayınlanmamış doktora tezi. Marmara Üniversitesi, İstanbul.
24. Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research. Orlonda, FL.: Wadsworth Publishing
25. Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., & Levin, J. R. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68, 350–386.
26. Kieffer, K. M., Reese, R. J., & Thompson, B. (2001). Statistical techniques employed in AERJ and JCP articles from 1988 to 1997: A methodological review. Journal of Experimental Education, 69(3), 280-309.
27. Köhler, T. & Stemmler, M. (1997). Normative versus impassive configure frequency analysis in personality research -their use discussed in a reanalysis of data on situation- bound anxiety. European Journal of Personality, 11(1), 69-79.
28. Kümbetoğlu, B. (2005). Sosyolojide ve antropolojide niteliksel yöntem ve araştırma. İstanbul: Bağlam Yayıncılık.
29. Lienert, G. A., & Oeveste, H. Z. (1985). Configural frequency analysis as a statistical tool for developmental research. Educational and Psychological Measurement, 45(2), 301-307.
30. Martinmäki, K., & Rusko, H. (2008). Time-frequency analysis of heart rate variability during immediate recovery from low and high intensity exercise. European Journal of Applied Physiology, 102(3), 353-360.
31. Mayring, R. (2000). Nitel sosyal araştırmaya giriş (A. Gümüş & M.S. Durgun, Çev.) Adana: Baki Kitapevi.
32. McMillan, J. H., & Schumacher, S. (2006). Research in education: Evidence based inquiry. Boston: Brown and Company.
33. Merriam, S. (1998). Qualitative research and case study applications in education. San Francisco: Jossey-Bass.
34. Neuman, L. W. (2007). Toplumsal araştırma yöntemleri: Nitel ve nicel yaklaşımlar (S. Özge, Çev.). İstanbul: Yayın Odası Yayıncılık.
35. Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K.R. (2005). Research in special education: Scientific methods and evidence-based practices. Council for Exiceptional Children, 71(2), 137–148.
36. Onwuegbuzie, A. J. (2002). Common analytical and interpretational errors in educational research: an analysis of the 1998 volume of the British Journal of Educational Psychology. Educational Research Quarterly, 26, 11-22.
37. Onwuegbuzie, A. J., & Daniel L.G. (2003). Typology of analytical and interpretational errors in quantitative and qualitative educational research. Current Issues in Education [On-line], 6(2). http://cie.ed.asu.edu/ volume6/number2/
38. Rowlinson, M. (2004). Historical analysis of company documents. C. Cassell & G. Symon (Eds). Essential guide to qualitative methods in organizational research (s.301-312). London: Sage Pub.
39. Shaver, J. P., & Norton, R. S. (1980). Randomness and replication in ten years of the American educational research. Educational Researcher, 9(1), 9-15.
40. Thomas, G., & Gorard, S. (2007). Quality in education research. International Journal of Research & Method in Education, 30(3), 239-242.
41. Thompson, B. (1998). Five methodology errors in educational research: The pantheon of statistical significance and other faux pas. (ERIC Document Reproduction Service No: ED419023).
42. Türkdoğan, O. (2003). Bilimsel araştırma metodolojisi. İstanbul: Timaş Yayınları.
43. Vockell, E. L., & Asher, W. (1974). Perceptions of document quality and use by educational decision makers and researchers. American Educational Research Journal, 11, 249-258.
44. Walberg, H. J. (1986). Syntheses of research on teaching. M.C. Wittrock (Edit.). Handbook of research on teaching (214-229). New York: Macmillan.
45. Walberg, H. J., Vukosavich, P., & Tsai, S. (1981). Scope and structure of the journal literature in educational research. Educational Researcher, 10(8), 11-13.
46. Ward, A. W., Hall, B. W., & Schramm, C. E. (1975). Evaluation of published educational research: A national survey. American Educational Research Journal, 12, 109-128.
47. Willson, V. L. (1980). Research techniques in AERJ articles: 1969 to 1978. Educational Researcher, 9(6), 5-10.
48. Witta, E. L., & Daniel, L. G. (1998). The reliability and validity of test scores: Are editorial policy changes reflected in journal articles? (ERIC Document Reproduction Service No. ED422366).
49. Yıldırım, A., & Şimşek, H. (2005). Sosyal bilimlerde nitel araştırma yöntemleri. Ankara: Seçkin Yayıncılık.