Research Article
PDF Zotero Mendeley EndNote BibTex Cite

Year 2019, Volume 10, Issue 3, 302 - 314, 04.09.2019
https://doi.org/10.21031/epod.524511

Abstract

References

  • Cohen, L., Manion, L., & Morrison, K. (2002). Research methods in education. London; Routledge.
  • Fish, L. J. (1988). Why multivariate methods are usually vital. Measurement and Evaluation in Counseling and Development, 21, 130-137.
  • Glenn A. Bowen. (2009). Document Analysis as a Qualitative Research Method. Qualitative Research Journal, 9 (2), 27-40.
  • Huberty, C. J., & Morris, J. D. (1989). Multivariate analysis versus multiple univariate analysis. Psychological Bulletin, 105, 302-308.
  • Hummel, T. J., & Sligo, J. R. (1971). Empirical comparison of univariate and multivariate analysis of variance procedures. Psychological Bulletin, 76, 49-57.
  • Kieffer, K. M., Reese, R. J., & Thompson, B. (2001). Statistical techniques employed in AERJ and JCP articles from 1988 to 1997: A methodological review. Journal of Experimental Education, 69, 280-309.
  • R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http:www.R-project.org/.
  • Sherry, A. (2006). Discriminant analysis in counseling psychology research. The Counseling Psychologist, 34, 661-683.
  • Stevens, J. P. (2002). Applied multivariate statistics for the social sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
  • Tonidandel, S., & LeBreton, J. M. (2013). Beyond step-down analysis: A new test for decomposing the importance of dependent variables in MANOVA. Journal of Applied Psychology, 98, 469-477.
  • Venables, W. N. & Ripley, B. D. (2002) Modern Applied Statistics with S. Fourth Edition. Springer, New York. ISBN 0-387-95457-0
  • Warne, R. T. (2014). A primer on multivariate analysis of variance (MANOVA) for behavioral scientists. Practical Assessment, Research & Evaluation, 19(17), 1-10.
  • Warne, R. T., Lazo, M., Ramos, T., & Ritter, N. (2012). Statistical methods used in gifted education journals, 2006-2010. Gifted Child Quarterly, 56, 134-149.
  • Zientek, L. R., & Thompson, B. (2009). Matrix summaries improve research reports: Secondary analyses using published literature. Educational Researcher, 38, 343-352.

Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA

Year 2019, Volume 10, Issue 3, 302 - 314, 04.09.2019
https://doi.org/10.21031/epod.524511

Abstract

ANOVA and MANOVA address different research questions and decision on conducting one or the other of these tests relies on the research purpose. One prominent illegitimate analysis of multivariate data is developed out of conducting multiple ANOVAs rather than conducting a MANOVA. Another common mistake about MANOVA applications is the use of improper post hoc procedure. Post hoc procedures are needed to determine why the null hypothesis was rejected. Although the correct post hoc procedure for MANOVA is descriptive discriminant analysis (DDA), many researchers fail to conduct DDA to interpret their MANOVA results. The purpose of this study is two-fold; (1) we aim to emphasize the theory behind the MANOVA and its appropriate post hoc procedure and make clear distinction between surrogate statistical procedures such as ANOVA; and (2) this study also investigates the extent of incorrect analysis of multivariate dependent variables in educational research in Turkey. First, we provided a small simulation study to demonstrate the extent to which multiple ANOVAs yields contradictory results when they are inadvertently used to test group mean differences on multiple dependent variables. Results of the simulations indicated that MANOVA and multiple ANOVAs had severe disagreements under many conditions. Disagreement rate is elevated under the conditions where MANOVA retains the null hypothesis. Then, we systematically reviewed the archives of three education journals, which are classified as higher-, medium, and lower quality journals. Results indicated that correct use of MANOVA with its proper post hoc procedure is not common practice across educational researchers who publish in Turkish education journals.

References

  • Cohen, L., Manion, L., & Morrison, K. (2002). Research methods in education. London; Routledge.
  • Fish, L. J. (1988). Why multivariate methods are usually vital. Measurement and Evaluation in Counseling and Development, 21, 130-137.
  • Glenn A. Bowen. (2009). Document Analysis as a Qualitative Research Method. Qualitative Research Journal, 9 (2), 27-40.
  • Huberty, C. J., & Morris, J. D. (1989). Multivariate analysis versus multiple univariate analysis. Psychological Bulletin, 105, 302-308.
  • Hummel, T. J., & Sligo, J. R. (1971). Empirical comparison of univariate and multivariate analysis of variance procedures. Psychological Bulletin, 76, 49-57.
  • Kieffer, K. M., Reese, R. J., & Thompson, B. (2001). Statistical techniques employed in AERJ and JCP articles from 1988 to 1997: A methodological review. Journal of Experimental Education, 69, 280-309.
  • R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http:www.R-project.org/.
  • Sherry, A. (2006). Discriminant analysis in counseling psychology research. The Counseling Psychologist, 34, 661-683.
  • Stevens, J. P. (2002). Applied multivariate statistics for the social sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
  • Tonidandel, S., & LeBreton, J. M. (2013). Beyond step-down analysis: A new test for decomposing the importance of dependent variables in MANOVA. Journal of Applied Psychology, 98, 469-477.
  • Venables, W. N. & Ripley, B. D. (2002) Modern Applied Statistics with S. Fourth Edition. Springer, New York. ISBN 0-387-95457-0
  • Warne, R. T. (2014). A primer on multivariate analysis of variance (MANOVA) for behavioral scientists. Practical Assessment, Research & Evaluation, 19(17), 1-10.
  • Warne, R. T., Lazo, M., Ramos, T., & Ritter, N. (2012). Statistical methods used in gifted education journals, 2006-2010. Gifted Child Quarterly, 56, 134-149.
  • Zientek, L. R., & Thompson, B. (2009). Matrix summaries improve research reports: Secondary analyses using published literature. Educational Researcher, 38, 343-352.

Details

Primary Language English
Subjects Social
Published Date Sonbahar
Journal Section Articles
Authors

Lokman AKBAY
BURDUR MEHMET AKİF ERSOY ÜNİVERSİTESİ, EĞİTİM FAKÜLTESİ
0000-0003-4026-5241
Türkiye


Tuncer AKBAY (Primary Author)
BURDUR MEHMET AKİF ERSOY ÜNİVERSİTESİ, EĞİTİM FAKÜLTESİ
0000-0003-3938-1026
Türkiye


Osman EROL
BURDUR MEHMET AKİF ERSOY ÜNİVERSİTESİ, EĞİTİM FAKÜLTESİ
0000-0002-9920-5211
Türkiye


Mustafa KILINÇ
BURDUR MEHMET AKİF ERSOY ÜNİVERSİTESİ, EĞİTİM FAKÜLTESİ
0000-0002-2759-4916
Türkiye

Publication Date September 4, 2019
Published in Issue Year 2019, Volume 10, Issue 3

Cite

Bibtex @research article { epod524511, journal = {Journal of Measurement and Evaluation in Education and Psychology}, issn = {1309-6575}, eissn = {1309-6575}, address = {}, publisher = {Eğitimde ve Psikolojide Ölçme ve Değerlendirme Derneği}, year = {2019}, volume = {10}, pages = {302 - 314}, doi = {10.21031/epod.524511}, title = {Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA}, key = {cite}, author = {Akbay, Lokman and Akbay, Tuncer and Erol, Osman and Kılınç, Mustafa} }
APA Akbay, L. , Akbay, T. , Erol, O. & Kılınç, M. (2019). Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA . Journal of Measurement and Evaluation in Education and Psychology , 10 (3) , 302-314 . DOI: 10.21031/epod.524511
MLA Akbay, L. , Akbay, T. , Erol, O. , Kılınç, M. "Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA" . Journal of Measurement and Evaluation in Education and Psychology 10 (2019 ): 302-314 <https://dergipark.org.tr/en/pub/epod/issue/48297/524511>
Chicago Akbay, L. , Akbay, T. , Erol, O. , Kılınç, M. "Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA". Journal of Measurement and Evaluation in Education and Psychology 10 (2019 ): 302-314
RIS TY - JOUR T1 - Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA AU - Lokman Akbay , Tuncer Akbay , Osman Erol , Mustafa Kılınç Y1 - 2019 PY - 2019 N1 - doi: 10.21031/epod.524511 DO - 10.21031/epod.524511 T2 - Journal of Measurement and Evaluation in Education and Psychology JF - Journal JO - JOR SP - 302 EP - 314 VL - 10 IS - 3 SN - 1309-6575-1309-6575 M3 - doi: 10.21031/epod.524511 UR - https://doi.org/10.21031/epod.524511 Y2 - 2019 ER -
EndNote %0 Journal of Measurement and Evaluation in Education and Psychology Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA %A Lokman Akbay , Tuncer Akbay , Osman Erol , Mustafa Kılınç %T Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA %D 2019 %J Journal of Measurement and Evaluation in Education and Psychology %P 1309-6575-1309-6575 %V 10 %N 3 %R doi: 10.21031/epod.524511 %U 10.21031/epod.524511
ISNAD Akbay, Lokman , Akbay, Tuncer , Erol, Osman , Kılınç, Mustafa . "Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA". Journal of Measurement and Evaluation in Education and Psychology 10 / 3 (September 2019): 302-314 . https://doi.org/10.21031/epod.524511
AMA Akbay L. , Akbay T. , Erol O. , Kılınç M. Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA. JMEEP. 2019; 10(3): 302-314.
Vancouver Akbay L. , Akbay T. , Erol O. , Kılınç M. Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA. Journal of Measurement and Evaluation in Education and Psychology. 2019; 10(3): 302-314.
IEEE L. Akbay , T. Akbay , O. Erol and M. Kılınç , "Inadvertent Use of ANOVA in Educational Research: ANOVA is not A Surrogate for MANOVA", Journal of Measurement and Evaluation in Education and Psychology, vol. 10, no. 3, pp. 302-314, Sep. 2019, doi:10.21031/epod.524511