Comparison of the performances of parametric k-sample test procedures as an alternative to one-way analysis of variance

Year 2023, , 39 - 48, 04.01.2023

Gökhan Ocakoğlu , Aslı Ceren Macunluoglu

https://doi.org/10.18621/eurj.1030038

Abstract

Objectives: The performances of the Welch test, the Alexander-Govern test, the Brown-Forsythe test and the James Second-Order test, which are among the parametric alternatives of one-way analysis of variance and included in the literature, to protect the Type-I error probability determined at the beginning of the trial at a nominal level, were compared with the F test.

Methods: Performance of the tests to protect Type-I error; in cases where the variances are homogeneous and heterogeneous, the sample sizes are balanced and unbalanced, the distribution of the data is in accordance with the normal distribution and the log-normal distribution, how it is affected by the change in the number of groups to be compared has been examined on simulation scenarios.

Results: The Welch test, the Alexander-Govern test and the James Second-Order test were not affected by the distribution and performed well in situations where variances were heterogeneous. The Brown-Forsythe test was not affected by the distribution, it performed well when the variance was homogeneous and the sample size in the groups to be compared was not equal.

Conclusions: The Welch test, the Alexander-Govern test and the James Second-Order test are the tests that can be recommended as an alternative to the F test.

Keywords

Analysis of variance, conformity of normal distribution, parametric k-sample tests

References

• 1. Luepsen H. Comparison of nonparametric analysis of variance methods: a vote for van der Waerden. Commun Stat Simul Comput 2017;47:2547-76.
• 2. Moder K. Alternatives to F-test in one way ANOVA in case of heterogeneity of variances (a simulation study). Psychol Test Assess Model 2010;52:343-53.
• 3. Pearson ES. The analysis of variance in cases of non-normal variation. Biometrika 1931;23:114-33.
• 4. Glass GV, Peckham PD, Sanders JR. Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Rev Educ Res 1972;42:237-88.
• 5. Wilcox RR. ANOVA: a paradigm for low power and misleading measures of effect size? Rev Educ Res 1995;65:51-77.
• 6. Buning H. Robust analysis of variance. J Appl Stat 1997;24:319-32.
• 7. R Development Core Team. R: A Language and Environment for Statistical Computing [Computer software manual]. Vienna, Austria:. [cited 2018] Available from http://www.Rproject.org/
• 8. Peterson K. Six modifications of the aligned rank transform test for interaction. J Modern Appl Stat Methods 2002;1:100-9.
• 9. Cribbie RA, Fiksenbaum L, Keselman HJ, Wilcox RR. Effect of non-normality on test statistics for one-way independent groups designs. Br J Math Stat Psychol 2012;65:56-73.
• 10. Alexander R, Govern D. A new and simpler approximation for anova under variance heterogeneity. J Educ Stat 1994;19:91-101.
• 11. Hill G. Algorithm 395. Student's t-distribution. Commun ACM 1970;13:617-9.
• 12. James GS. The comparison of several groups of observations when the ratios of the population variances are unknown. Biometrika 1951;38:324-9.
• 13. Oshima T, Algina J. Type-I error rates for James’s second-order test and Wilcox’s Hm test under heteroscedasticity and non-normality. Br J Math Stat Psychol 1992;45:255-63.
• 14. Dag O, Dolgun A, Konar N. One-way tests: an R package for one-way tests in independent groups designs. R J 2018;10:175-99.
• 15. Brown M, Forsythe A. The small sample behavior of some statistics which test the equality of several means. Technometrics 1994:16:129-32.
• 16. Satterhwaite FE. Synthesis of variance. Psychometrika 1941;6:309-316.
• 17. Blanca M, Alarcón R, Arnau J, Bono R, Bendayan R. Non-normal data: Is ANOVA still a valid option? Psicothema 2017;29:552-7.
• 18. Clinch J, Kesselman H. Parametric alternatives to the analysis of variance. J Educ Behav Stat 1982;7:207-14.
• 19. Gamage J, Weerahandi S. Size performance of some tests in one-way ANOVA. Commun Stat Simul Comput 1998;27:625-40.
• 20. Lantz B. The impact of sample non-normality on ANOVA and alternative methods. Br J Math Stat Psychol 2013;66:224-44.
• 21. Schmider E, Ziegler M, Danay E, et al. Is it really robust? Reinvestigating the robustness of ANOVA against violations of the normal distribution assumption. Methodology 2010;6:147-51.
• 22. Bishop TA, Dudewicz EJ. Exact analysis of variance with unequal variances: test procedures and tables. Technometrics 1978;20:419-30.
• 23. Brown MB, Forsythe AB. The small sample behavior of some statistics which test the equality of several means. Technometrics 1974;16:129-32.
• 24. De Beuckelaer A. A closer examination on some parametric alternatives to the ANOVA F-test. Stat Pap (Berl) 1996;37:291-305.
• 25. Lee S, Ahn C. Modified ANOVA for unequal variances. Commun Stat Simul Comput 2003;32:987-1004.
• 26. Li X, Wang J, Liang H. Comparison of several means: a fiducial based approach. Comput Stat Data Anal 2011;55:1993-2002.
• 27. Lu F, Mathew T. A parametric bootstrap approach for ANOVA with unequal variances: fixed and random models. Comput Stat Data Anal 2007;51:5731-42.
• 28. Markowski CA. Conditions for the effectiveness of a preliminary test of variance. Am Stat 1990;44:322-6.
• 29. Keselman HJ, Rogan JC, Fier-Walsh BJ. An evaluation of some non-parametric and parametric tests for location equality. Br J Math Stat Psychol 1977;30:213-21.
• 30. Tomarken A, Serlin RC. Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures. Psychol Bull 1986;99:90-9.
• 31. Penfield D. Choosing a two- sample location test. J Exp Educ 1994;62:343-60.
• 32. Lix L, Keselman J, Keselman H. Concequences of assumption violations revisited: a quantitative review of alternatives to the one-way analysis of variance F test. Rev Educ Res 1996;66:579-619.
• 33. Hartung J, Argaç D, Makambi K. Small sample properties of tests on homogeneity in one-way anova and meta-analysis. Stat Pap (Berl) 2002;43:197-235.
• 34. Rafinetti R. Demonstrating the concequences of violations of assumptions in between-subjects analysis of variance. Teach Psychol 1996;23:51-4.
• 35. Wilcox RR, Charlin V, Thompson KL. NewMonte Carlo results on the robustness of the ANOVA F, W, and F statistics. Commun Stat Simul Comput 1986;15:933-44.
• 36. Myers L. Comparability of the James’s second-order aproximantion tets and the Alexander and Govern a statistic for non-normal heterosdecatic data. J Stat Comput Simul 1998;60:207-23.
• 37. Wilcox RR. A new alternative to the ANOVA F and new results on James's second-order method. Br J Math Stat Psychol 2011;41:109-17.
• 38. Roth AJ. Robust trend tests derived and simulated: analogs of the Welch and Brown-Forsythe tests. J Am Stat Assoc 1983;78:972-80.
• 39. Steel R, Torrie J, Dickey D. Principles and procedures of Statistics: A Biometrical Approach. 3rd ed. New York, NY: McGraw-Hill; 1997.