Comparison of the performances of parametric k-sample test procedures as an alternative to one-way analysis of variance

Abstract

Objectives: The performances of the Welch test, the Alexander-Govern test, the Brown-Forsythe test and the James Second-Order test, which are among the parametric alternatives of one-way analysis of variance and included in the literature, to protect the Type-I error probability determined at the beginning of the trial at a nominal level, were compared with the F test.

Methods: Performance of the tests to protect Type-I error; in cases where the variances are homogeneous and heterogeneous, the sample sizes are balanced and unbalanced, the distribution of the data is in accordance with the normal distribution and the log-normal distribution, how it is affected by the change in the number of groups to be compared has been examined on simulation scenarios.

Results: The Welch test, the Alexander-Govern test and the James Second-Order test were not affected by the distribution and performed well in situations where variances were heterogeneous. The Brown-Forsythe test was not affected by the distribution, it performed well when the variance was homogeneous and the sample size in the groups to be compared was not equal.

Conclusions: The Welch test, the Alexander-Govern test and the James Second-Order test are the tests that can be recommended as an alternative to the F test.

Keywords

• Analysis of variance
• conformity of normal distribution
• parametric k-sample tests

References

1. 1. Luepsen H. Comparison of nonparametric analysis of variance methods: a vote for van der Waerden. Commun Stat Simul Comput 2017;47:2547-76.
2. 2. Moder K. Alternatives to F-test in one way ANOVA in case of heterogeneity of variances (a simulation study). Psychol Test Assess Model 2010;52:343-53.
3. 3. Pearson ES. The analysis of variance in cases of non-normal variation. Biometrika 1931;23:114-33.
4. 4. Glass GV, Peckham PD, Sanders JR. Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Rev Educ Res 1972;42:237-88.
5. 5. Wilcox RR. ANOVA: a paradigm for low power and misleading measures of effect size? Rev Educ Res 1995;65:51-77.
6. 6. Buning H. Robust analysis of variance. J Appl Stat 1997;24:319-32.
7. 7. R Development Core Team. R: A Language and Environment for Statistical Computing [Computer software manual]. Vienna, Austria:. [cited 2018] Available from http://www.Rproject.org/
8. 8. Peterson K. Six modifications of the aligned rank transform test for interaction. J Modern Appl Stat Methods 2002;1:100-9.

9. 9. Cribbie RA, Fiksenbaum L, Keselman HJ, Wilcox RR. Effect of non-normality on test statistics for one-way independent groups designs. Br J Math Stat Psychol 2012;65:56-73.
10. 10. Alexander R, Govern D. A new and simpler approximation for anova under variance heterogeneity. J Educ Stat 1994;19:91-101.
11. 11. Hill G. Algorithm 395. Student's t-distribution. Commun ACM 1970;13:617-9.
12. 12. James GS. The comparison of several groups of observations when the ratios of the population variances are unknown. Biometrika 1951;38:324-9.
13. 13. Oshima T, Algina J. Type-I error rates for James’s second-order test and Wilcox’s Hm test under heteroscedasticity and non-normality. Br J Math Stat Psychol 1992;45:255-63.
14. 14. Dag O, Dolgun A, Konar N. One-way tests: an R package for one-way tests in independent groups designs. R J 2018;10:175-99.
15. 15. Brown M, Forsythe A. The small sample behavior of some statistics which test the equality of several means. Technometrics 1994:16:129-32.
16. 16. Satterhwaite FE. Synthesis of variance. Psychometrika 1941;6:309-316.
17. 17. Blanca M, Alarcón R, Arnau J, Bono R, Bendayan R. Non-normal data: Is ANOVA still a valid option? Psicothema 2017;29:552-7.
18. 18. Clinch J, Kesselman H. Parametric alternatives to the analysis of variance. J Educ Behav Stat 1982;7:207-14.
19. 19. Gamage J, Weerahandi S. Size performance of some tests in one-way ANOVA. Commun Stat Simul Comput 1998;27:625-40.
20. 20. Lantz B. The impact of sample non-normality on ANOVA and alternative methods. Br J Math Stat Psychol 2013;66:224-44.
21. 21. Schmider E, Ziegler M, Danay E, et al. Is it really robust? Reinvestigating the robustness of ANOVA against violations of the normal distribution assumption. Methodology 2010;6:147-51.
22. 22. Bishop TA, Dudewicz EJ. Exact analysis of variance with unequal variances: test procedures and tables. Technometrics 1978;20:419-30.
23. 23. Brown MB, Forsythe AB. The small sample behavior of some statistics which test the equality of several means. Technometrics 1974;16:129-32.
24. 24. De Beuckelaer A. A closer examination on some parametric alternatives to the ANOVA F-test. Stat Pap (Berl) 1996;37:291-305.
25. 25. Lee S, Ahn C. Modified ANOVA for unequal variances. Commun Stat Simul Comput 2003;32:987-1004.
26. 26. Li X, Wang J, Liang H. Comparison of several means: a fiducial based approach. Comput Stat Data Anal 2011;55:1993-2002.
27. 27. Lu F, Mathew T. A parametric bootstrap approach for ANOVA with unequal variances: fixed and random models. Comput Stat Data Anal 2007;51:5731-42.
28. 28. Markowski CA. Conditions for the effectiveness of a preliminary test of variance. Am Stat 1990;44:322-6.
29. 29. Keselman HJ, Rogan JC, Fier-Walsh BJ. An evaluation of some non-parametric and parametric tests for location equality. Br J Math Stat Psychol 1977;30:213-21.
30. 30. Tomarken A, Serlin RC. Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures. Psychol Bull 1986;99:90-9.
31. 31. Penfield D. Choosing a two- sample location test. J Exp Educ 1994;62:343-60.
32. 32. Lix L, Keselman J, Keselman H. Concequences of assumption violations revisited: a quantitative review of alternatives to the one-way analysis of variance F test. Rev Educ Res 1996;66:579-619.
33. 33. Hartung J, Argaç D, Makambi K. Small sample properties of tests on homogeneity in one-way anova and meta-analysis. Stat Pap (Berl) 2002;43:197-235.
34. 34. Rafinetti R. Demonstrating the concequences of violations of assumptions in between-subjects analysis of variance. Teach Psychol 1996;23:51-4.
35. 35. Wilcox RR, Charlin V, Thompson KL. NewMonte Carlo results on the robustness of the ANOVA F, W, and F statistics. Commun Stat Simul Comput 1986;15:933-44.
36. 36. Myers L. Comparability of the James’s second-order aproximantion tets and the Alexander and Govern a statistic for non-normal heterosdecatic data. J Stat Comput Simul 1998;60:207-23.
37. 37. Wilcox RR. A new alternative to the ANOVA F and new results on James's second-order method. Br J Math Stat Psychol 2011;41:109-17.
38. 38. Roth AJ. Robust trend tests derived and simulated: analogs of the Welch and Brown-Forsythe tests. J Am Stat Assoc 1983;78:972-80.
39. 39. Steel R, Torrie J, Dickey D. Principles and procedures of Statistics: A Biometrical Approach. 3rd ed. New York, NY: McGraw-Hill; 1997.