Comparison of the performances of non-parametric k-sample test procedures as an alternative to one-way analysis of variance

Abstract

Objectives: The performances of the Kruskal-Wallis test, the van der Waerden test, the modified version of Kruskal-Wallis test based on permutation test, the Mood's Median test and the Savage test, which are among the non-parametric alternatives of one-way analysis of variance and included in the literature, to protect the Type-I error probability determined at the beginning of the trial at a nominal level, were compared with the F test.

Methods: Performance of the tests to protect Type-I error; in cases where the variances are homogeneous/heterogeneous, the sample sizes are balanced/unbalanced, the distribution of the data is in accordance with the normal distribution/the log-normal distribution, how it is affected by the change in the number of groups to be compared has been examined on simulation scenarios.

Results: The Kruskal-Wallis test, the van der Waerden test, the modified version of the Kruskal-Wallis test based on the permutation test were not affected by the distribution of the data, but by the violation of the homogeneity of the variances. The performance of the Mood's Median test and the Savage test were not found to be sufficient in terms of protection of theType-I error compared to other tests.

Conclusions: It was determined that the Kruskal-Wallis test, the van der Waerden test, the modified version of Kruskal-Wallis test based on permutation test were not affected by the distribution of the data and tended to preserve the Type-І error when the variances were homogeneous.

Keywords

• Analysis of variance
• conformity of normal distribution
• non-parametric k-sample tests

References

1. 1. Luepsen H. Comparison of nonparametric analysis of variance methods: a vote for van der Waerden. Commun Stat Simul Comput 2018;47:2547-76.
2. 2. Moder K. Alternatives to F-test in one way ANOVA in case of heterogeneity of variances (a simulation study). Psychol Test Assess Model 2010;52:343-53.
3. 3. Bishop TA, Dudewicz EJ. Exact analysis of variance with unequal variances: test procedures and tables. Technometrics 1978;20:419-30.
4. 4. McSeeney M, Katz B. Nonparametric statistics: use and nonuse. Percept Mot Skills 1978;46(3_suppl):1023-32.
5. 5. Pearson ES. The analysis of variance in cases of non-normal variation. Biometrika 1931;23:114-33.
6. 6. Glass G, Peckham P, Sande J. Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Rev Educ Res 1972;42:237-88.
7. 7. Wilcox RR. ANOVA: a paradigm for low power and misleading measures of effect size? Rev Educ Res 1995;65:51-77.
8. 8. Buning H. Robust analysis of variance. J Appl Stat 1997;24:319-32.

9. 9. R Development Core Team. R: A Language and Environment for Statistical Computing [Computer software manual]. Vienna, Austria:. [cited 2018] Available from http://www.Rproject.org/
10. 10. Peterson K. Six modifications of the aligned rank transform test for interaction. J Modern Appl Stat Methods 2002;1:100-9.
11. 11. Kruskal WH, Wallis A. Use of ranks in one-criterion variance analysis. J Am Stat Assoc 1952;47:583-621.
12. 12. Fisher RA. The Design of Experiments. Edinburgh: Oliver and Boyd; 1935.
13. 13. Hecke TV. Power Study of Anova versus Kruskal-Wallis Test, 2010.
14. 14. Odiase JI, Ogbonmwan SM. JMASM20: exact permutation critical values for the Kruskal-Wallis One-way ANOVA. J Modern Appl Stat Methods 2005;4:609-20.
15. 15. Brown GW, Mood AM. On Median Tests for Linear Hypotheses. University of California Press, 1951: pp. 159-66.
16. 16. Conover WJ. Practical Nonparameteric Statistics. 3rd ed. Wiley; 1999: p. 396-406.
17. 17. van der Waerden B. Order Tests for The Two-Sample Problem II, III, Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappen. Serie A 1953;564:303-10 and 311-6.
18. 18. Hajek J. A Course in Nonparametric Statistics. San Francisco: Holden-Day, 1969: p.83.
19. 19. Blanca M, Alarcón R, Arnau J, Bono R, Bendayan R. Non-normal data: Is ANOVA still a valid option? Psicothema 2017;29:552-7.
20. 20. Clinch J, Kesselman H. Parametric alternatives to the analysis of variance. J Educ Stat 1982;7:207-14.
21. 21. Gamage J, Weerahandi S. Size performance of some tests in one-way ANOVA. Commun Stat Simul Comput 1998;27:625-40.
22. 22. Lantz B. The impact of sample non-normality on ANOVA and alternative methods. Br J Math Stat Psychol 2013;66:224-44.
23. 23. Schmider E, Ziegler M, Danay E, Beyer L, Bühner M. Is it really robust? Reinvestigating the robustness of ANOVA against violations of the normal distribution assumption. Methodology (Gott) 2010;6:147-51.
24. 24. Brown MB, Forsythe AB. The small sample behavior of some statistics which test the equality of several means. Technometrics 1974;16:129-32.
25. 25. De Beuckelaer A. A closer examination on some parametric alternatives to the ANOVA F-test. Stat Papers 1996;37:291-305.
26. 26. Lee S, Ahn C. Modified ANOVA for unequal variances. Commun Stat Simul Comput 2003;32:987-1004.
27. 27. Li X, Wang J, Liang H. Comparison of several means: a fiducial based approach. Comput Stat Data Analysis 2011;55;1993-2002.
28. 28. Lu F, Mathew T. A parametric bootstrap approach for ANOVA with unequal variances: fixed and random models. Comput Stat Data Analysis, 2007;51:5731-42.
29. 29. Markowski CA. Conditions for the effectiveness of a preliminary test of variance. Am Stat 1990;44:322-6.
30. 30. Keselman HJ, Rogan JC, Fier-Walsh BJ. An evaluation of some non-parametric and parametric tests for location equality. Br J Math Stat Psychol 1977;30:213-21.
31. 31. Tomarken A, Serlin RC. Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures. Psychol Bull 1986;99:90-9.
32. 32. Hoeffding W. Optimum" nonparametric tests. Berkeley Symposium on Mathematical Statistics and Probability. Universy of California 2nd ed. 1951: pp.83-92.
33. 33. Terry MH. Some rank order test which are most powerful aganist specific parametric alternatives. Ann Math Stat 1952;23:346-66.
34. 34. Luh W, Guo J. Approximate transformation trimmed mean methods to the test of simple linear regression slope equality. J Appl Stat 2000;27:843-57.
35. 35. Jett D, Speer J. Comparison of parametric and nonparametric tests for differences in distribution. Proceedings of The National Conference On Undergraduate Research (NCUR) 2016 University of North Carolina-Asheville Asheville, North Carolina April 7-9, 2016: 1765-70.