Investigating homogeneity of variance in normal, skewed-normal, and gamma distributions: A simulation study

Abstract

It is an important point to test the homogeneity of variances in statistical methods such as the t-test or F-test used to make comparisons between groups. An erroneous decision regarding the homogeneity of variances will affect the test to be selected and thus lead to different results. For this reason, there are many tests for homogeneity of variance in the literature. This study aims to examine the type I error and power ratios of Levene, Bartlett, Brown-Forsythe, and Fligner-Killeen tests under different conditions. In this study, conducted within the scope of basic research, analyses were performed using simulated data. The simulation conditions included variance ratio (1:1, 1:2, 1:3, 2:1, 3:1), distributions (normal, skewed-normal, gamma), sample sizes (60, 120, and 240), and ratio of group sizes (1/1, 1/2, 1/4, 1/9). According to the study results, when controlling for type I error is a primary concern, the Brown–Forsythe and Fligner–Killeen tests are recommended, particularly under non-normal distributions. If the power is a major concern for research, the Bartlett’s test and the Levene’s test should be used in general.

Keywords

• Homogeneity of variances
• Bartlett’s test
• Levene’s test
• Brown-Forsythe test
• Fligner-Killeen test

References

1. Abdullah N.F., & Muda, N. (2022). An overview of homogeneity of variance tests on various conditions based on type 1 error rate and power of a test. Journal of Quality Measurement and Analysis, 18(3), 111-130.
2. Ahsanullah, M. (2017). Characterizations of univariate continuous distributions (Vol. 1). Amsterdam: Atlantis Press.
3. Arsham, H., & Lovric, M. (2011). Bartlett's Test. In M. Lovric (Ed.), International encyclopedia of statistical science (pp. 87–88). Springer. https://doi.org/10.1007/978-3-642-04898-2_132
4. Arnold, B.C., Gómez, H.W., & Salinas, H.S. (2014). A doubly skewed normal distribution. Statistics, 49(4), 842-858. https://doi.org/10.1080/02331888.2014.918618
5. Azzalini, A. (1985). A class of distributions which includes the normal ones. Scandinavian Journal of Statistics, 12(2), 171-178. http://www.jstor.org/stable/4615982
6. Bartlett, M.S. (1939, April). A note on tests of significance in multivariate analysis. In Mathematical Proceedings of the Cambridge Philosophical Society (Vol. 35, No. 2, pp. 180-185). Cambridge University Press.
7. Bono, R., Blanca, M.J., Arnau, J., & Gómez-Benito, J. (2017). Non-normal distributions commonly used in health, education, and social sciences: A systematic review. Frontiers in Psychology, 8, 1602. https://doi.org/10.3389/fpsyg.2017.01602
8. Brown, M.B., & Forsythe, A.B. (1974). Robust Tests for the Equality of Variances. Journal of the American Statistical Association, 69(346), 364 367. https://doi.org/10.1080/01621459.1974.104829557

9. Box, G.E.P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance. I: Effect of inequality of variance in one-way classification. Annals of Mathematical Statistics, 25, 290–302. http://www.jstor.org/stable/2236731
10. Chang, C.H., Pal, N., & Lin, J.J. (2017). A revisit to test the equality of variances of several populations. Communications in Statistics-Simulation and Computation, 46(8), 6360-6384. https://doi.org/10.1080/03610918.2016.1202277
11. Conover, W.J., Johnson, M.E., & Johnson, M.M. (1981). A comparative study of test for homogeneity of variances, with applications to the outer continental shelf bidding data. Technometrics, 23, 351–361. https://doi.org/10.2307/1268225
12. Field, A. (2018). Discovering statistics using IBM SPSS statistics. Sage publications limited.
13. Fligner, M.A., & Killeen, T.J. (1976). Distribution-free two-sample tests for scale. Journal of the American Statistical Association 71(353), 210 213. https://doi.org/10.1080/01621459.1976.10481517
14. Fraenkel, J., & Wallen, N. (2009). How to Design and Evaluate Research in Education (7th ed.) McGraw-Hill Education.
15. Gamst, G., Meyers, L.S., & Guarino, A.J. (2008). Analysis of variance designs: A conceptual and computational approach with SPSS and SAS. Cambridge University. https://doi.org/10.1017/CBO9780511801648
16. Gastwirth, J.L., Gel, Y.R., & Miao, W. (2009). The impact of Levene’s test of equality of variances on statistical theory and practice. Statistical Science, 24(3), 343-360. http://dx.doi.org/10.1214/09-STS301
17. Glass, G.V. (1966). Testing homogeneity of variances. American Educational Research Journal, 3(3), 187-190. https://doi.org/10.3102/00028312003003187
18. Gökpınar, E. (2022). Standardized likelihood ratio test for homogeneity of variance of several normal populations. Communications in Statistics-Simulation and Computation, 51(11), 6309-6319. https://doi.org/10.1080/03610918.2020.1800037
19. Howell, D.C. (2010). Statistical methods for psychology. PWS-Kent Publishing Co.
20. Katsileros, A., Antonetsis, N., Mouzaidis, P., Tani, E., Bebeli, P.J., & Karagrigoriou, A. (2024). A comparison of tests for homoscedasticity using simulation and empirical data. Communications for Statistical Applications and Methods, 31(1), 1 35. https://doi.org/10.29220/CSAM.2024.31.1.001
21. Keppel, G., & Wickens, T.D. (2004). Design and analysis: A researcher’s handbook (4th ed.). Upper Saddle River, Pearson Prentice Hall.
22. Keskin, S. (2002). Varyansların homojenliğini test etmede kullanılan bazı yöntemlerin I. tip hata ve testin gücü bakımından irdelenmesi [An examination of some methods used in testing the homogeneity of variances in terms of type I error and test power] [Unpublished doctoral dissertation]. Ankara University.
23. Kim, Y.J., & Cribbie, R.A. (2018). ANOVA and the variance homogeneity assumption: Exploring a better gatekeeper. British Journal of Mathematical and Statistical Psychology, 71(1), 1-12. https://doi.org/10.1111/bmsp.12103
24. Kirk, R.E. (2008). Statistics an Introduction (5th ed.). Thomson Wadsworth
25. Moitra, S.D. (1990). Skewness and the beta distribution. Journal of the Operational Research Society, 41(10), 953-961. https://doi.org/10.1057/jors.1990.147
26. Orcan, F. (2020). Parametric or non-parametric: skewness to test normality for mean comparison. International Journal of Assessment Tools in Education, 7(2), 255-265. https://doi.org/10.21449/ijate.656077
27. Öztürk, N.N. (2020). Varyansların homojenliği için bazı testler ve karşılaştırmaları [Some tests for the homogeneity of variances and comparisons] [Unpublished master dissertation]. Gazi University.
28. Park, H.I. (2018). Tests of equality of several variances with the likelihood ratio principle. Communications for Statistical Applications & Methods, 25(4), 329 339. https://doi.org/10.29220/CSAM.2018.25.4.329
29. Roscoe, J.T. (1975). Fundamental Research Statistics for the Behavioural Sciences (2nd ed.). Holt Rinehart & Winston.
30. Sedgwick, P. (2015). A comparison of parametric and non-parametric statistical tests. BMJ, 350. https://doi.org/10.1136/bmj.h2053
31. Shavelson, R.J. (1996). Statistical reasoning for the behavioral sciences (3rd ed.). Pearson Education.
32. Sarısoy, E.E., Potas, N., & Kara, M. (2013). A simulation study goodness-of-fit tests for the skewed normal distribution. In Chaos, Complexity and Leadership 2012 (pp. 277-283). Springer Netherlands.
33. Sarısoy, E.E., Potas, N., Kara, M. (2014). A simulation study goodness-of-fit tests for the skewed normal distribution. In S. Banerjee and Ş. Erçetin (eds). Chaos, Complexity and Leadership 2012 (pp. 277-283). Springer Proceedings in Complexity. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-7362-2_36
34. Tabachnick, B.G., & Fidell, L.S. (2007). Using Multivariate Statistics (5th ed.). Pearson.
35. Wang, Y., Rodríguez de Gil, P., Chen, Y.H., Kromrey, J.D., Kim, E.S., Pham, T., Nguyen, D., & Romano, J.L. (2017). Comparing the Performance of Approaches for Testing the Homogeneity of Variance Assumption in One-Factor ANOVA Models. Educ. Psychol. Meas., 77, 305–329. https://doi.org/10.1177/0013164416645162
36. Woodbury, G. (2002). An Introduction to Statistics: Improving Your Grade. Belmont, CA: Brooks/Cole.
37. Yi, Z., Chen, Y.H., Yin, Y., Cheng, K., Wang, Y., Nguyen, D., … Kim, E. (2020). Brief Research Report: A Comparison of Robust Tests for Homogeneity of Variance in Factorial ANOVA. The Journal of Experimental Education, 90(2), 505 520. https://doi.org/10.1080/00220973.2020.1789833
38. Yonar, A., Yonar, H., Demirsöz, M., & Tekindal, M.A. (2024). A comparative analysis for homogeneity of variance tests. Journal of Science and Arts, 24(2), 305 328. https://doi.org/10.46939/J.Sci.Arts-24.2-a06
39. Zhou, Y., Zhu, Y., & Wong, K.Y. (2023). Statistical tests for homogeneity of variance for clinical trials and recommendations. Contemporary Clinical Trials Communications, 33, 101119. https://doi.org/10.1016/j.conctc.2023.101119