Two-way ANOVA by using Cholesky decomposition and graphical representation

Abstract

In general, the coefficient estimates of linear models are carried out using the ordinary least squares (OLS) method. Since the analysis of variance is also a linear model, the coefficients can be estimated using the least-squares method. In this study, the coefficient estimates in the two-way analysis of variance were performed by using the Cholesky decomposition. The purpose of using the Cholesky decomposition in finding coefficient estimates make variables used in model being orthogonal such that important variables can be easily identified. The sum of squares in two-way analysis of variance (row, column, interaction) were also found by using the coefficient estimates obtained as a result of the Cholesky decomposition. Thus, important variables that affect the sum of squares can be determined more easily because the Cholesky decomposition makes the variables in the model orthogonal. By representing the sum of squares with vectors, how the prediction vector in two-way ANOVA model was created was shown. It was mentioned how the Cholesky decomposition affected the sum of squares. This method was explained in detail on a sample data and shown geometrically.

Keywords

• Cholesky decomposition
• aalysis of variance and covariance (ANOVA)
• linear equations

References

1. [1] D. Bradu and K.R. Gabriel, Simultaneous statistical inference on interactions in twoway analysis of variance, J. Amer. Statist. Assoc. 69 (346), 428-436, 1974.
2. [2] J.A. Cano, C. Carazo and D. Salmerón, Objective Bayesian model selection approach to the two way analysis of variance, Comput. Stat. 33 (1), 235-248, 2018.
3. [3] A. Cichocki, D. Mandic, L. De Lathauwer, G. Zhou, Q. Zhao, C. Caiafa and H.A. Phan, Tensor decompositions for signal processing applications from two-way to multiway component analysis, IEEE Signal Process. Mag. 32 (2), 145-163, 2015.
4. [4] R.B. Darlington and A.F. Hayes, Regression Analysis and Linear Models Concepts, Applications, and Implementation, The Guilford Press, New York, 2017.
5. [5] C.R. Dietrich, Computationally efficient Cholesky factorization of a covariance matrix with block toeplitz structure, J. Stat. Comput. Simul. 45 (3-4), 203-218, 1993.
6. [6] J.D. Finn, A General Model for Multivariate Snalysis, Rinehart and Winston, New York: Holt, 1974.
7. [7] J. Fox, Applied Regression Analysis and Generalized Linear Models, SAGE, Los Angeles, 2016.
8. [8] L. Freitag, S. Knecht, C. Angeli and M. Reiher, Multireference perturbation theory with Cholesky decomposition for the density matrix renormalization group, J. Chem. Theory Comput. 13 (2), 451-459, 2017.

9. [9] W.H. Greene, Econometric Analysis, EUA: Prentice-Hall, Upper Saddle River, 2003.
10. [10] R.A. Horn and C.R. Johnson, Matrix Analysis, Cambridge University Press, New York, 2018.
11. [11] X. Kang and X. Deng, On variable ordination of Cholesky-based estimation for a sparse covariance matrix, Canad. J. Statist. 49 (2), 2020.
12. [12] P. Kohli, T.P. Garcia and M. Pourahmadi, Modeling the Cholesky factors of covariance matrices of multivariate longitudinal data, J. Multivariate Anal. 145, 87-100, 2016.
13. [13] A. Krishnamoorthy and D. Menon, Matrix inversion using Cholesky decomposition, Signal Processing: Algorithms, Architectures, Arrangements, and Applications, 70- 72, 2013.
14. [14] R.S. Krutchkoff, Two-way fixed effects analysis of variance when the error variances may be unequal, J. Stat. Comput. Simul. 32 (3), 177-183, 1989.
15. [15] K. Lee, C. Baek and M.J. Daniels, ARMA Cholesky factor models for the covariance matrix of linear models, Comput. Statist. Data Anal. 115, 267-280, 2017.
16. [16] K. Lee and J.K. Yoo, Bayesian Cholesky factor models in random effects covariance matrix for generalized linear mixed models, Comput. Statist. Data Anal. 80, 111-116, 2014.
17. [17] R.I.M. Lira, A.A. Trindade and V. Howle, QR versus Cholesky: A probabilistic analysis, Int. J. Numer. Anal. Model. 13 (1), 114-121, 2016.
18. [18] J.H. Maindonald, Least squares computations based on the Cholesky decomposition of the correlation matrix, J. Stat. Comput. Simul. 5 (4), 247-258, 1977.
19. [19] M. Pourahmadi, Cholesky decompositions and estimation of a covariance matrix: orthogonality of variancecorrelation parameters, Biometrika 94 (4), 1006-1013, 2007.
20. [20] A.C. Rencher and G.B. Schaalje, Linear Models in Statistics, Wiley-Interscience, Hoboken, 2008.
21. [21] S.R. Searle, Linear Models, Wiley, New York, 1971.
22. [22] I. Sumiati, F. Handoyo and S. Purwani, Multiple linear regression using Cholesky decomposition, World Sci. News 140, 12-25, 2020.
23. [23] M. Tekin and H. Ekelik, Linear regression approach to analysis of cariance (ANOVA) with the Cholesky decomposition and excel application, Ankara Haci Bayram Veli University Journal of the Faculty of Economics and Administrative Sciences 20, 58- 77, 2020.
24. [24] T. Terao, K. Ozaki and T. Ogita, LU-Cholesky QR algorithms for thin QR decomposition, Parallel Comput. 92, 102571, 2020.
25. [25] T.D. Wickens, The Geometry of Multivariate Statistics, Pychology Press, New York, 2014.
26. [26] G. Younis, Practical method to solve large least squares problems using Cholesky decomposition, Geod. Cartogr. 41 (3), 113-118, 2015.