Identification of Leverage Points in Principal Component Regression and r-k Class Estimators with AR(1) Error Structure

Year 2020, Volume: 6 Issue: 2, 353 - 363, 29.12.2020

Tuğba Söküt

https://doi.org/10.28979/jarnas.845208

Abstract

The determination of leverage observations have been frequently investigated through ordinary least squares and some biased estimators proposed under the multicollinearity problem in the linear regression models. Recently, the identification of leverage and influential observations have been also popular on the general linear regression models with correlated error structure. This paper proposes a new projection matrix and a new quasiprojection matrix to determination of leverage observations for principal component regression and r-k class estimators, respectively, in general linear regression model with first-order autoregressive error structure. Some useful properties of these matrices are presented. Leverage observations obtained by generalized least squares and ridge regression estimators available in the literature have been compared with proposed principal component regression and r-k class estimators over a simulation study and a numerical example. In the literature, the first leverage is considered separately due to the first-order autoregressive error structure. Therefore, the behaviours of first leverages obtained by principal component regression and r-k class estimators has been also investigated according to the autocorrelation coefficient and biasing parameter through applications. The results showed that the leverage of the first observation obtained by principal component regression and r-k estimators is smaller than that obtained by generalized least squares and ridge regression estimators. In addition, as the autocorrelation coefficient goes to -1, the leverage of the first transformed observation decreases for PCR and r-k class estimators, while its increases while the autocorrelation coefficient goes to 1.

Keywords

Autocorrelation, first-order autoregressive error, leverages, multicollinearity, biased estimators

AR (1) Hata Yapısı ile Temel Bileşen Regresyon ve r-k Sınıf Tahmincilerinde Kaldıraç Noktalarının Belirlenmesi

Abstract

Lineer regresyon modellerinde kaldıraç gözlemlerin belirlenmesi sıklıkla sıradan en küçük kareler ve bazı yanlı tahmin ediciler üzerinden araştırılmıştır. Son zamanlarda kaldıraç ve etkin gözlemlerin belirlenmesi genel lineer regresyon modellerinde de popüler olmuştur. Bu çalışmada birinci dereceden otoregresif hata yapısına sahip genel lineer regresyon modelinde temel bileşenler regresyon ve r-k sınıf tahmin edicileri için sırasıyla yeni bir projeksiyon matrisi ve bir yarı projeksiyon matrisi önerilmektedir. Bu matrislerin bazı yararlı özellikleri sunulmuştur. Literatürde bulunan genelleştirilmiş en küçük kareler ve ridge regresyon tahmin edicilerinden elde edilen kaldıraç gözlemleri, bir simülasyon çalışması ve sayısal bir örnek üzerinden önerilen temel bileşen regresyonu ve r-k sınıfı tahmin edicileriyle karşılaştırılmıştır. Literatürde birinci dereceden otoregresif hata yapısı nedeniyle birinci kaldıraç ayrı olarak ele alınmaktadır. Bu nedenle, temel bileşen regresyonu ve r-k sınıfı tahmin edicileriyle elde edilen ilk kaldıraçların davranışları, uygulamalar aracılığıyla otokorelasyon katsayısına ve önyargı parametresine göre de incelenmiştir. Sonuçlar, temel bileşen regresyonu ve r-k tahmin edicileriyle elde edilen ilk gözlemin kaldıraçlığının, genelleştirilmiş en küçük kareler ve sırt regresyon tahmin edicileriyle elde edilenden daha küçük olduğunu göstermiştir.

Keywords

Otokorelasyon, birinci dereceden otoregresif hata, kaldıraçlar, çoklu iç ilişki, yanlı tahmin ediciler

References

• Ac¸ar, T.S., & O¨ zkale M.R. (2016). Influence measures in ridge regression when the error terms follow an Ar(1) process. Computational Statistics, 31(3), 879-898. https://doi.org/10.1007/s00180-015-0615-5
• Aitken, A.C. (1935). IV.— On least square and linear combinations of observations. Proceedings of Royal Statistical Society, 55, 42-48. https://doi.org/10.1017/S0370164600014346 Cook, R.D., & Weisberg, S.(1982). Residuals and influence in regression. Chapman and Hall, New York, pp. 11.
• Dodge, Y., & Hadi, A.S. (2010). Simple graphs and bounds for the elements of the hat matrix. Journal of Applied Statistics, 26(7), 817-823. https://doi.org/10.1080/02664769922052
• Durbin, J., & Watson, G.S. (1950). Testing for serial correlation in least squares regression I, Biometrica, 37(3/4), 409-428. https://doi.org/10.2307/2332391
• Gujarati, D.N. (2004). Basic Econometrics, 4th ed., McGraw-Hill, New Jersey.
• Hoerl, A.E., & Kennard, R.W. (1970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67. https://doi.org/10.1080/00401706.1970.10488634
• Hoerl, A.E., Kennard, R.W., & Baldwin, K.F. (1975). Ridge regression: some simulation. Communications in Statistics, 4(2), 105-123. https://doi.org/10.1080/03610927508827232
• Judge, G.G., Grifths, W.E., Hill, R.C., L¨utkepohl, H., & Lee, T.C. (1985). The Theory and Practice of Econometrics, 2nd ed. John Wiley & Sons Inc, New York.
• Kibria, B.M.G. (2003). Performance of some new ridge regression estimators. Communications in Statistics- Simulation and Computation, 32(2), 419-435. https://doi.org/10.1081/SAC-120017499
• Liu, K. (1993). A new class of biased estimate in linear regression. Communications in Statistics-Theory and Methods, 22(2), 393-402. https://doi.org/10.1080/03610929308831027
• Marquardt, D.W. (1970). Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation. Technometrics, 12(3), 591–612. https://doi.org/10.2307/1267205
• McDonald, G.C.,& Galarneau, D.I. (1975). A Monte Carlo evaluation of some ridge type estimators. Journal of the American Statistical Association, 70(350), 407–416. https://doi.org/10.2307/2285832
• Montgomery, D.C., Peck, E.A., & Vining, G.G. (2001). Introduction to Linear Regression Analysis. JohnWiley & Sons, New York.
• Myers, R.H. (1990). Classical and Modern Regression with Applications. Duxbury Press, California, 1990.
• Özkale, M.R., & Ac¸ar, T.S. (2015). Leverages and influential observations in regression model with autocorrelated errors. Communications in Statistics -Theory and Methods, 44(11), 2267-2290. https:// doi.org/10.1080/03610926.2013.781646
• Puterman, M.L. (1988). Leverage and influence in autocorrelated regression models. Journal of the Royal Statistical Society, 37(1), 76-86. https://doi.org/10.2307/2347495
• Roy, S.S., & Guria, S. (2004). Regression diagnostics in an autocorrelated model. Brazilian Journal of Probability and Statistics, 18(2), 103-112.
• Steece, B.M. (1986). Regresor space outliers in ridge regression. Communications in Statistics-Theory and Methods, 15(12), 3599-3605. https://doi.org/10.1080/03610928608829333
• Stemann, D., & Trenkler, G. (1993). Leverage and cochrane-orcutt estimation in linear regression. Communications in Statistics-Theory and Methods, 22(5), 1315-1333. https://doi.org/10.1080/ 03610929308831088
• Şıray, G.Ü., Kac¸ıranlar, S., & Sakallıog˘lu, S. (2014). r -k class estimator in linear regression model with correlated errors. Statistical Papers, 55(2), 393–407. https://doi.org/10.1007/s00362-012-0484 -8
• Trenkler, G. (1984). On the performance of biased estimators in the linear regression model with correlated or heteroscedastic errors. Journal of Econometrics, 25(1/2), 179-190. https://doi.org/10.1016/ 0304-4076(84)90045-9
• Tripp, R.E. (1983). Nonstochastic ridge regression and effective rank of the regressors matrix, unpublished Ph.D. dissertation, Virginia Polytechnic Institute and State University, Dept. of Statistics.
• Walker, E. & Birch, J.B. (1988). Influence measures in ridge regression. Technometrics, 30(2), 221-227. https://doi.org/10.2307/1270168