Identification of Leverage Points in Principal Component Regression and r-k Class Estimators with AR(1) Error Structure

Abstract

The determination of leverage observations have been frequently investigated through ordinary least squares and some biased estimators proposed under the multicollinearity problem in the linear regression models. Recently, the identification of leverage and influential observations have been also popular on the general linear regression models with correlated error structure. This paper proposes a new projection matrix and a new quasiprojection matrix to determination of leverage observations for principal component regression and r-k class estimators, respectively, in general linear regression model with first-order autoregressive error structure. Some useful properties of these matrices are presented. Leverage observations obtained by generalized least squares and ridge regression estimators available in the literature have been compared with proposed principal component regression and r-k class estimators over a simulation study and a numerical example. In the literature, the first leverage is considered separately due to the first-order autoregressive error structure. Therefore, the behaviours of first leverages obtained by principal component regression and r-k class estimators has been also investigated according to the autocorrelation coefficient and biasing parameter through applications. The results showed that the leverage of the first observation obtained by principal component regression and r-k estimators is smaller than that obtained by generalized least squares and ridge regression estimators. In addition, as the autocorrelation coefficient goes to -1, the leverage of the first transformed observation decreases for PCR and r-k class estimators, while its increases while the autocorrelation coefficient goes to 1.

Keywords

• Autocorrelation
• first-order autoregressive error
• leverages
• multicollinearity
• biased estimators

AR (1) Hata Yapısı ile Temel Bileşen Regresyon ve r-k Sınıf Tahmincilerinde Kaldıraç Noktalarının Belirlenmesi

Abstract

Lineer regresyon modellerinde kaldıraç gözlemlerin belirlenmesi sıklıkla sıradan en küçük kareler ve bazı yanlı tahmin ediciler üzerinden araştırılmıştır. Son zamanlarda kaldıraç ve etkin gözlemlerin belirlenmesi genel lineer regresyon modellerinde de popüler olmuştur. Bu çalışmada birinci dereceden otoregresif hata yapısına sahip genel lineer regresyon modelinde temel bileşenler regresyon ve r-k sınıf tahmin edicileri için sırasıyla yeni bir projeksiyon matrisi ve bir yarı projeksiyon matrisi önerilmektedir. Bu matrislerin bazı yararlı özellikleri sunulmuştur. Literatürde bulunan genelleştirilmiş en küçük kareler ve ridge regresyon tahmin edicilerinden elde edilen kaldıraç gözlemleri, bir simülasyon çalışması ve sayısal bir örnek üzerinden önerilen temel bileşen regresyonu ve r-k sınıfı tahmin edicileriyle karşılaştırılmıştır. Literatürde birinci dereceden otoregresif hata yapısı nedeniyle birinci kaldıraç ayrı olarak ele alınmaktadır. Bu nedenle, temel bileşen regresyonu ve r-k sınıfı tahmin edicileriyle elde edilen ilk kaldıraçların davranışları, uygulamalar aracılığıyla otokorelasyon katsayısına ve önyargı parametresine göre de incelenmiştir. Sonuçlar, temel bileşen regresyonu ve r-k tahmin edicileriyle elde edilen ilk gözlemin kaldıraçlığının, genelleştirilmiş en küçük kareler ve sırt regresyon tahmin edicileriyle elde edilenden daha küçük olduğunu göstermiştir.

Keywords

• Otokorelasyon
• birinci dereceden otoregresif hata
• kaldıraçlar
• çoklu iç ilişki
• yanlı tahmin ediciler

References

1. Açar, T.S., & Özkale M.R. (2016). Influence measures in ridge regression when the error terms follow an Ar(1) process. Computational Statistics, 31(3), 879-898. doi: 0.1007/s00180-015-0615-5
2. Aitken, A.C. (1935). IV.— On least square and linear combinations of observations. Proceedings of Royal Statistical Society, 55, 42-48. doi: 10.1017/S0370164600014346
3. Cook, R.D., & Weisberg, S.(1982). Residuals and influence in regression. Chapman and Hall, New York, pp. 11.
4. Dodge, Y., & Hadi, A.S. (2010). Simple graphs and bounds for the elements of the hat matrix. Journal of Applied Statistics,26(7), 817-823. doi:10.1080/02664769922052
5. Durbin, J., & Watson, G.S. (1950). Testing for serial correlation in least squares regression I, Biometrica, 37(3/4), 409-428. doi: 10.2307/2332391
6. Gujarati, D.N. (2004). Basic Econometrics, 4th ed., McGraw-Hill, New Jersey.
7. Hoerl, A.E., & Kennard, R.W. (1970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67. doi: 10.1080/00401706.1970.10488634
8. Hoerl, A.E., Kennard, R.W., & Baldwin, K.F. (1975). Ridge regression: some simulation. Communications in Statistics, 4(2), 105-123. doi: 10.1080/03610927508827232

9. Judge, G.G., Grifths, W.E., Hill, R.C., L¨utkepohl, H., & Lee, T.C. (1985). The Theory and Practice of Econometrics, 2nd ed. John Wiley & Sons Inc, New York.
10. Kibria, B.M.G. (2003). Performance of some new ridge regression estimators. Communications in Statistics-Simulation and Computation, 32(2) 419-435. doi: 10.1081/SAC-120017499
11. Liu, K. (1993). A new class of biased estimate in linear regression. Communications in Statistics-Theory and Methods, 22(2), 393-402. doi: 10.1080/03610929308831027
12. Marquardt, D.W. (1970). Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation. Technometrics, 12(3), 591–612. doi: 10.2307/1267205
13. McDonald, G.C.,& Galarneau, D.I. (1975). A Monte Carlo evaluation of some ridge type estimators. Journal of the American Statistical Association, 70(350), 407–416. doi: 10.2307/2285832
14. Montgomery, D.C., Peck, E.A., & Vining, G.G. (2001). Introduction to Linear Regression Analysis. John Wiley & Sons, New York.
15. Myers, R.H. (1990). Classical and Modern Regression with Applications. Duxbury Press, California, 1990.
16. Özkale, M.R., & Açar, T.S. (2015). Leverages and influential observations in regression model with autocorrelated errors. Communications in Statistics -Theory and Methods, 44(11), 2267-2290. doi: 10.1080/03610926.2013.781646
17. Puterman, M.L. (1988). Leverage and influence in autocorrelated regression models. Journal of the Royal Statistical Society, 37(1), 76-86. doi: 10.2307/2347495
18. Roy, S.S., & Guria, S. (2004). Regression diagnostics in an autocorrelated model. Brazilian Journal of Probability and Statistics, 18(2), 103-112.
19. Steece, B.M. (1986). Regresor space outliers in ridge regression. Communications in Statistics-Theory and Methods, 15(12), 3599-3605. doi: 10.1080/03610928608829333
20. Stemann, D., & Trenkler, G. (1993). Leverage and cochrane-orcutt estimation in linear regression. Communications in Statistics-Theory and Methods, 22(5), 1315-1333. doi: 10.1080/03610929308831088
21. Şiray, G.Ü ., Kaçıranlar, S., & Sakallıoğlu, S. (2014). r-k class estimator in linear regression model with correlated errors. Statistical Papers, 55(2), 393–407. doi: 10.1007/s00362-012-0484-8
22. Trenkler, G. (1984). On the performance of biased estimators in the linear regression model with correlated or heteroscedastic errors. Journal of Econometrics, 25(1/2), 179-190. doi: 10.1016/0304-4076(84)90045-9
23. Tripp, R.E. (1983). Nonstochastic ridge Rregression and effective rank of the regressors matrix, unpublished Ph.D. dissertation, Virginia Polytechnic Institute and State University, Dept. of Statistics.
24. Walker, E. & Birch, J.B. (1988). Influence measures in ridge regression. Technometrics, 30(2), 221-227. doi: 10.2307/1270168