A REVIEW ON SHRINKAGE PARAMETERS IN RIDGE REGRESSION

Abstract

In the regression analysis, it is desired that no multicollinearity between the independent (explanatory) variables exists. In the cases where this is not achieved, the use of Least Square (LS) estimation method leads to mismodelling. Some methods have been developed to solve this problem; one of which is the ‘biased estimation method’. When there exists collinearity, selection of the shrinkage parameter is important. In this study, a test statistics for Ridge estimator that is kind of shrinkage biased estimators was investigated. Also the estimators of shrinkage parameter are compared via simulation. 

Keywords

• Ridge Estimator,Mean Square Error,Optimum Ridge Parameter

References

1. [1] Alkhamisi, M., Khalaf, G. and Shukur, G., “Some Modifications for Choosing Ridge Parameters”, Communications in Statistics-Theory and Methods, 35:2005-2020, (2006).
2. [2] Alkhamisi, M. and Shukur, G., “Developing Ridge Parameters for SUR Model”, Communications in Statistics-Theory and Methods, 37(4):544-564, (2008).
3. [3] Dempster, A.P., Schatzoff, M. and Wermuth, N., “A Simulation Study of Alternatives to Ordinary Least Squares”, Journal of the American Statistical Association, 72:77-91, (1977).
4. [4] Ebegil, M., Gökpınar, F. and Ekni, M., “A Simulation Study on Some shrinkage Estimators”, Hacettepe Journal of Mathematics and Statistics, 35:213-226, (2006).
5. [5] Farebrother, R. W., “A Class of shrinkage Estimators”, Journal of the Royal Statistical Society B, 40:47-49, (1978).
6. [6] Gibbons, D.G., “A Simulation Study of Some Ridge Estimators”, Journal of the American Statistical Association, 76:131-139, (1981).
7. [7] Hoerl, A.E. and Kennard, R.W., “Ridge regression: biased estimation for non- orthogonalproblems”, Technometrics, 12:55–67, (1970).
8. [8] Hoerl, A.E., Kennard, R.W. and Baldwin, K.F., “Ridge regression: some simulation”, Communications in Statistics, 5:105–123, (1975).

9. [9] Hocking, R.R., Speed, F.M. and Lynn, M.J., “A Class of Biased Estimators in Linear Regression”, Technometrics, 18:425-438, (1976).
10. [10] Johnson, N.L. and Kotz, S., Distributions in Statistics Vol. 1, Wiley, New York, (1972).
11. [11] Khalaf, G. and Shukur, G., “Choosing Ridge Parameters for Regression Problems”, Communications in Statistics: Theory and Methods, 34:1177-1182, (2005).
12. [12] Kibria, B.M.G., “Performance of Some New Ridge Regression Estimators”, Communications in Statistics: Simulation and Computation, 32:419-435, (2003).
13. [13] Lawless, J.F., and Wang, P., “A Simulation Study of Ridge and Other Regression Estimators”, Communications in Statistics: Theory and Methods,7:139-164, (1976).
14. [14] Liski, E. P., “A Test of the Mean Square Error Criterion for shrinkage Estimators”, Communications in Statistics: Theory and Methods, 11:543-562, (1982).
15. [15] Liski, E.P., “Choosing a shrinkage Estimator-a test of the Mean Square Error Criterion”, Proc. First Tampere Sem. Linear Models, 245-262, (1983).
16. [16] Muniz, G. and Kibria, B.M.G., “On Some Ridge Regression Estimators: An Empirical Comparisons”, Communications in Statistics-Simulation and Computation, 38:621-630, (2009).
17. [17] Patnaik, P.B., “The Noncentral Chi-Square and F Distributions and their Applications”, Biometrika, 36:202-232, (1949).
18. [18] Rao, C. R., “Estimation of Parameters in a Linear Model”, The Annals of Statistics, 4:1023-1037, (1976).
19. [19] Saleh A.K. Md. E. and Kibria B.M.G., “Performances of Some New Preliminary Test Ridge Regression Estimators and Their Properties”, Communications in Statistics: Theory and Methods, 22:2747-2764, (1993).
20. [20] Saleh A. K. Md. E., Theory of Preliminary Test and Stein-Type Estimation with Applications, Wiley, New York, (2006).
21. [21] Searle S. R., Matrix Algebra Useful for Statistic, Wiley, New York, (1982).
22. [22] Theobald C.M., “Generalizations of Mean Square Error Applied to Ridge Regression”, Journal of the Royal Statistical Society. Series B (Methodolojical), 36,1:103-106, (1974).
23. [23] Zhang J. and Ibrahim M., “A Simulation Study on SPSS Ridge Regression and Ordinary Least Squares Regression Procedures for Multicolliearity data”, Journal of Applied Statistics, 32:571-588, (2005).