A REVIEW ON SHRINKAGE PARAMETERS IN RIDGE REGRESSION

Year 2017, Volume: 30 Issue: 4, 565 - 582, 11.12.2017

Esra Gökpınar , Meral Ebegil Fikri Gökpınar

Abstract

In the regression
analysis, it is desired that no multicollinearity between the independent
(explanatory) variables exists. In the cases where this is not achieved, the
use of Least Square (LS) estimation method leads to mismodelling. Some methods
have been developed to solve this problem; one of which is the ‘biased
estimation method’. When there exists collinearity, selection of the shrinkage
parameter is important. In this study, a test statistics for Ridge estimator
that is kind of shrinkage biased estimators was investigated. Also the estimators
of shrinkage parameter are compared via simulation. 

Keywords

Ridge Estimator, Mean Square Error, Optimum Ridge Parameter

References

• [1] Alkhamisi, M., Khalaf, G. and Shukur, G., “Some Modifications for Choosing Ridge Parameters”, Communications in Statistics-Theory and Methods, 35:2005-2020, (2006).
• [2] Alkhamisi, M. and Shukur, G., “Developing Ridge Parameters for SUR Model”, Communications in Statistics-Theory and Methods, 37(4):544-564, (2008).
• [3] Dempster, A.P., Schatzoff, M. and Wermuth, N., “A Simulation Study of Alternatives to Ordinary Least Squares”, Journal of the American Statistical Association, 72:77-91, (1977).
• [4] Ebegil, M., Gökpınar, F. and Ekni, M., “A Simulation Study on Some shrinkage Estimators”, Hacettepe Journal of Mathematics and Statistics, 35:213-226, (2006).
• [5] Farebrother, R. W., “A Class of shrinkage Estimators”, Journal of the Royal Statistical Society B, 40:47-49, (1978).
• [6] Gibbons, D.G., “A Simulation Study of Some Ridge Estimators”, Journal of the American Statistical Association, 76:131-139, (1981).
• [7] Hoerl, A.E. and Kennard, R.W., “Ridge regression: biased estimation for non- orthogonalproblems”, Technometrics, 12:55–67, (1970).
• [8] Hoerl, A.E., Kennard, R.W. and Baldwin, K.F., “Ridge regression: some simulation”, Communications in Statistics, 5:105–123, (1975).
• [9] Hocking, R.R., Speed, F.M. and Lynn, M.J., “A Class of Biased Estimators in Linear Regression”, Technometrics, 18:425-438, (1976).
• [10] Johnson, N.L. and Kotz, S., Distributions in Statistics Vol. 1, Wiley, New York, (1972).
• [11] Khalaf, G. and Shukur, G., “Choosing Ridge Parameters for Regression Problems”, Communications in Statistics: Theory and Methods, 34:1177-1182, (2005).
• [12] Kibria, B.M.G., “Performance of Some New Ridge Regression Estimators”, Communications in Statistics: Simulation and Computation, 32:419-435, (2003).
• [13] Lawless, J.F., and Wang, P., “A Simulation Study of Ridge and Other Regression Estimators”, Communications in Statistics: Theory and Methods,7:139-164, (1976).
• [14] Liski, E. P., “A Test of the Mean Square Error Criterion for shrinkage Estimators”, Communications in Statistics: Theory and Methods, 11:543-562, (1982).
• [15] Liski, E.P., “Choosing a shrinkage Estimator-a test of the Mean Square Error Criterion”, Proc. First Tampere Sem. Linear Models, 245-262, (1983).
• [16] Muniz, G. and Kibria, B.M.G., “On Some Ridge Regression Estimators: An Empirical Comparisons”, Communications in Statistics-Simulation and Computation, 38:621-630, (2009).
• [17] Patnaik, P.B., “The Noncentral Chi-Square and F Distributions and their Applications”, Biometrika, 36:202-232, (1949).
• [18] Rao, C. R., “Estimation of Parameters in a Linear Model”, The Annals of Statistics, 4:1023-1037, (1976).
• [19] Saleh A.K. Md. E. and Kibria B.M.G., “Performances of Some New Preliminary Test Ridge Regression Estimators and Their Properties”, Communications in Statistics: Theory and Methods, 22:2747-2764, (1993).
• [20] Saleh A. K. Md. E., Theory of Preliminary Test and Stein-Type Estimation with Applications, Wiley, New York, (2006).
• [21] Searle S. R., Matrix Algebra Useful for Statistic, Wiley, New York, (1982).
• [22] Theobald C.M., “Generalizations of Mean Square Error Applied to Ridge Regression”, Journal of the Royal Statistical Society. Series B (Methodolojical), 36,1:103-106, (1974).
• [23] Zhang J. and Ibrahim M., “A Simulation Study on SPSS Ridge Regression and Ordinary Least Squares Regression Procedures for Multicolliearity data”, Journal of Applied Statistics, 32:571-588, (2005).