Review and classications of the ridge parameter estimation techniques

Abstract

Ridge parameter estimation techniques under the inuence of multi-collinearity in Linear regression model were reviewed and classified into
different forms and various types. The different forms are Fixed Maximum (FM), Varying Maximum (VM), Arithmetic Mean (AM), Geometric Mean (GM), Harmonic Mean (HM) and Median (M) and the various types are Original (O), Reciprocal (R), Square Root (SR) and Reciprocal of Square Root (RSR). These classications resulted into proposing some other techniques of Ridge parameter estimation. Investigation of the existing and proposed ones were done by conducting 1000 Monte-Carlo experiments under five (5) levels of multicollinearity ( $\rho=0.8, 0.9, 0.95, 0.99, 0.999$), three (3) levels of error variance ($\sigma^2=0.25,1,25$) and five levels of sample size ($n=10,20,30,40,50$). The relative efficiency ($RF\leq 0.75$) of the techniques resulting from the ratio of their mean square error and that of the ordinary least square was used to compare the techniques. 

Results show that the proposed techniques perform better than the existing ones in some situations; and that the best technique is generally
the ridge parameter in the form of Harmonic Mean, Fixed Maximum and Varying Maximum in their Original and Square Root types.

Keywords

• Linear Regression Model,Multicollinearity,Ridge Parameter Estimation Techniques,Relative Efficiency

References

1. Alkhamisi, M., Khalaf, G. and Shukur, G. (2006). Some modications for choosing ridge parameters.Communications in Statistics- Theory and Methods, 35(11), 2005-2020.
2. Gibbons, D. G. (1981). A simulation study of some ridge estimators. Journal of the American Statistical Association, 76, 131-139.
3. Gujarati, D.N.(1995). Basic Econometrics, McGraw-Hill, New York. Hoerl, A.E. and Kennard, R.W. (1970). Ridge regression: biased estimation for non- orthogonal problems. Technometrics, 12, 55-67.
4. Hoerl, A. E., Kennard, R. W. and Baldwin, K. F. (1975). Ridge regression: Some simulation. Communications in Statistics 4 (2), 105123.
5. Khalaf, G. and Shukur, G. (2005). Choosing ridge parameters for regression problems. Com- munications in Statistics- Theory and Methods, 34, 1177-1182.
6. Kibria, B. M. G. (2003). Performance of some new ridge regression estimators. Communi- cations in Statistics-Simulation and Computation, 32, 419-435.
7. Lawless, J. F. and Wang, P. (1976). A simulation study of ridge and other regression esti- mators. Communications in Statistics A, 5, 307-323.
8. Lukman, A. F (2015): Review and classication of the Ridge Parameter Estimation Tech- niques. Ladoke Akintola University of Technology, Ogbomoso, Oyo State, Nigeria. Unpub- lished P.hD. Thesis.

9. Mansson, K., Shukur, G. and Kibria, B. M. G. (2010). A simulation study of some ridge regression estimators under dierent distributional assumptions. Communications in Statistics-Simulations and Computations, 39(8), 1639 1670.
10. Mardikyan, S. and Cetin, E. (2008). Ecient Choice of Biasing Constant for Ridge Regres- sion, Int.J. Contemp.Math.Sciences, 3, 527-547.
11. McDonald, G. C. and Galarneau, D. I. (1975). A Monte Carlo evaluation of some ridge-type estimators. Journal of the American Statistical Association, 70, 407-416.
12. Muniz, G. and Kibria, B. M. G. (2009). On some ridge regression estimators: An empirical comparison. Communications in Statistics-Simulation and Computation, 38, 621-630.
13. Muniz, G., Kibria, B.M.G., Mansson, K., Shukur, G. (2012). On Developing Ridge Regres- sion Parameters: A Graphical Investigation. SORT. 36(2), 115-138.
14. Saleh, A. K. Md. E. and Kibria, B. M. G. (1993). Performances of some new preliminary test ridge regression estimators and their properties. Communications in Statistics-Theory and Methods, 22, 2747-2764.
15. Vinod, D. and Ullah, A. (1981). Recent Advances in Regression Methods, Marcel Dekker Inc. Publication.
16. Wichern, D. and Churchill, G. (1978). A Comparison of Ridge Estimators. Technometrics, 20, 301311.