Alternative Ridge Parameters in Linear Model

Year 2022, Volume: 4 Issue: 1, 22 - 46, 30.06.2022

Kayode Ayinde , Emmanuel Adewuyi , Lukman Adewale Folaranmi

https://doi.org/10.51541/nicel.1042316

Cited By: 1

Abstract

The ridge regression estimator produces efficient estimates than the Ordinary Least Square Estimator in a linear regression model that has multicollinearity problem. However, the efficiency of the ridge estimator depends on the choice of the ridge parameter, k. This parameter being the biasing parameter that shrinks the coefficient as it tends towards positive infinity needs to be chosen optimally to minimize the mean squared errors of the parameters. In this study, the ridge parameters are classified into different forms, various types and diverse kinds. These classifications resulted into proposing some other techniques of Ridge parameter estimation. Investigation of the existing and proposed ridge parameters were done by conducting Monte-Carlo experiments. Results from simulation study and reallife data application show that some newly proposed ridge parameters are among those that provide efficient estimates.

Keywords

Linear regression, Ridge regression, Biasing parameters, Multicollinearity

Doğrusal Regresyonda Alternatif Ridge Parametreleri

Abstract

Ridge regresyon tahmin edicisi çoklu iç ilişki problemi olan doğrusal regresyon modelinde En Küçük Kareler tahmin edicisinden daha etkin sonuçlar verir. Fakat, Ridge tahmin edicisinin performansı Ridge parametresinin seçimine bağlıdır. Ridge parametreleri farklı türlerde sınıflandırılmaktadır. Bu nedenle Ridge parametrelerinin tahmini için farklı teknikler önerilmiştir. Varolan ve yeni önerilen Ridge parametrelerinin karşılaştırılması için Monte Carlo simülasyon çalışması yapılmıştır. Simülasyon çalışması ve gerçek veri seti sonuçlarına bakıdığında önerilen Ridge parametresi tahmincilerinin etkin sonuçlar verdiği gösterilmiştir.

Keywords

Doğrusal regresyon, Ridge regresyon, Çoklu iç ilişki, Yanlılık parametresi

References

• Ajiboye S. A., Adewuyi, E., Ayinde, K. and Lukman, A. F. (2016), A comparative study of some robust ridge and liu estimators, Science World Journal, 11(4),16-20.
• Alkhamisi, M. and Shukur, G. (2008), Developing ridge parameters for SUR model, Communications in Statistics - Theory and Methods, 37(4), 544-564.
• Alkhamisi, M., Khalaf, G. and Shukur, G. (2006), Some modifications for choosing ridge parameters, Communications in Statistics - Theory and Methods, 35(11), 2005-2020.
• Bhat, S. S. (2016), A comparative study on the performance of new ridge estimators, Pakistan Journal of Statistics and Operation Research, 12(2), 317-325.
• Batach, F., Gore, S. and Verma, M. (2008), Effect of jackknifing on various ridge type estimators, Model Assisted Statistics and Applications, 3, 121-130.
• Daniel, C. and Wood, F. S. (1980), Fitting equations to data: Computer analysis of multifactor data, 2th Edition, Wiley, Newyork.
• Dempster, A. P., Schatzoff, M. and Wermuth, N. (1977), A Simulation study of alternatives to ordinary least squares, Journal of the American Statistical Association, 72(357), 77-91.
• Dorugade, A. V. and Kashid, D. N. (2010), Alternative method for choosing ridge parameter for regression, International Journal of Applied Mathematical Sciences, 4(9), 447-456.
• Faraway, J. (2002), Practical Regression and Anova using R, 109 – 113.
• Fayose, T. S. and Ayinde, K. (2019), Different forms biasing parameter for generalized ridge regression estimator. International Journal of Computer Applications, 181(37), 21-29.
• Ghadban, K. and Mohamed, I. (2014), Ridge regression and Ill-conditioning, Journal of Modern Applied Statistical Methods, 13(2), 355-363.
• Gibbons, D. G. (1981), A Simulation study of some ridge estimators, Journal of the American Statistical Association, 76, 131-139.
• Gorman, J. W. and Toman, R. J. (1966), Selection of variables for fitting equations to data, Technometrics, 8, 27–51.
• Gujarati, D.N. (1995), Basic econometrics, (3rd Edition), Mcgraw-Hill, New York.
• Hald, A. (1952), Statistical theory with engineering applications, Wiley, New York.
• Hamaker, H. C. (1962), On multiple regression analysis, Statistica Neerlandica, 16, 31–56.
• Hoerl, A.E. and Kennard, R.W. (1970), Ridge regression: Biased estimation for non-orthogonal problems, Technometrics, 12, 55-67.
• Longley, J. W. (1967), An appraisal of least-squares programs from the point of view of the user, Journal of the American Statistical Association, 62, 819–841.
• Khalaf, G. (2013), A Comparison between biased and unbiased estimators, Journal of Modern Applied Statistical Methods, 12(2), 293-303.
• Khalaf, G. and Shukur, G. (2005), Choosing ridge parameters for regression problems, Communications in Statistics - Theory and Methods, 34, 1177-1182.
• Kibria, B. M. G. (2003), Performance of some new ridge regression estimators, Communications in Statistics - Simulation and Computation, 32, 419-435.
• Kibria, B. M. G. and Shipra, B. (2016), Some ridge regression estimators and their performances, Journal of Modern Applied Statistical Methods, 15(1), 206-231.
• Lawless, J. F. and Wang, P. (1976), A simulation study of ridge and other regression estimators, Communications in Statistics A, 5, 307-323.
• Lukman A. F. and Ayinde K. (2017), Review and classifications of the ridge parameter estimation techniques, Hacettepe Journal of Mathematics and Statistics, 46(5), 953-967.
• Lukman, A. F (2015), Review and classification of the ridge parameter estimation techniques, Unpublished Thesis. PhD. Thesis, Ladoke Akintola University of Technology, Ogbomoso, Oyo State, Nigeria.
• Mansson, K., Shukur, G. and Kibria, B. M. G. (2010), A simulation study of some ridge regression estimators under different distributional assumptions, Communications in Statistics - Simulations and Computations, 39(8), 1639 –1670.
• McDonald, G. C. and Galarneau, D. I. (1975), A monte carlo evaluation of some ridge-type estimators, Journal of the American Statistical Association, 70, 407-416.
• Muniz, G. and Kibria, B. M. G. (2009), On some ridge regression estimators: An empirical comparison, Communications in Statistics-Simulation and Computation, 38, 621-630.
• Nomura, M. (1988), On the almost unbiased ridge regression estimator, Communications in Statistics–Simulation and Computation, 17, 729–743.
• Wichern, D. and Churchill, G. (1978), A comparison of ridge estimators, Technometrics, 20, 301–311.
• Woods, H., Steinour, H. H. and Starke, H. R. (1932), Effect of composition of Portland cement on heat evolved during hardening, Industrial and Engineering Chemistry, 24, 1207–1214.
• Algamal, Z.Y. (2018), Shrinkage estimators for gamma regression model, Electronic Journal of Applied Statistical Analysis, 11(1), 253-268.
• Algamal, Z.Y. (2018), Shrinkage parameter selection via modified cross-validation approach for ridge regression model, Communications in Statistics - Simulation and Computation, doi: 10.1080/03610918.2018.1508704.
• Algamal, Z.Y. (2018), A new method for choosing the biasing parameter in ridge estimator for generalized linear model, Chemometrics and Intelligent Laboratory Systems, 183, 96-101.