A new ridge type estimator and its performance for the linear regression model: Simulation and application

Abstract

Ridge regression is employed to address the issue of multicollinearity among independent variables. The shrinkage parameter (k) plays a key role in balancing the bias and variance tradeoff. This paper reviewed several promising existing ride regression estimators designed for estimating the ridge or shrinkage parameter k within the Gaussian linear regression model. In addition, we have proposed a new estimator (CK), which is a function of number of independent variables, sample size and standard error of regression model. The performance of our proposed estimator with OLS and existing shrinkage estimators, is compared using extensive Monte Carlo simulations in terms of minimum mean squared error (MSE). Simulation results demonstrated that the proposed CK estimator outperformed other in the majority of the considered simulation scenarios. A real-life data is analyzed to illustrate the findings of the paper.

Keywords

• Linear regression
• MSE
• multicollinearity
• OLS
• ridge regression
• shrinkage parameters

References

1. [1] Z.Y. Algamal, Performance of ridge estimator in inverse Gaussian regression model, Comm. Statist. Theory Methods 48 (15), 3836–3849, 2019.
2. [2] Z.Y. Algamal, Shrinkage parameter selection via modified cross-validation approach for ridge regression model, Comm. Statist. Simulation Comput. 49 (7), 1922–1930, 2020.
3. [3] S. Ali, H. Khan, I. Shah, M.M. Butt and M. Suhail, A comparison of some new and old robust ridge regression estimators, Comm. Statist. Simulation Comput. 50 (8), 2213–2231, 2021.
4. [4] I. Dar, S. Chand, M. Shabbir and B.M.G. Kibria, Condition-index based new ridge regression estimator for linear regression model with multicollinearity, Kuwait J. Sci. 50 (2), 91-96, 2023.
5. [5] A. Dorugade, Improved ridge estimator in linear regression with multicollinearity, heteroscedastic errors and outliers, J. Mod. Appl. Stat. Methods 15 (2), 362–381, 2016.
6. [6] D.N. Gujarati, Basic Econometrics, Mc Graw-Hill International Edition, New York, 2009.
7. [7] A.E. Hoerl and R.W. Kennard, Ridge regression: biased estimation for nonorthogonal problems, Technometrics 12 (1), 55–67, 1970.
8. [8] A. Karakoca, A new type iterative ridge estimator: applications and performance evaluations, J. Math., Doi: 10.1155/2022/3781655, 2022.

9. [9] G. Khalaf, Improving the ordinary least squares estimator by ridge regression, OA Library 9 (5), 1–8, 2022.
10. [10] G. Khalaf, K. Mansson and G. Shukur, Modified ridge regression estimators, Comm. Statist. Theory Methods 42, (8), 1476–1487, 2013.
11. [11] B.M.G. Kibria, Performance of some new ridge regression estimators, Comm. Statist. Simulation Comput. 32 (2), 419-435, 2003.
12. [12] B.M.G. Kibria, More than hundred (100) estimators for estimating the shrinkage parameter in a linear and generalized linear ridge regression models, Journal of Econometrics and Statistics 2 (2), 233–252, 2022.
13. [13] B.M.G. Kibria and S. Banik, Some ridge regression estimators and their performances, JMASM 15 (1), 206-238, 2016.
14. [14] B.M.G. Kibria and A. Lukman, A new ridge-type estimator for the linear regression model: Simulations and applications, Scientifica, Doi: 10.1155/2020/9758378, 2020.
15. [15] A.F. Lukman and A. Olatunji, Newly proposed estimator for ridge parameter: An application to the Nigerian economy, Pakistan J. Statist. 32 (2), 91–98, 2018.
16. [16] G.C. McDonald and D.I. Galarneau, A Monte Carlo evaluation of some ridge-type estimators, J. Amer. Statist. Assoc. 70 (350), 407–416, 1975.
17. [17] G. Muniz and B.M.G. Kibria, On some ridge regression estimators: An empirical comparison, Comm. Statist. Simulation Comput. 38 (3), 621–630, 2009.
18. [18] M. Shabbir, S. Chand, and F. Iqbal, Bagging-based ridge estimators for a linear regression model with non-normal and heteroscedastic errors, Comm. Statist. Simulation Comput., Doi: 10.1080/03610918.2022.2109675, 2022.
19. [19] C.M. Stein, Multiple regression contributions to probability and statistics, in: Essays in Honor of Harold Hoteling 103, Stanford University Press, 1960.
20. [20] M. Suhail, S. Chand and B.M.G. Kibria, Quantile based estimation of biasing parameters in ridge regression model, Comm. Statist. Simulation Comput. 49 (10), 2732–2744, 2020.
21. [21] M. Suhail, S. Chand and B.M.G. Kibria, Quantile-based robust ridge M-estimator for linear regression model in presence of multicollinearity and outliers, Comm. Statist. Simulation Comput. 50 (11), 3194-3205, 2021.
22. [22] A. Yasin, A. Karaibrahimoglu and G. Asir, Modified ridge regression parameters: A comparative Monte Carlo study, Hacet. J. Math. Stat. 43 (5), 827–841, 2014.
23. [23] M.A. Zubair and M.O. Adenomon, Comparison of estimators efficiency for linear regressions with joint presence of autocorrelation and multicollinearity, Sci. World J. 16 (2), 103–109, 2021.