Coverage-based performance of confidence intervals for linear regression coefficients under multicollinearity: Simulation and application

Abstract

The shrinkage parameters in the ridge regression model have been extensively discussed and compared in the literature. Typically, the mean square error is used as the primary criterion for comparison. However, it does not fully explain the inferential performance of the estimator. This paper aims to examine 18 ridge regression regularization parameters based on their coverage probability and confidence interval widths using a simulation approach under various conditions. The results reveal that even though most estimators exhibit narrower confidence intervals compared to ordinary least squares, the shrinkage parameters that demonstrate a lower mean square error do not consistently maintain a coverage probability of 95. Additionally, increasing collinearity widens the width of the confidence interval. This paper studies the impact of multicollinearity on confidence interval coverage in linear regression models and provides information for researchers interested in inference based on confidence intervals.

Keywords

• Confidence interval
• coverage probability
• multicollinearity
• multiple linear regression
• ridge regression

References

1. [1] M. M. Al-Kassab and M. Q. Al-Awjar, A Monte Carlo comparison between least squares and the new ridge regression parameters, J. Adv. Appl. Stat. 62 (1), 97-105, 2020.
2. [2] M. Alkhamisi, G. Khalaf and G. Shukur, Some modifications for choosing ridge parameters, Commun. Stat. Theory Methods 35 (11), 2005-2020, 2006.
3. [3] Y. Asar, A. Karaibrahimoglu and A. Genç, Modified ridge regression parameters: a comparative Monte Carlo study, Hacet. J. Math. Stat. 43 (5), 827-841, 2014.
4. [4] R. A. Bottenberg and H. W. Joe, Applied multiple linear regression, 6570th Personnel Research Laboratory, Aerospace Medical Division, Air Force Systems Command, Lackland Air Force Base, 1963.
5. [5] S. Chand and B. M. G. Kibria, A new ridge-type estimator and its performance for the linear regression model: simulation and application, Hacet. J. Math. Stat. 1-14, 2024.
6. [6] Y. P. Chaubey, M. Khurana and S. Chandra, Confidence intervals based on resampling methods using ridge estimator in linear regression model, New Trends Math. Sci. 6 (4), 2018.
7. [7] A. Crivelli, L. Firinguetti, R. Montano and M. Munóz, Confidence intervals in ridge regression by bootstrapping the dependent variable: a simulation study, Commun. Stat. Simul. Comput. 24 (3), 631-652, 1995.
8. [8] E. Cule, P. Vineis and M. De Iorio, Significance testing in ridge regression for genetic data, BMC Bioinformatics 12, 1-15, 2011.

9. [9] A. V. Dorugade, New ridge parameters for ridge regression, J. Assoc. Arab Univ. Basic Appl. Sci. 15, 94-99, 2014.
10. [10] R. Frisch, Statistical confluence analysis by means of complete regression systems, 1934.
11. [11] M. J. Gardner and D. G. Altman, Confidence intervals rather than p values: estimation rather than hypothesis testing, Br. Med. J. (Clin. Res. Ed.) 292 (6522), 746-750, 1986.
12. [12] E. Gökpnar and M. Ebegil, A study on tests of hypothesis based on ridge estimator, Gazi Univ. J. Sci. 29 (4), 769-781, 2016.
13. [13] A. M. Halawa and M. Y. El Bassiouni, Tests of regression coefficients under ridge regression models, J. Stat. Comput. Simul. 65 (14), 341356, 2000.
14. [14] R. R. Hocking, F. M. Speed and M. J. Lynn, A class of biased estimators in linear regression, Technometrics 18 (4), 425-437, 1976.
15. [15] A. E. Hoerl and R. W. Kennard, Ridge regression: biased estimation for nonorthogonal problems, Technometrics 12 (1), 55-67, 1970.
16. [16] A. E. Hoerl and R. W. Kennard, Ridge regression iterative estimation of the biasing parameter, Commun. Stat. Theory Methods 5 (1), 77-88, 1976.
17. [17] M. A. Hoque and B. M. G. Kibria, Some one and two parameter estimators for the multicollinear Gaussian linear regression model: simulations and applications, Surv. Math. Appl. 18, 183-221, 2023.
18. [18] G. Khalaf and G. Shukur, Choosing ridge parameter for regression problems, Commun. Stat. Theory Methods 34 (5), 1177-1182, 2005.
19. [19] B. M. G. Kibria, Performance of some new ridge regression estimators, Commun. Stat. Simul. Comput. 32 (2), 419-435, 2003.
20. [20] B. M. G. Kibria, More than hundred (100) estimators for estimating the shrinkage parameter in a linear and generalized linear ridge regression models, Econom. Stat. 2, 2022.
21. [21] B. M. G. Kibria and S. Banik, Some ridge regression estimators and their performances, J. Mod. Appl. Stat. Methods 15, 206-238, 2016.
22. [22] B. M. G. Kibria and A. F. Lukman, A new ridge-type estimator for the linear regression model: simulations and applications, Scientifica 2020 (1), 9758378, 2020.
23. [23] K. Liu, A new class of biased estimate in linear regression, Commun. Stat. Theory Methods 22 (2): 393-402, 1993.
24. [24] G. C. McDonald and D. I. Galarneau, A Monte Carlo evaluation of some ridge-type estimators, J. Am. Stat. Assoc. 70 (350), 407-416, 1975.
25. [25] S. Mermi, Ö. Akkus, A. Göktas and N. Gündüz, A new robust ridge parameter estimator having no outlier and ensuring normality for linear regression model, J. Radiat. Res. Appl. Sci. 17 (1), 100788, 2024.
26. [26] R. S. Nickerson, Null hypothesis significance testing: a review of an old and continuing controversy, Psychol. Methods 5 (2), 241, 2000.
27. [27] M. Nomura, On the almost unbiased ridge regression estimator, Commun. Stat. Simul. Comput. 17 (3), 729-743, 1988.
28. [28] R. L. Obenchain, Classical f-tests and confidence regions for ridge regression, Technometrics 19 (4), 429-439, 1977.
29. [29] S. Perez-Melo and B. M. G. Kibria, On some test statistics for testing the regression coefficients in presence of multicollinearity: a simulation study, Stats 3 (1), 40-55, 2020.
30. [30] R Core Team, R: A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna (Austria), 2021. Available at: https: //www.R-project.org/.
31. [31] A. K. M. D. E. Saleh, M. Arashi and B. M. G. Kibria, Theory of Ridge Regression Estimation with Applications, John Wiley & Sons, 2019.
32. [32] S. Schaffer, Higher order multigrid methods, Math. Comput. 43 (167), 89-115, 1984.
33. [33] C. Stein, et al., Inadmissibility of the usual estimator for the mean of a multivariate normal distribution, Proc. Third Berkeley Symp. Math. Stat. Probab. 1, 197-206, 1956.
34. [34] R. A. Thisted, Ridge Regression, Minimax Estimation, and Empirical Bayes Methods, Stanford University, Stanford (CA), 1976.
35. [35] R. Tibshirani, Regression shrinkage and selection via the Lasso, J. R. Stat. Soc. Ser. B Stat. Methodol. 58 (1), 267-288, 1996.
36. [36] H. D. Vinod, Confidence intervals for ridge regression parameters, Time Ser. Econom. Model. Adv. Stat. Sci. Festschr. Honor Prof. V.M. Joshi’s 70th Birthday, Vol. III, 279-300, 1987.
37. [37] M. R. Özkale and H. Altuner, Bootstrap confidence interval of ridge regression in linear regression model: a comparative study via a simulation study, Commun. Stat. Theory Methods 52 (20), 7405-7441, 2023.
38. [38] N. S. Hawa, M. Y. Mustafa, B. M. G. Kibria and Z. Y. Algamal, Bootstrap Liutype estimator for Conway-Maxwell-Poisson regression model, Commun. Stat. Simul. Comput. 0 (0), 1-12, 2025. doi:10.1080/03610918.2025.2462680.
39. [39] Z. Y. Algamal, Shrinkage parameter selection via modified cross-validation approach for ridge regression model, Commun. Stat. Simul. Comput. 49 (7), 1922-1930, 2020. doi:10.1080/03610918.2018.1508704.
40. [40] Z. Y. Algamal, A new method for choosing the biasing parameter in ridge estimator for generalized linear model, Chemom. Intell. Lab. Syst. 183, 96-101, 2018. doi:10.1016/j.chemolab.2018.10.014.