Jackknifed estimators for generalized linear models with multicollinearity

Abstract

Generalized linear models applications have become very popular in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. In this paper, we introduce a new Jackknifed two-parameter estimator and a new modified Jackknifed two-parameter estimator in the case of Poisson, negative binomial and gamma distributed response variables in generalized linear models. We examine bias vectors, covariance matrices, and matrix mean squared error of the Jackknifed ridge estimator, modified Jackknifed ridge estimator, Jackknifed Liu estimator, modified Jackknifed Liu estimator, Jackknifed Liu-type estimator and modified Jackknifed Liu-type estimator given in the literature. According to bias vectors and covariance matrices, the superiority of the Jackknifed two-parameter estimator has been demonstrated theoretically. The generalization of some estimation methods for ridge and Liu parameters in generalized linear models is provided. Also, the superiority of the Jackknifed two-parameter estimator and the modified Jackknifed two-parameter estimator are assessed by the simulated mean squared error via Monte-Carlo simulation study where the response follows a Poisson, negative binomial, and gamma distribution with the log link function. Finally, we consider real data applications. The proposed estimators are compared and interpreted.

Keywords

• generalized linear models
• Jackknifed estimator
• Monte-Carlo simulation
• multicollinearity
• two-parameter estimator

References

1. [1] L.A. Abduljabbar and Z.Y. Algamal, Jackknifed KL estimator in Bell regression model, Math. Stat. Eng. Appl. 71(3s2), 267-278, 2022.
2. [2] E. Akdeniz Duran and F. Akdeniz, Efficiency of the modified Jackknifed Liu-type estimator, Stat. Pap. 53, 265-280, 2012.
3. [3] Z.Y. Algamal, Developing a ridge estimator for the gamma regression model, J. Chemom. 32(10), e3054, 2018.
4. [4] Z.Y. Algamal and N. Hammood, A new Jackknifing ridge estimator for logistic regression model, Pak. J. Stat. Oper. Res. 955-961, 2022.
5. [5] Z.Y. Algamal, M.R. Abonazel and A.F. Lukman, Modified Jackknife ridge estimator for Beta regression model with application to chemical data, Int. J. Math. Stat. Comput. Sci. 1, 15-24, 2023.
6. [6] Z. Algamal, A. Lukman, B.K. Golam, B.K. and A. Taofik, Modified Jackknifed ridge estimator in bell regression model: theory, simulation and applications, Iraqi J. Comput. Sci. Math. 4(1), 146-154, 2023.
7. [7] Z.Y. Algamal, M.R. Abonazel, F.A. Awwad and E.T. Eldin, Modified Jackknife ridge estimator for the Conway-Maxwell-Poisson model, Sci. Afr. (19), e01543, 2023.
8. [8] A. Alkhateeb and Z. Algamal, Jackknifed Liu-type estimator in Poisson regression model, J. Iran. Stat. Soc. 19(1), 21-37, 2022.

9. [9] M. Alkhamisi,G. Khalaf and D. Shukur, Some modifications for choosing ridge parameters, Commun. Stat. Theory Methods 35(11), 2005-2020, 2006.
10. [10] M. Amin, M. Qasim, A. Yasin and M. Amanullah, Almost unbiased ridge estimator in the gamma regression model, Commun. Stat. Simul. Comput. 51(7), 3830-3850, 2022.
11. [11] Y. Asar and K. Kılınç, A Jackknifed ridge estimator in probit regression model, Stat. 54(4), 667-685, 2020.
12. [12] F.S.M. Batah, T.V. Ramanathan and S.D. Gore, The efficiency of modified Jackknife and ridge type regression estimators: a comparison, Surv. Math. Appl. 3, 111-122, 2008.
13. [13] S. Chatterjee and A.S. Hadi, Sensitivity analysis in linear regression, John Wiley & Sons, 2009.
14. [14] R Core Team, R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. , 2023.
15. [15] M.K. Çetinkaya, S. Kacıranlar and F. Kurtoglu, Developed first-order approximated estimators for the gamma distributed response variable, Commun. Stat. Simul. Comput. 52(8), 3919-3938, 2023.
16. [16] M.L. Delignette-Muller, C. Dutang, fitdistrplus: An R Package for Fitting Distributions, J. Stat. Softw. 64(4), 1-34, 2015. DOI 10.18637/jss.v064.i04.
17. [17] C.Y. Deng, A generalization of the ShermanMorrisonWoodbury formula, Appl. Math. Lett. 24(9), 1561-1564, 2011.
18. [18] F. Erdugan and F. Akdeniz, Computational method for Jackknifed generalized ridge tuning parameter based on generalized maximum entropy, Commun. Stat. Simul. Comput. 41(8), 1411-1429, 2012.
19. [19] A. Golan, G. Judge and D. Miller,Maximum Entropy Econometrics: Robust Estimation with Limited Data, (Chichester UK, John Wiley), 1996.
20. [20] M.H. Gruber, The efficiency of jack-knifed and usual ridge type estimators, Stat. Probab. Lett. 11(1), 49-51, 1991.
21. [21] A.A. Hamad and Z.Y. Algamal, Jackknifing KL estimator in generalized linear models, Int. J. Nonlinear Anal. Appl. 12(Special Issue), 2093-2104, 2021.
22. [22] D.V. Hinkley, Jackknifing in unbalanced situations, Technometrics 19(3), 285-292, 1977.
23. [23] A.E. Hoerl and R.W. Kennard, Ridge regression: Biased estimation for nonorthogonal problems, Technometrics 12(1), 55-67, 1970.
24. [24] D.M. Jabur, N.K. Rashad and Z.Y. Algamal, Jackknifed Liu-type estimator in the negative binomial regression model. Int. J. Nonlinear Anal. Appl.,13(1), 2675-2684, 2022.
25. [25] N.H. Jadhav and D.N. Kashid. A Jackknifed ridge M-estimator for regression model with multicollinearity and outliers, J. Stat. Theory Pract. 5(4), 659-673, 2011.
26. [26] K. Kadiyala, A class of almost unbiased and efficient estimators of regression coefficients, Econ. Lett. 16(3-4), 293-296, 1984.
27. [27] L. Kejian, A new class of blased estimate in linear regression, Commun. Stat. Theory Methods. 22(2), 393-402, 1993.
28. [28] B.G. Kibria, Performance of some new ridge regression estimators, Commun. Stat. Simul. Comput. 32(2), 419-435 2003.
29. [29] M. Khurana, Y.P., Chaubey and S. Chandra, Jackknifing the ridge regression estimator: A revisit, Commun. Stat. Theory Methods. 43(24), 5249-5262, 2014.
30. [30] F. Kurtolu and M.R. Özkale, Liu estimation in generalized linear models: application on gamma distributed response variable, Stat. Pap. 57, 911-928, 2016.
31. [31] K. Liu, Using Liu-type estimator to combat collinearity, Commun. Stat. Theory Methods. 32(5), 1009-1020, 2003.
32. [32] S. Mandal, R. Arabi Belaghi, A. Mahmoudi and M. Aminnejad, Steintype shrinkage estimators in gamma regression model with application to prostate cancer data, Stat. Med. 38(22), 4310-4322, 2019.
33. [33] K. Månsson, Developing a Liu estimator for the negative binomial regression model: method and application, J. Stat. Comput. Simul. 83(9), 1773-1780, 2013.
34. [34] G.C. McDonald and D.I. Galarneau, A Monte Carlo evaluation of some ridge-type estimators, J. Am. Stat. Assoc. 70(350), 407-416, 1975.
35. [35] R.G. Miller Jr, An unbalanced Jackknife, Ann. Stat. 880-891, 1974.
36. [36] R.G. Miller, The Jackknife-a review, Biometrika, 61(1), 1-15, 1974.
37. [37] G. Muniz, B.M. Kibria and G. Shukur, On developing ridge regression parameters: a graphical investigation, Department of Mathematics and Statistics, 10, 2012.
38. [38] R.H. Myers, D.C. Montgomery, G.G. Vining and T.J. Robinson, Generalized linear models: with applications in engineering and the sciences, John Wiley & Sons, 2012.
39. [39] F. Noeel and Z.Y. Algamal, Almost unbiased ridge estimator in the count data regression models, Electron. J. Appl. Stat. Anal. 14(1), 44-57, 2021.
40. [40] M. Nomura, On the almost unbiased ridge regression estimator, Commun. Stat. Simul. Comput. 17(3), 729-743, 1988.
41. [41] M. Nooi Asl, H. Bevrani, R. Arabi Belaghi and K. Mansson, Ridge-type shrinkage estimators in generalized linear models with an application to prostate cancer data, Stat. Pap. 62, 1043-1085, 2021.
42. [42] M. Noori Asl, H. Bevrani and R. Arabi Belaghi, Penalized and ridge-type shrinkage estimators in Poisson regression model, Commun. Stat. Simul. Comput. 51(7), 4039- 4056, 2022.
43. [43] H. Nyquist, Applications of the Jackknife procedure in ridge regression, Comput. Stat. Data Anal. 6(2), 177-183, 1988.
44. [44] K. Ohtani, On small sample properties of the almost unbiased generalized ridge estimator, Commun. Stat. Theory Methods. 15(5), 1571-1578, 1986.
45. [45] M.H. Quenouille, Notes on bias in estimation, Biometrika, 43(3/4), 353-360, 1956.
46. [46] M.R. Özkale and S. Kaciranlar, The restricted and unrestricted two-parameter estimators, Commun. Stat. Theory Methods. 36(15), 2707-2725, 2007.
47. [47] M.R. Özkale, A Jackknifed ridge estimator in the linear regression model with heteroscedastic or correlated errors, Stat. Probab. Lett. 78(18), 3159-3169, 2008.
48. [48] H.A. Rasheed, N.J. Sadik and Z.Y. Algamal, Jackknifed Liu-type estimator in the Conway-Maxwell Poisson regression model, Int. J. Nonlinear Anal. Appl. 13(1), 3153- 3168, 2022.
49. [49] R.L. Schaefer, Alternative estimators in logistic regression when the data are collinear, J. Stat. Comput. Simul. 25(1-2), 75-91, 1986.
50. [50] S. Seifollahi, H. Bevrani and O. Albalawi, Reducing Bias in Beta Regression Models Using Jackknifed LiuType Estimators: Applications to Chemical Data, J. Math. (1), 6694880, 2024.
51. [51] B. Singh, Y.P. Chaubeyand T.D. Dwivedi, An almost unbiased ridge estimator, Sankhya B. 342-346, 1986.
52. [52] B. Singh and Y.P. Chaubey, On some improved ridge estimators, Stat. Hefte. 28, 53-67, 1987.
53. [53] J. Tukey, Bias and confidence in not quite large samples, Ann. Math. Statist. 29, 614, 1958.
54. [54] S. Türkan and G. Özel, A new modified Jackknifed estimator for the Poisson regression model, J. Appl. Stat. 43(10), 1892-1905, 2016.
55. [55] S. Türkan and G. Özel, A Jackknifed estimators for the negative binomial regression model, Commun. Stat. Simul. Comput. 47(6), 1845-1865, 2018.
56. [56] H. Wickham, ggplot2: Elegant Graphics for Data Analysis, Springer-Verlag, New York, 2016.
57. [57] N. Yıldız, On the performance of the Jackknifed Liu-type estimator in linear regression model, Commun. Stat. Theory Methods. 47(9), 2278-2290, 2018.