The $r$-$d$ class estimator under exact linear restrictions in generalized linear models: Theory, simulation and application

Abstract

In this paper, we propose a restricted $r-d$ class estimator in generalized linear models by combining Liu and Principal component regression estimators, when exact linear restrictions are available as prior information along with the sample data. In addition, Particle Swarm Optimization is introduced and utilized to estimate the biasing parameter $ d $ of the newly constructed restricted estimator. In the presence of multicollinearity problem, the new estimator is compared with the current estimators that are maximum likelihood, principal components regression and $r-d$ class estimators, respectively. The performance of the proposed estimators is examined through simulation studies and a numerical example, considering response variables that follow Poisson, Binomial, and Negative binomial distributions. The evaluation is based on the scalar mean square error and the estimated mean square error criteria. The results indicate that the proposed estimator consistently outperforms all competing estimators considered in this study, both in simulation experiments and the numerical example, for suitably chosen values of the biasing parameter $ d $.

Keywords

• Exact restrictions
• logistic regression
• mean square error
• negative binomial response
• Poisson response
• particle swarm optimization
• principal components regression

References

1. [1] A. Abbasi and M.R. Özkale, 2024. Iterative stochastic restricted r−d class estimator in generalized linear models: application to binomial, Poisson and negative binomial distributions, Hacet. J. Math. Stat. 53(2), 1419-1437, 2024.
2. [2] A. Abbasi and M.R. Özkale, 2025. Iterative Stochastic Restricted r-k Class Estimator in Generalized Linear Models: Application on Logistic Regression, Iran. J. Sci. 49, 357367, 2025.
3. [3] A. Abbasi and M.R. Özkale, 2021. The r-k class estimator in generalized linear models applicable with simulation and empirical study using a Poisson and Gamma responses, Hacet. J. Math. Stat. 50(2), 594-611, 2021.
4. [4] A. Abbasi and M.R. Özkale, 2023. Restricted Liu estimator under stochastic linear restrictions in generalized linear models: theory and applications, Commun. Stat. Simul. Comput. 54(5), 4001422.
5. [5] Q. Abdul Kareem and Algamal, Z. 2020. Generalized ridge estimator shrinkage estimation based on particle swarm optimization algorithm, Iraqi Journal of Statistical Sciences, 17(2), 26-35.
6. [6] A.J. Dobson and A. G. Barnett, An introduction to generalized linear models. Chapman and Hall/CRC, 2018.
7. [7] R. C. Eberhart and Y.Shi, Particle swarm optimization: developments, applications and resources.In Proceedings of the 2001 congress on evolutionary computation (IEEE Cat. No. 01TH8546) (Vol. 1, pp. 81-86). IEEE, 2021
8. [8] J. Huang and H. Yang, A two-parameter estimator in the negative binomial regression model. J. Stat. Comput. Simul, 84(1), 124-134, 2014.

9. [9] D. Inan, E. Egrioglu, B. Sarica, S. Eichberg, O.E., Askin and M. Tez, Particle swarm optimization based Liu-type estimator. Commun. Stat. Theory Methods., 46(22), 11358-11369, 2017.
10. [10] B.M.G. Kibria, Performance of some new ridge regression estimators. Commun. Stat. Simul. Comput 32(2), 419-435, 2003.
11. [11] F. Kurtoglu and M. R. Özkale, Restricted ridge estimator in generalized linear models: Monte Carlo simulation studies on Poisson and binomial distributed responses, Commun. Stat. Simul. Comput. 48(4), 1-28, 2019.
12. [12] F. Kurtoglu and M.R. Özkale, Restricted Liu estimator in generalized linear models: Monte Carlo simulation studies on gamma and Poisson distributed responses, Hacet. J. Math. Stat. 48(4), 1191-1218, 2019.
13. [13] J. Kennedy and R. Eberhart, Particle swarm optimization. In Proceedings of ICNN’95-international conference on neural networks. ISPRS int. j. geo-inf., 1942- 1948, 1995.
14. [14] Y. Lamari, B. Freskura, A.Abdessamad, S. Eichberg and S. de Bonviller, Predicting spatial crime occurrences through an efficient ensemble-learning model, ISPRS int. j. geo-inf, 9(11), 645, 2020.
15. [15] B.D. Marx, A continuum of principal component generalized linear regressions.Comput. Stat. Data Anal. 13(4), 385-393, 1992.
16. [16] K. Mansson and B.MG. Kibria, Estimating the unrestricted and restricted Liu estimators for the Poisson regression model: Method and application, Comput. Econ., 58, 311-326, 2021.
17. [17] R.H. Myers, Classical and modern regression with applications, Belmont, CA: Duxbury press, 1990.
18. [18] G.C. McDonald, D.I. Galarneau, A monte carlo evaluation of some ridge-type estimators, J. Am. Stat. Assoc. 70(350), 407-416, 1975.
19. [19] H. Nyquist, Restricted Estimation of Generalized Linear Models, J. R. Stat. Soc.Ser.C. 40(1), 133-141, 1991.
20. [20] M.R. Özkale and E. Arıcan, First-order r − d class estimator in binary logistic regression model, Stat Probabil. Lett. 106, 19-29, 2016.
21. [21] M.R. Özkale, The r-d class estimator in generalized linear models: applications on gamma, Poisson and binomial distributed responses, J. Stat. Comput. Simul. 89(4), 615-640, 2019.
22. [22] M.R. Özkale and H. Nyquist, The stochastic restricted ridge estimator in generalized linear models, Stat. Pap. 62(3), 1421-1460, 2021.
23. [23] M.R. Özkale and A. Abbasi, Iterative restricted OK estimator in generalized linear models and the selection of tuning parameters via MSE and genetic algorithm, Stat. Pap. 1979-2040, 2022.
24. [24] Rao and Toutenburg, Linear Models: Least Squares and Alternatives Springer, New York, 1995.
25. [25] E.P. Smith and B.D. Marx, Ill-conditioned information matrices, generalized linear models and estimation of the effects of acid rain, Environmetrics. 1(1), 57-71, 1990.
26. [26] N. Sancar and D. Inan, A new alternative estimation method for Liu-type logistic estimator via particle swarm optimization: an application to data of collapse of Turkish commercial banks during the Asian financial crisis, Appl. Stat. 48, 2499-2514, 2021.
27. [27] M. Settles, An introduction to particle swarm optimization. Department of Computer Science, University of Idaho, 2 12, 2005.
28. [28] V.R. Uslu, E. Egrioglu, and E. Bas, Finding optimal value for the shrinkage parameter in ridge regression via particle swarm optimization, Am. J. Intell. Syst., 4, 142-147, 2014.
29. [29] D. Wang, D. Tan and L. Liu, Particle swarm optimization algorithm: an overview., Soft Comput. 22(2), 387-408, 2018.
30. [30] K. Wiktorowicz, T. Krzeszowski and K. Przednowek, Sparse regressions and particle swarm optimization in training high-order TakagiSugeno fuzzy systems, Neural Comput. Appl., 33(7), 2705-2717, 2021.