Performance of a New Restricted Biased Estimator in Logistic Regression

Year 2018, Volume: 22 Issue: 1, 53 - 59, 16.04.2018

Yasin Asar

Abstract

It is known that the variance of the maximum likelihood estimator (MLE) inflates when the explanatory variables are correlated. This situation is called the multicollinearity problem. As a result, the estimations of the model may not be trustful. Therefore, this paper introduces a new restricted estimator (RLTE) that may be applied to get rid of the multicollinearity when the parameters lie in some linear subspace  in logistic regression. The mean squared errors (MSE) and the matrix mean squared errors (MMSE) of the estimators considered in this paper are given. A Monte Carlo experiment is designed to evaluate the performances of the proposed estimator, the restricted MLE (RMLE), MLE and Liu-type estimator (LTE). The criterion of performance is chosen to be MSE. Moreover, a real data example is presented. According to the results, proposed estimator has better performance than MLE, RMLE and LTE.

Keywords

Estimation, Liu-type estimator; MLE; MSE; Multicollinearity; Monte Carlo simulation

References

• [1] Asar, Y. 2017. Some new methods to solve multicollinearity in logistic regression. Communications in Statistics-Simulation and Computation, 46(4) 2576-2586.
• [2] Arashi, M., Kibria, B. M. G., Norouzirad, M., Nadarajah, S. 2014. Improved preliminary test and Stein-rule Liu estimators for the ill-conditioned elliptical linear regression model. Journal Multivariate Analysis. 126 53–74.
• [3] Asar, Y., Genç, A. 2016. New shrinkage parameters for the Liu-type logistic estimators. Communications in Statistics-Simulation and Computation, 45(3) 1094-1103.
• [4] Belsley, D. A., Kuh, E.,Welsch, R.E. 1980. Regression diagnostics: Identifying influential data and sources of collinearity. Wiley, New York.
• [5] Cox, D. R., Snell, E. J. 1989. Analysis of Binary Data (2nd ed.): Chapman & Hall.
• [6] Duffy, D. E., Santner, T. J. 1989. On the small sample properties of norm-restricted maximum likelihood estimators for logistic regression models. Communications in Statistics-Theory and Methods, 18(3) 959-980.
• [7] Hoerl, A. E., Kennard, R. W. 1970. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1) 55-67.
• [8] Hoerl, A. E., Kennard, R. W., Baldwin, K. F. 1975. Ridge regression: some simulations. Communications in Statistics-Theory and Methods, 4(2) 105-123.
• [9] Hosmer, D.D., Lemeshow, S., Sturdivant, R.X. 2013. Applied Logistic Regression. Hoboken, New Jersey: John Wiley & Sons.
• [10] İnan, D., Erdoğan, B. E. 2013. Liu-Type Logistic Estimator. Communications in Statistics-Simulation and Computation, 42(7) 1578-1586.
• [11] LeCessie, S., VanHouwelingen, J.C. 1992. Ridge estimators in logistic regression. Applied Statistics 41(1) 191-201.
• [12] Liu, K. 1993. A new class of blased estimate in linear regression. Communications in Statistics-Theory and Methods, 22(2) 393-402.
• [13] Liu, K. 2003. Using Liu-type estimator to combat collinearity. Communications in Statistics-Theory and Methods, 32(5), 1009-1020.
• [14] Kibria, B. M. G. 2003. Performance of some new ridge regression estimators. Communications in Statistics-Simulation and Computation, 32(2) 419-435.
• [15] Mansson, K., Kibria, B. M. G., Shukur, G. 2012. On Liu estimators for the logit regression model. Economic Modelling, 29(4) 1483-1488.
• [16] Mansson, K., and Shukur, G. 2011.On Ridge Parameters in Logistic Regression. Communications in Statistics-Theory and Methods, 40(18) 366-3381.
• [17] Mansson, K., Kibria, B. M. G., Shukur, G. 2016. A restricted Liu estimator for binary regression models and its application to an applied demand system. Journal of Applied Statistics, 43(6) 1119-1127.
• [18] Rao, R.C., Shalabh, Toutenburg, H.S., Heumann, C. 2008. Linear models and generalizations, least squares and alternatives. ISBN 978-3-540-74226-5 3rd edition Springer Berlin Heidelberg New York.
• [19] Roozbeh, M., Arashi, M. 2014. Feasible ridge estimator in seemingly unrelated semiparametric models. Communications in Statistics-Simulation and Computation, 43(10), 2593-2613.
• [20] Özkale, M. R., Lemeshow, S., Sturdivant, R. 2017. Logistic regression diagnostics in ridge regression. Computational Statistics, https://doi.org/10.1007/s00180-017-0755-x.
• [21] Saleh, A. M. E., Kibria, B. M. G. 2013. Improved ridge regression estimators for the logistic regression model. Computational Statistics, 28(6) 2519-2558.
• [22] Schaefer, R. L., Roi, L. D., Wolfe, R. A. 1984. A Ridge Logistic Estimator. Communications in Statistics-Theory and Methods, 13(1) 99-113.
• [23] Yüzbaşı, B., Ahmed, S. E., Güngör, M. 2017. Improved Penalty Strategies in Linear Regression Models, REVSTAT–Statistical Journal, 15(2), 251-276.
• [24] Yüzbaşı, B., Ejaz Ahmed, S. 2016. Shrinkage and penalized estimation in semi-parametric models with multicollinear data. Journal of Statistical Computation and Simulation, 86(17), 3543-3561.
• [25] Wu, J. 2016. Modified restricted Liu estimator in logistic regression model. Computational Statistics, 31(4) 1557-1567.