Comparison of Restricted and Unrestricted estimators with a Monte-Carlo Simulation Study

Öz

This study deals with the problem of multicollinearity in the linear regression model. Restricted and unrestricted parameter estimates are chosen among biased estimators to be studied and compared as two corresponding groups, with the aim of identifying which group gives better parameter estimates in the case of multicollinearity. Estimators' performance is compared according to matrix mean square error and scalar mean square error. Proceeding from this, it has been shown that, in the sense of Scalar Mean Square Error (SMSE), the Restricted Ridge regression (RRR) estimator outperforms all constrained and unconstrained estimators, while the Ridge regression is superior to the unconstrained set of estimators. A real-life application and Monte-Carlo simulation study are conducted to compare the performance of restricted and unrestricted estimators. As a result, it was decided that the most effective estimators are the restricted biased estimators when it comes to the state of multicollinearity.

Anahtar Kelimeler

• Multicollinearity
• Mean square error
• Biased estimators
• Monte-Carlo simulation
• linear restrictions.

Monte-Carlo Simülasyon Çalışması ile Sınırlı ve Sınırsız Tahmin Edicilerin Karşılaştırılması

Öz

Bu çalışma, doğrusal regresyon modelinde çoklu doğrusallık sorununu ele almaktadır. Çoklu doğrusal bağlantı durumunda hangi grubun daha iyi parametre tahminleri verdiğini belirlemek amacıyla, incelenecek ve karşılaştırılacak yanlı tahmin ediciler arasından kısıtlı ve kısıtlı olmayan parametre tahminleri iki grup olarak seçilmiştir. Tahmin edicilerin performansı matris ortalama kare hatası ve skaler ortalama kare hatasına göre karşılaştırılmıştır. Buradan hareketle, Skaler Ortalama Kare Hata (SMSE) anlamında, Kısıtlı Ridge regresyonu (RRR) tahmin edicisinin tüm kısıtlı ve kısıtlı olmayan tahmin edicilerden daha iyi performans gösterdiği, Ridge regresyonunun ise kısıtlı olmayan tahmin ediciler kümesinden daha üstün olduğu gösterilmiştir. Kısıtlı ve kısıtlı olmayan tahmin edicilerin performansını karşılaştırmak için gerçek hayattan bir uygulama ve Monte-Carlo simülasyon çalışması yapılmıştır. Sonuç olarak, çoklu doğrusal bağlantı durumu söz konusu olduğunda en etkin tahmin edicilerin kısıtlı yanlı tahmin ediciler olduğuna karar verilmiştir.

Anahtar Kelimeler

• Çoklu doğrusallık
• Ortalama karesel hata
• Yanlı tahminciler
• Monte-Carlo simülasyonu
• doğrusal kısıtlamalar

Kaynakça

1. Akdeniz, F., and Erol, H. (2003), Mean squared error matrix comparisons of some biased estimators in linear regression. Communications in statistics-theory and methods, 32(12), 2389-2413.
2. Akdeniz, F. and Kaçiranlar, S. (1995), On the almost unbiased generalized liu estimator and unbiased estimation of the bias and mse, Communications in Statistics - Theory and Methods, 24(7), 1789-1797.
3. Akdeniz, F., and Roozbeh, M. (2019), Generalized difference-based weighted mixed almost unbiased ridge estimator in partially linear models, Statistical Papers, 60(5), 1717-1739.
4. Albert, A. (1973), The Gauss–Markov theorem for regression models with possibly singular covariances, SIAM Journal on Applied Mathematics, 24(2), 182-187.
5. Aslam, M. (2014). Performance of Kibria's Method for the Heteroscedastic Ridge Regression Model: Some Monte Carlo Evidence, Communications in Statistics - Simulation and Computation, 43(4), 673-686.
6. Dorugade, A. V. (2014), A modified two-parameter estimator in linear regression, Statistics in Transition. New Series, 15(1), 23-36.
7. Gibbons, D. G. (1981), A simulation study of some ridge estimators, Journal of the American Statistical Association, 76(373), 131-139.
8. Graybill, F. A. (1976), Theory and application of the linear model, 183, Duxbury press North Scituate, MA.

9. Graybill, F. A. (1983),Matrices with Applications in Statistics (Belmont, CA: Wadsworth), Graybill2Matrices With Applications in Statistics1983.
10. Groß, J. (2003), Restricted ridge estimation, Statistics &Probability letters, 65(1), 57-64.
11. Gruber, M. H. J. (1998),Improving efficiency by shrinkage, Statistics: Textbooks and Monographs,156. In: Marcel Dekker Inc., New York. Gruber, M. H. J., Regression estimators: A comparative study: JHU Press.
12. Hoerl, A. E., Kannard, R. W.and Baldwin, K. F. (1975), Ridge regression: Some simulations, Communications in Statistics, 4(2), 105-123.
13. Hoerl, A. E.,and Kennard, R. W. (1970a), Ridge Regression: Applications to Nonorthogonal Problems. Technometrics, 12(1), 69-82.
14. Hoerl, A. E.,and Kennard, R. W. (1970b), Ridge Regression: Biased Estimation for Nonorthogonal Problems, Technometrics, 12(1), 55-67.
15. Idowu, J. I., Oladapo, O. J., Owolabi, A. T., Ayinde, K. and Akinmoju, O. (2023), Combating multicollinearity: A new two-parameter approach, Nicel Bilimler Dergisi, 5(1), 90-116.
16. Kejian, L. (1993), A new class of blased estimate in linear regression, Communications in Statistics - Theory and Methods, 22(2), 393-402.
17. Kibria, B. M. G. (2003), Performance of Some New Ridge Regression Estimators. Communications in Statistics-Simulation and Computation, 32(2), 419-435.
18. Kibria, B. M. G., and Banik, S. (2016), Some ridge regression estimators and their performances. Journal of Modern Applied Statistical Methods, 15, 206-238.
19. Kibria, B. M., and Lukman, A. F. (2020), A new ridge-type estimator for the linear regression model: Simulations and applications, Scientifica.
20. Liu, K. (2003), Using Liu-Type Estimator to Combat Collinearity. Communications in Statistics-Theory and Methods, 32(5), 1009-1020.
21. Lukman, A. F., Ayinde, K., Aladeitan, B. and Bamidele, R. (2020), An unbiased estimator with prior information, Arab Journal of Basic and Applied Sciences, 27(1), 45-55.
22. Lukman, A. F., Ayinde, K., Binuomote, S. and Clement, O. A. (2019), Modified ridge ‐type estimator to combat multicollinearity: Application to chemical data. Journal of Chemometrics, 33(5), e3125.
23. Månsson, K. and Kibria, B. G. (2021), Estimating the unrestricted and restricted Liu estimators for the Poisson regression model: Method and application, Computational Economics, 58, 311-326.
24. Massy, W. F. (1965), Principal components regression in exploratory statistical research, Journal of the American Statistical Association, 60(309), 234-256.
25. McDonald, G. C., and Galarneau, D. I. (1975), A Monte Carlo Evaluation of Some Ridge type Estimators, Journal of the American Statistical Association, 70(350), 407- 416.
26. Montgomery, D. C., Peck, E. A.and Vining, G. G. (2012),Introduction to linear regression analysis,821, John Wiley & Sons.
27. Montgomery, D. C., Peck, E. A.and Vining, G. G. (2021), Introduction to linear regression analysis, John Wiley & Sons.
28. Muniz, G.and Kibria, B. M. G. (2009), On Some Ridge Regression Estimators: An Empirical Comparisons, Communications in Statistics - Simulation and Computation, 38(3), 621-630.
29. Najarian, S., Arashi, M.and Kibria, B. M. G. (2013), A Simulation Study on Some Restricted Ridge Regression Estimators, Communications in Statistics - Simulation and Computation, 42(4), 871-890.
30. Newhouse, J. P.and Oman, S. D. (1971),An Evaluation Of Ridge Estimators.
31. Özkale, M. R. (2014), The relative efficiency of the restricted estimators in linear regression models. Journal of Applied Statistics, 41(5), 998-1027.
32. Özkale, M. R., and Kaciranlar, S. (2007). The restricted and unrestricted two-parameter estimators, Communications in Statistics—Theory and Methods, 36(15), 2707- 2725.
33. Rao, C. R.and Toutenburg, H. (1995),Linear models. In Linear models (pp. 3-18): Springer.
34. Roozbeh, M. (2018), Optimal QR-based estimation in partially linear regression models with correlated errors using GCV criterion, Computational Statistics & Data Analysis, 117, 45-61.
35. Sakallıoğlu, S.and Kaçıranlar, S. (2008), A new biased estimator based on ridge estimation, Statistical Papers, 49(4), 669-689.
36. Shaltoot, I. (2021), Comparison of restricted and unrestricted estimators in the multiple linear regression analysis, Unpublished Master's thesis, Eskisehir Technical University.
37. Şiray, G. Ü. (2014). Modified and Restricted r-k Class Estimators. Communications in Statistics-Theory and Methods, 43(24), 5130-5155.
38. Stein, C. (1956, January). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley symposium on mathematical statistics and probability, 1(1), 197-206.
39. Swindel, B. F. (1976), Good ridge estimators based on prior information, Communications in Statistics - Theory and Methods, 5(11), 1065-1075.
40. Toutenburg, H., and Trenkler, G. (1990). Mean square error matrix comparisons of optimal and classical predictors and estimators in linear regression, Computational Statistics and Data Analysis, 10(3), 297-305.
41. Trenkler, G. (1983), A note on superiority comparisons of homogeneous linear estimators. Communications in statistics-theory and methods, 12(7), 799-808.
42. Vinod, H. D., and Ullah, A. (1981),Recent advances in regression methods, 41, Marcel Dekker Incorporated.
43. Vizcarrondo, C., and Wallace, T. D. (1968), A test of the mean square error criterion for restrictions in linear regression, Journal of the American Statistical Association, 63(322), 558-572.
44. Wencheko, E. (2000), Estimation of the signal-to-noise in the linear regression model. Statistical Papers, 41(3), 327.
45. Woods, H., Steinour, H. H., and Starke, H. R. (1932), Effect of composition of Portland cement on heat evolved during hardening,Industrial & Engineering Chemistry, 24(11), 1207-1214.
46. Yang, H., and Chang, X. (2010), A new two-parameter estimator in linear regression. Communications in Statistics—Theory and Methods, 39(6), 923-934.
47. Yang, H., and Cui, J. (2011), A stochastic restricted two-parameter estimator in linear regression model. Communications in statistics-theory and methods, 40(13), 2318-2325.
48. Zhong, Z., andYang, H. (2007). Ridge estimation to the restricted linear model. Communications in Statistics—Theory and Methods, 36(11), 2099-2115.