Combating Multicollinearity: A New Two-Parameter Approach

Öz

The ordinary least square (OLS) estimator is the Best Linear Unbiased Estimator (BLUE) when all linear regression model assumptions are valid. The OLS estimator, however, becomes inefficient in the presence of multicollinearity. Various one and two-parameter estimators have been proposed to circumvent the problem of multicollinearity. This paper presents a new twoparameter estimator called Liu-Kibria Lukman Estimator (LKL) estimator. The proposed estimator is compared theoretically and through Monte Carlo simulation with existing estimators such as the ordinary least square, ordinary ridge regression, Liu, Kibria-Lukman, and Modified Ridge estimators. The results show that the proposed estimator performs better than existing estimators considered in this study under some conditions, using the mean square error criterion. A real-life application to Portland cement and Longley datasets supported the theoretical and simulation results by giving the smallest mean square error compared to the existing estimators.

Anahtar Kelimeler

• LKL Estimator
• OLS Estimator
• Monte Carlo Simulation
• Multicollinearity
• Mean Square Error

Çoklu İç İlişki İle Mücadele: Yeni İki Parametre Yaklaşımı

Öz

Regresyon analizinde tüm varsayımlarının sağlanması durumunda en küçük kareler (EKK) tahmin edicisi, en iyi doğrusal yansız tahmin edicidir. Fakat, EKK tahmin edicisi çoklu iç ilişki durumunda etkinliğini kaybetmektedir. Çoklu iç ilişki problemini çözmek için tek ve iki parametreli bazı tahmin ediciler önerilmiştir. Bu çalışmada, Liu-Kibria Lukman tahmin edicisi olarak adlandırılan yeni iki parametreli bir tahmin edici önerilmiştir. Önerilen yeni tahmin edici EKK, Ridge, Liu, Kibria-Lukman, Modifiye Ridge tahmin edicileri ile teorik olarak ve Monte Carlo simülasyon çalışması ile karşılaştırılmıştır. Yapılan çalışmalarda ele alınan tahmin ediciler için hata kareler ortalaması kriterine göre önerilen tahmin edicinin daha iyi performansa sahip olduğu gösterilmiştir. Portland çimento ve Longley veri setleri için yapılan gerçek veri uygulamasında elde edilen sonuçlar ile teorik karşılaştırma ve simülasyon çalışmasının sonuçları desteklenmiştir.

Anahtar Kelimeler

• LKL Tahmin Edicisi
• EKK Tahmin Edicisi
• Monte Carlo Simulasyon
• Çoklu İç İlişki
• Hata Kereler Ortalaması.

Kaynakça

1. Ahmad, S. and Aslam, M. (2020), Another proposal about the new two-parameter estimator for a linear regression model with correlated regressors, Communications in Statistics-Simulation and Computation, 1-19.
2. Aslam, M. and Ahmad, S. (2020), The modified Liu-ridge-type estimator: A new class of biased estimators to address multicollinearity, Communications in Statistics - Simulation and Computation.
3. Ayinde, K., Lukman, A.F., Samuel, O.O. and Ajiboye, S.A. (2108), Some new adjusted ridge estimators of linear regression model, Int. J. Civ. Eng. Technol., 9(11), 2838‐2852.
4. Chatterjee, S. and Hadi, A. S. (2006), Regression analysis by example, 4th ed., John Wiley and Sons, NJ, US.
5. Chatterjee, S., Hadi, A. S. and Price, B. (2000), Regression by example, 3rd ed., John Wiley and Sons, New York, US.
6. Dawoud, I. and Kibria, B. M. G. (2020), A new biased estimator to combat the multicollinearity of the gaussian linear regression model, Stats, 3(4), 526-541.
7. Dawouda, I., Lukman, A. F. and Haadi, A.R. (2022), A new biased regression estimator: Theory, simulation and application., Scientific African, 15, p. e01100.
8. Dorugade, A. V. (2016), New ridge parameters for ridge regression, Journal of the Association of Arab Universities for Basic and Applied Sciences, 1-6.

9. Farebrother, R. (1976), Further results on the mean square error of ridge regression, Journal of the Royal Statistical Society, B38, 248‐250.
10. Greene, W. H. (2003), Econometric analysis, 5th ed., Prentice Hall Saddle River, New Jersey, US.
11. Helland, I. S. (1988), On the structure of partial least squares regression, Communication is Statistics, Simulations and Computations, 17, 581–607.
12. Helland, I. S. (1990), Partial least squares regression and statistical methods, Scandinavian Journal of Statistics, 17, 97 – 114.
13. Hoerl, A. E. (1962), Application of ridge analysis to regression problems, Chemical Engineering Progress, 58, 54 –59.
14. Hoerl, A. E., and Kennard, R. W. (1970), Ridge regression biased estimation for non-orthogonal problems, Technometrics, 27–51.
15. Kaçıranlar, S., Sakallıoğlu, S., Akdeniz, F., Styan, G.P.H. and Werner, H.J. (1999), A new biased estimator in linear regression and a detailed analysis of the widely– analyzed dataset on portland cement, Sankhya 61, 443–459.
16. Kibria, B. M. (2003), Performance of some new ridge regression estimators, Communications in Statistics - Simulation and Computation, 32, 419-435.
17. Kibria, B.M.G. and Lukman, A.F. (2020), A new ridge-type estimator for the linear regression model: Simulations and applications, Scientifica, 1–16.
18. Li, Y. and Yang, H. (2012), A new Liu-type estimator in linear regression model, Statistical Papers., 53, 427–437.
19. Liu, K. (1993), A new class of biased estimate in linear regression, Communications in Statistics.-Theory and Methods, 22, 393–402.
20. Longley, J. (1967), An appraisal of least-squares programs for electronic computer from the point of view of the user, Journal of the American Statistical Association, 62, 819–841.
21. Lukman A.F. and Ayinde, K. (2017), Review and classifications of the ridge parameter estimation techniques, Hacettepe Journal of Mathematics and Statistics, 46 (5), 953‐967.
22. Lukman, A. F., Ayinde, K., Sek, S. K. and Adewuyi, E. (2019), A modified new two-parameter estimator in a linear regression model, Modelling and Simulation in Engineering 2019:6342702.
23. Maddala, G. S. (2002), Introduction to econometrics, 3rd ed., John Willey and Sons Limited, England.
24. Marquardt, D.W. (1970), Generalized inverse, ridge regression, biased linear estimation and non–linear estimation, Technometrics, 12, 591–612.
25. Massy, W. F. (1965), Principal component regression in exploratory statistical research, Journal of the American Statistical Association, 60, 234 –246.
26. McDonald, G. C., and Galarneau, D. I. (1975), A Monte Carlo evaluation of some ridge-type estimators, Journal of the American Statistical Association, 70, 407-416.
27. Naes, T., and Marten, H. (1988), Principal component regression in NIR analysis: Viewpoints, background details selection of components, Journal of Chemometrics, 2, 155 –167.
28. Newhouse, J. P., and Oman, S. D. (1971), An evaluation of ridge estimators., A report prepared for the United States air force project RAND.
29. Owolabi, A.T., Ayinde, K. and Alabi, O.O. (2022a), A New Ridge-Type Estimator for the Linear Regression Model with correlated regressors, Concurrency and Computation: Practice and Experience, p. CPE6933.
30. Owolabi, A. T., Ayinde, K., Idowu, J. I., Oladapo, O. J. and Lukman, A. F. (2022b), A New two-parameter estimator in the linear regression model with correlated regressors, Journal of Statistics Applications & Probability, 11, 499-512.
31. Phatak, A. and Jony, S. D. (1997), The geometry of partial least squares, Journal of Chemometrics, 11, 311–338.
32. Qasim, M., Månsson, K., Sjolander, P. and Kibria, B. G. (2021), A new class of efficient and debiased two-step shrinkage estimators: method and application, Journal of Applied Statistics, 1-25.
33. Saleh, A. K., Arashi, M. E. M. and Kibria, B. M. G. (2019), Theory of Ridge Regression Estimation with Applications., New Jersey: Wiley, Hoboken.
34. Trenkler, G. and Toutenburg, H. (1990), Mean squared error matrix comparisons between biased estimators-an overview of recent results, Statistical Papers, 31(1), 165‐179.
35. Wang, S.G., Wu, M.X. and Jia, Z.Z. (2006), Matrix Inequalities, 2nd ed., Chinese Science Press, Beijing.
36. Wichern, D. and Churchill, G. (1978), A comparison of Ridge estimators, Technometrics, 20, 301–311.
37. Woods, H., Steinour, H. H., and Starke, H. R. (1932), Effect of composition of Portland cement on heat evolved during hardening, Industrial & Engineering Chemistry, 24(11), 1207–1214.
38. Yang, H. and Chang, X. (2010), A new two-parameter estimator in linear regression, Communications in Statistics -Theory and Methods, 39(6), 923–934.