Çoklu Doğrusallık ve Değişen Varyans Altında Farklı Ridge Parametrelerinin Bir Karşılaştırması

Year 2019, Volume: 23 Issue: 2, 381 - 389, 25.08.2019

Volkan Sevinç , Atila Göktaş

https://doi.org/10.19113/sdufenbed.484275

Cited By: 4

Abstract

Uygun bir
doğrusal regresyon modeli tahmin edilmesi sırasında karşılaşılan ana
problemlerden biri bağımsız değişkenler yüksek korelasyona sahip olduğu zaman
ortaya çıkan çoklu doğrusallıktır. Bu sorunun giderilmesi için sıradan en küçük
karelere bir alternatif yöntem olarak tanıtılan ridge regresyon tahmincisi
kullanılmaktadır. Sabit varyanslar varsayımını bozan değişen varyans durumu,
regresyon tahmininde diğer ana sorunlardan biridir. Daha sağlam bir doğrusal
regresyon eşitliği tahmin edebilmek için bu bozulma sorununa çözüm olarak
ağırlıklı en küçük kareler tahmini kullanılır. Ancak, hem çoklu doğrusallık hem
de değişen varyans sorunu mevcut olduğunda, ağırlıklı ridge regresyon tahminine
başvurulmalıdır. Ridge regresyon, kesin bir hesaplama formülü bulunmayan ridge
parametresine bağlıdır. Literatürde önerilen bir çok ridge parametresi
bulunmaktadır. Hem çoklu doğrusallık hem de değişen varyans içeren veri için bu
ridge parametrelerinin performanslarını analiz etmeye yönelik bir simülasyon
çalışması düzenlenmiştir. Farklı örnek hacimleri, farklı bağımsız değişken
sayıları ve farklı çoklu doğrusallık dereceleri kullanılmıştır. Ridge
parametrelerinin performansları ortalama hata kareleri değerleri göz önüne
alınarak karşılaştırılmıştır. Çalışma aynı zamanda, verinin hem çoklu
doğrusallık hem de değişen varyansa sahip olduğu durumda, ridge
parametrelerinin performanslarının, verinin sadece çoklu doğrusallığa sahip
olduğu durumdakinden farklı olduğunu göstermiştir.

Keywords

Çoklu doğrusallık, Ridge parametresi, Değişen varyans, Ridge regresyon, Ağırlıklı ridge regresyon

A Comparison of Different Ridge Parameters under Both Multicollinearity and Heteroscedasticity

Abstract

One of
the major problems in fitting an appropriate linear regression model is multicollinearity
which occurs when regressors are highly correlated. To overcome this problem, ridge
regression estimator which is an alternative method to the ordinary least
squares (OLS) estimator, has been used. Heteroscedasticity, which violates the
assumption of constant variances, is another major problem in regression
estimation. To solve this violation problem, weighted least squares estimation
is used to fit a more robust linear regression equation. However, when there is
both multicollinearity and heteroscedasticity problem, weighted ridge
regression estimation should be employed. Ridge regression depends on the ridge
parameter which does not have an explicit form of calculation. There are
various ridge parameters proposed in the literature. A simulation study was conducted to compare the
performances of these ridge parameters for both multicollinear and
heteroscedastic data. The following factors were varied: the number of
regressors, sample sizes and degrees of multicollinearity. The performances of the parameters were compared
using mean square error. The
study also shows that when the data are both heteroscedastic and multicollinear,
the estimation performances of the ridge parameters differs from the case for
only multicollinear data.

Keywords

Multicollinearity, Ridge parameter, Heteroscedasticity, Ridge regression, Weighted ridge regression

References

• [1] Hoerl, A. E., Kennard, R. 1970a. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12(1)(1970a), 55-67.
• [2] Hoerl, A.E. and Kennard, R. 1970b. Ridge Regression: Applications to Nonorthogonal Problems. Technometrics 12(1)(1970b), 69-82.
• [3] Hoerl, A. E., Kennard, R. and Baldwin, K. 1975. Ridge Regression: Some Simulations. Communications in Statistics. – Simulation and Computation, 4(2)(1975), 105-123.
• [4] Lawless, J., Wang, P. A. 1976. Simulation Study of Ridge and Other Regression Estimators. Communications in Statistics – Theory and Methods, 5(4)(1976), 307-323.
• [5] Schaeffer, R.L., Roi, L.D., Wolfe, R. A. 1894. A Ridge Logistic Estimator. Communications in Statistics - Theory and Methods, 13(1)(1984), 99-113.
• [6] Nomura, M. 1988. On The Almost Unbiased Ridge Regression Estimator. Communications in Statistics - Simulation and Computation, 17(3)(1988), 729-743.
• [7] Kibria, B. M. G. 2003. Performance of Some New Ridge Regression Estimators. Communications in Statistics - Simulation and Computation, 32(2)(2003), 419-435.
• [8] Khalaf, G., Shukur, G 2005. Choosing Ridge Parameter for Regression Problems. Communications in Statistics - Theory and Methods, 34(5)(2005), 1177-1182.
• [9] Norliza, A., Maizah, H. A., Ahmad, R. A. 2006. A Comparative Study On Some Methods for Handling Multicollinearity Problems. Mathematika, 22(2)(2006), 109-119.
• [10] Alkhamisi, M. A., Shukur, G. 2007. A Monte Carlo Study of Recent Ridge Parameters. Communications in Statistics - Simulation and Computation, 36(3)(2007), 1177-1182.
• [11] Batah, F. S., Ramnathan, T., Gore, S. D. 2008. The Efficiency of Modified Jackknife and Ridge Type Regression Estimators: A Comparison. 24(2)(2008), 111-122.
• [12] Muniz, G., Kibria, B. M. G. 2009. On Some Ridge Regression Estimators: An Empirical Comparisons. Communications in Statistics - Simulation and Computation 38(3)(2009), 621-630.
• [13] Kibria, B. M. G., Månsson, K., Shukur, G. 2011. Performance of Some Logistic Ridge Regression Estimators. Comput Econ., 40.4 (2011), 401-414.
• [14] Dorugade, A. V. 2014. On Comparison of Some Ridge Parameters in Ridge Regression. Sri Lankan Journal of Applied Statistics, 15(1)(2014), 31-46.
• [15] Asar, Y., Karaibrahimoğlu, A., Genç, A. 2014. Modified Ridge Regression Parameters: A Comparative Monte Carlo Study. Hacettepe Journal of Mathematics and Statistics, 43(5)(2014), 827-841.
• [16] Göktaş, A., Sevinç, V. 2016. Two New Ridge Parameters and A Guide for Selecting an Appropriate Ridge Parameter in Linear Regression. Gazi University Journal of Science, 29(1)(2016), 201-211.
• [17] Macedo, P., Scotto, M., Silva, E. 2010. On the Choice of the Ridge Parameter: A Maximum Entropy Approach. Communications in Statistics - Simulation and Computation, 39(8)(2010), 1628-1638.
• [18] Månsson, K., Shukur, G., Kibria, B. M. G. 2010. A Simulation Study of Some Ridge Regression Estimators under Different Distributional Assumptions. Communications in Statistics - Simulation and Computation, 39(8)(2010), 1639-1670.
• [19] Månsson, K., Shukur, G. 2011. On Ridge Parameters in Logistic Regression. Communications in Statistics - Theory and Methods, 40(18)(2011), 3366-3381.
• [20] Salam, M. E. F. A. E. 2015. Alternative Ridge Robust Regression Estimator for Dealing with Collinear Influential Data Points. International Journal of Contemporary Mathematical Sciences, 10(2015), 119-130.
• [21] Khalaf, G. 2012. A Proposed Ridge Parameter to Improve the Least Square Estimator. Journal of Modern Applied Statistical Methods, 11(2)(2012), 443-449.
• [22] Hamed, R., Hefnawy, A. E., Farag, A. 2013. Selection of the Ridge Parameter Using Mathematical Programming. Communications in Statistics - Simulation and Computation, 42(6)(2013), 1409-1432.
• [23] Månsson, K., Shukur, G., Sjölander, P. 2013. A New Ridge Regression Causality Test in the Presence of Multicollinearity. Communications in Statistics - Theory and Methods, 43(2)(2013), 235-248.
• [24] Dorugade, A. 2015. Correlation Based Ridge Parameters in Ridge Regression with Heteroscedastic Errors and Outliers. Journal of Statistical Theory and Applications, 14(4)(2015), 413-424.
• [25] Wong, K. Y., Chiu, S. N. 2015. An Iterative Approach to Minimize the Mean Squared Error in Ridge Regression. Computational Statistics, 30(2)(2015), 625-639.
• [26] Somahi, A. A., Mousa, S., Turk, L. I. 2015. Some New Proposed Ridge Parameters for the Logistic Regression Model. International Journal of Research in Applied, Natural and Social Sciences, 3(1)(2015), 67-82.
• [27] Duzan, H., Shariff, N. S. 2016. Solution to the Multicollinearity Problem by Adding some Constant to the Diagonal. Journal of Modern Applied Statistical Methods, 15(1)(2016), 752-773.
• [28] Kibria, B. M. G., Banik, S. 2016. Some Ridge Regression Estimators and Their Performances. Journal of Modern Applied Statistical Methods, 15(1)(2016), 206-238.
• [29] Alibuhtto, M. C. 2016. Relationship Between Ridge Regression Estimator and sample Size When Multicollinearity Present Among Regressors. World Scientific News, 59(2016), 12-23.
• [30] Lukman, A. F., Ayinde, K. 2016. Some Improved Classification-Based Ridge Parameter of Hoerl and Kennard Estimation Techniques. İstatistik: Journal of the Turkish Statistical Association, 9(3)(2016), 93-106.
• [31] Bhat, S., Raju, V. 2016. A Class of Generalized Ridge Estimators. Communications in Statistics - Simulation and Computation, 46(7)(2016), 5105-5112.
• [32] Uzuke, C.A., Mbegbu, J.I., Nwosu C. R. 2017. Performance of Kibria, Khalaf, and Shurkurs Methods When the Eigenvalues are Skewed. Communications in Statistics - Simulation and Computation, 46(3)(2017), 2071-2102.
• [33] Macedo, P. 2017. Ridge Regression and Generalized Maximum Entropy: An Improved Version of the Ridge–GME Parameter Estimator. Communications in Statistics - Simulation and Computation, 46(5)(2017), 3527-3539.
• [34] Lukman, A. F., Ayinde, K., Ajiboye, A. S. 2017. Monte Carlo Study of Some Classification-Based Ridge Parameter Estimators. Journal of Modern Applied Statistical Methods, 16(1)(2017), 428-451.
• [35] Giacalone, M., Panarello, D., Mattera, R. 2017. Multicollinearity in Regression: An Efficiency Comparison Between Lp-Norm and Least Squares Estimators. Quality & Quantity, 52(4)(2017), 1831–1859.