Journal of Statisticians: Statistics and Actuarial Sciences
Research Article
PDF EndNote BibTex RIS Cite

Çoklu Bağlantı Durumunda Regresyon Yöntemlerinin Karşılaştırılması

Year 2016, Volume 9, Issue 2, 47 - 53, 25.12.2016

Onur TOKA

Abstract

Bu çalışmada çoklu bağlantı sorununa çözüm getirebilmek için birkaç regresyon modelinin karşılaştırılması amaçlanmıştır. Doğrusal regresyonda çoklu bağlantının olması durumunda yanlış kestirim değerleri, kestirim için yanlı varyanslar gibi yanlışlıklar ortaya çıkar. Bu yüzden, kısmi en küçük kareler regresyonu (PLSR), ridge regresyon (RR), temel bileşenler regresyonu (PCR) gibi yöntemler, çoklu bağlantı sorununu aşmak için önerilmiştir. Bu çalışmada yöntemler, farklı derecelerde çoklu bağlantıya sahip veri kümeleri için bir benzetim çalışmasıyla karşılaştırılmıştır. Regresyon model kestirimleri için bütün sonuçlar hata kareler ortalaması bakımından birbirleriyle karşılaştırılmıştır. PLSR yönteminin bağımsız değişken sayısının çok olduğu durumda iyi bir yöntem olduğu ve RR yönteminin ise gözlem sayısı ve çoklu bağlantı sayısının çok olduğu durumda iyi bir yöntem olduğu sonuçları elde edilmiştir. PCR yönteminin benzetim çalışması senaryolarında tercih edilebilir bir yöntem olmadığı görülmüştür.

Keywords

Ridge regresyon, Kısmi en küçük kareler regresyonu, Çoklu bağlantı, Temel bileşenler regresyonu.

References

  • H. Abdi, 2003, Partial least squares (PLS) regression. – In: Lewis-Beck M. et al. (eds), Encyclopedia of social sciences research methods, Sage, 792–795.
  • E. Bulut and A. Alın, 2009, Kısmi En Küçük Kareler Regresyon Yöntemini Algoritmalarından Nipals Ve Pls-Kernel Algoritmalarının Karşılaştırılması Ve Bir Uygulama, Dokuz Eylül Üniversitesi Iktisadi Ve Idari Bilimler Fakültesi Dergisi, 24, 2, p. 127-138.
  • M. El-Fallah and A. El-Salam, 2014, A Note on Partial Least Squares Regression for Multicollinearity (A Comparative Study), International Journal of Applied Science and Technology, Vol. 4 No. 1; January 2014, 163-171. [4] P. Geladi and B. Kowalski, 1986, Partial Least-Squares Regression: A Tutorial, Analytica Chimica Acta, 185, 1–17.
  • J. Gonzalez , D. Pena, R. Romera, 2009, A robust partial least squares regression method with applications, J. Chemometr., 23, pp. 78–90.
  • I. S. Helland, 1990, Partial Least Squares Regression and Statistical Models, Scandinavian Journal of Statistics, 17(2), 97–114.
  • A. E. Hoerl and R. W. Kennard, 1970, Ridge Regression: Biased Estimation for Nonorthogonal Problems, Technometrics, 12, 1: 55-67.
  • A. E. Hoerl and R. W. Kennard and K. F. Baldwin, 1975. Ridge Regression: Some Simulation. Communication in Statistics, 4(2): 105-123.
  • L. Kejian, 2004, More on Liu-Type Estimator in Linear Regression, Communications in Statistics - Theory and Methods, 33:11, 2723-2733.
  • J. F. Lawless and P. Wang, 1976, A Simulation Study of Ridge and Other Regression Estimators, Communications in Statistics - Theory and Methods, A5 (4), 307-323.
  • S. Maitra and J. Yan, 2008, Principal Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression, 2008 Casualty Actuarial Society Discussion Paper Program, Presented June 15-18, 2008 Fairmont Le Château Frontenac Québec City, Québec, Canada.
  • W. F. Massy, 1965, Principal Component Regression in Exploratory Statistical Research, Journal of the American Statistical Association, 60, 234-256.
  • M. Stone and R. J. Brooks, 1990, Continuum Regression: Cross-Validated Sequentially Constructed Prediction Embracing Ordinary Least Squares, Partial Least Squares and Principal Components Regression. Journal of the Royal Statistical Society. Series B (methodological), 52(2), 237–269.
  • C. M. Theobald, 1974. Generalizations Of Mean Square Error Applied To Ridge Regression, Journal Of The Royal Statistical Society, Series B, 36 : 103-105.
  • R. Tibshirani, 1996, Regression Shrinkage and Selection via the Lasso, Journal of the Royal Statistical Society, Series B (methodological), 58(1), 267-288.
  • S. Toker, S. Kaçiranlar, 2013, On the Performance of Two Parameter Ridge Estimator under the Mean Square Error Criterion, Applıed Mathematıcs And Computatıon, vol.219, 4718-4728.
  • S. Wold, A. Ruhe, H. Wold, and W. J. Dunn, III, 1984, The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses, SIAM Journal on Scientific and Statistical Computing, 1984, Vol. 5, No. 3: 735-743.
  • O. Yeniay and A. Goktas, 2002, A comparison of partial least squares regression with other prediction methods, Hacettepe Journal of Mathematics and Statistics, Vol. 31, 99‐111

A Comparative Study on Regression Methods in the presence of Multicollinearity

Year 2016, Volume 9, Issue 2, 47 - 53, 25.12.2016

Onur TOKA

Abstract

Keywords

Ridge Regression, Partial Least Square Regression, Multicollinearity, Principal Component Regression

References

  • H. Abdi, 2003, Partial least squares (PLS) regression. – In: Lewis-Beck M. et al. (eds), Encyclopedia of social sciences research methods, Sage, 792–795.
  • E. Bulut and A. Alın, 2009, Kısmi En Küçük Kareler Regresyon Yöntemini Algoritmalarından Nipals Ve Pls-Kernel Algoritmalarının Karşılaştırılması Ve Bir Uygulama, Dokuz Eylül Üniversitesi Iktisadi Ve Idari Bilimler Fakültesi Dergisi, 24, 2, p. 127-138.
  • M. El-Fallah and A. El-Salam, 2014, A Note on Partial Least Squares Regression for Multicollinearity (A Comparative Study), International Journal of Applied Science and Technology, Vol. 4 No. 1; January 2014, 163-171. [4] P. Geladi and B. Kowalski, 1986, Partial Least-Squares Regression: A Tutorial, Analytica Chimica Acta, 185, 1–17.
  • J. Gonzalez , D. Pena, R. Romera, 2009, A robust partial least squares regression method with applications, J. Chemometr., 23, pp. 78–90.
  • I. S. Helland, 1990, Partial Least Squares Regression and Statistical Models, Scandinavian Journal of Statistics, 17(2), 97–114.
  • A. E. Hoerl and R. W. Kennard, 1970, Ridge Regression: Biased Estimation for Nonorthogonal Problems, Technometrics, 12, 1: 55-67.
  • A. E. Hoerl and R. W. Kennard and K. F. Baldwin, 1975. Ridge Regression: Some Simulation. Communication in Statistics, 4(2): 105-123.
  • L. Kejian, 2004, More on Liu-Type Estimator in Linear Regression, Communications in Statistics - Theory and Methods, 33:11, 2723-2733.
  • J. F. Lawless and P. Wang, 1976, A Simulation Study of Ridge and Other Regression Estimators, Communications in Statistics - Theory and Methods, A5 (4), 307-323.
  • S. Maitra and J. Yan, 2008, Principal Component Analysis and Partial Least Squares: Two Dimension Reduction Techniques for Regression, 2008 Casualty Actuarial Society Discussion Paper Program, Presented June 15-18, 2008 Fairmont Le Château Frontenac Québec City, Québec, Canada.
  • W. F. Massy, 1965, Principal Component Regression in Exploratory Statistical Research, Journal of the American Statistical Association, 60, 234-256.
  • M. Stone and R. J. Brooks, 1990, Continuum Regression: Cross-Validated Sequentially Constructed Prediction Embracing Ordinary Least Squares, Partial Least Squares and Principal Components Regression. Journal of the Royal Statistical Society. Series B (methodological), 52(2), 237–269.
  • C. M. Theobald, 1974. Generalizations Of Mean Square Error Applied To Ridge Regression, Journal Of The Royal Statistical Society, Series B, 36 : 103-105.
  • R. Tibshirani, 1996, Regression Shrinkage and Selection via the Lasso, Journal of the Royal Statistical Society, Series B (methodological), 58(1), 267-288.
  • S. Toker, S. Kaçiranlar, 2013, On the Performance of Two Parameter Ridge Estimator under the Mean Square Error Criterion, Applıed Mathematıcs And Computatıon, vol.219, 4718-4728.
  • S. Wold, A. Ruhe, H. Wold, and W. J. Dunn, III, 1984, The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses, SIAM Journal on Scientific and Statistical Computing, 1984, Vol. 5, No. 3: 735-743.
  • O. Yeniay and A. Goktas, 2002, A comparison of partial least squares regression with other prediction methods, Hacettepe Journal of Mathematics and Statistics, Vol. 31, 99‐111

Details

Primary Language English
Journal Section Articles
Authors

Onur TOKA> (Primary Author)
HACETTEPE ÜNİVERSİTESİ
Türkiye

Publication Date December 25, 2016
Published in Issue Year 2016, Volume 9, Issue 2

Cite

Bibtex @research article { jssa418127, journal = {İstatistikçiler Dergisi:İstatistik ve Aktüerya}, issn = {1308-0539}, eissn = {1308-0539}, address = {}, publisher = {Aktüerya Derneği}, year = {2016}, volume = {9}, number = {2}, pages = {47 - 53}, title = {A Comparative Study on Regression Methods in the presence of Multicollinearity}, key = {cite}, author = {Toka, Onur} }
APA Toka, O. (2016). A Comparative Study on Regression Methods in the presence of Multicollinearity . İstatistikçiler Dergisi:İstatistik ve Aktüerya , 9 (2) , 47-53 . Retrieved from https://dergipark.org.tr/en/pub/jssa/issue/36714/418127
MLA Toka, O. "A Comparative Study on Regression Methods in the presence of Multicollinearity" . İstatistikçiler Dergisi:İstatistik ve Aktüerya 9 (2016 ): 47-53 <https://dergipark.org.tr/en/pub/jssa/issue/36714/418127>
Chicago Toka, O. "A Comparative Study on Regression Methods in the presence of Multicollinearity". İstatistikçiler Dergisi:İstatistik ve Aktüerya 9 (2016 ): 47-53
RIS TY - JOUR T1 - A Comparative Study on Regression Methods in the presence of Multicollinearity AU - OnurToka Y1 - 2016 PY - 2016 N1 - DO - T2 - İstatistikçiler Dergisi:İstatistik ve Aktüerya JF - Journal JO - JOR SP - 47 EP - 53 VL - 9 IS - 2 SN - 1308-0539-1308-0539 M3 - UR - Y2 - 2016 ER -
EndNote %0 Journal of Statisticians: Statistics and Actuarial Sciences A Comparative Study on Regression Methods in the presence of Multicollinearity %A Onur Toka %T A Comparative Study on Regression Methods in the presence of Multicollinearity %D 2016 %J İstatistikçiler Dergisi:İstatistik ve Aktüerya %P 1308-0539-1308-0539 %V 9 %N 2 %R %U
ISNAD Toka, Onur . "A Comparative Study on Regression Methods in the presence of Multicollinearity". İstatistikçiler Dergisi:İstatistik ve Aktüerya 9 / 2 (December 2016): 47-53 .
AMA Toka O. A Comparative Study on Regression Methods in the presence of Multicollinearity. JSSA. 2016; 9(2): 47-53.
Vancouver Toka O. A Comparative Study on Regression Methods in the presence of Multicollinearity. İstatistikçiler Dergisi:İstatistik ve Aktüerya. 2016; 9(2): 47-53.
IEEE O. Toka , "A Comparative Study on Regression Methods in the presence of Multicollinearity", İstatistikçiler Dergisi:İstatistik ve Aktüerya, vol. 9, no. 2, pp. 47-53, Dec. 2016