Araştırma Makalesi

Comparison of Least Squares and Principal Components Regression Analysis

Yıl 2022, Cilt: 7 Sayı: 1, 7 - 11, 03.07.2022

Öz

The aim of this study is to compare Least Squares Regression (LSR) and Principal Components Regression (PCR) results when multicollinearity is determined in a dataset.In order to examine the effect of the degree of multicollinearity in the study, 10 datasets with different levels of multicollinearity were derived. Each data set consists of three independent and one dependent variable, and the variables were derived from the standard normal distribution. The multicollinearity status in the derived data has been demonstrated by the commonly used metrics to determine multicollinearity. Least Squares and Principal Components Regression was applied to all datasets.
When generating multicollinearity, all relationships were defined as positive in data simulation. However, the sign of the regression coefficients for the second (X2) and third (X3) independent variables were obtained as reverse (negative) as one of the expected effects of multicollinearity in Least Squares analysis. In the analysis of the Principal Components Regression, the sign of coefficients was found to be in the right direction (positive). The sign of the coefficients obtained from OLS and PCR were different and they also differed in magnitude. In addition, the standard errors of the coefficients in PCR results were lower than OLS results.
The existence of multicollinearity must be examined while performing multiple linear regression analysis, and if multicollinearity is determined, one of the methods that can solve this situation should be used. Otherwise, the estimations to be made as a result of regression may lead to wrong results. In line with the results of this study, it is recommended to use Principal Components Regression instead of Least Squares regression in case of multicollinearity in the data.

Kaynakça

• [1] F. Akdeniz, Olasılık ve istatistik: Akademisyen Kitabevi, 2015.
• [2] S. Şahinler, "En küçük kareler yöntemi ile doğrusal regresyon modeli oluşturmanın temel prensipleri," Mustafa Kemal Üniversitesi Ziraat Fakültesi Dergisi, vol. 5, pp. 57-73, 2000.
• [3] D. C. Montgomery, E. A. Peck, and G. G. Vining, Introduction to linear regression analysis vol. 821: John Wiley & Sons, 2012.
• [4] R. Özkale, "Çoklu İç İlişki ile İlgili Yöntemler," Doktora, Fen Bilimleri Enstitüsü, Çukurova Üniversitesi, 2007.
• [5] I. Corp, "IBM SPSS statistics for windows, version 22.0," Armonk, NY: IBM Corp, 2013.
• [6] R. Alpar, Spor, sağlık ve eğitim bilimlerinden örneklerle uygulamalı istatistik ve geçerlik-güvenirlik: Detay Yayıncılık, 2010.
• [7] K. Özdamar, "SPSS ile Biyoistatistik. Kaan Kitabevi.. 4," Baskı. Eskişehir, pp. 315-368, 2001.
• [8] J. Neter, M. H. Kutner, C. J. Nachtsheim, and W. Wasserman, Applied linear statistical models vol. 4: Irwin Chicago, 1996.
• [9] S. Cankaya, "A comparative study of some estimation methods for parameters and effects of outliers in simple regression model for research on small ruminants," Tropical animal health and production, vol. 41, pp. 35-41, 2009.
• [10] W. W. Hines, D. C. Montgomery, and D. M. G. C. M. Borror, Probability and statistics in engineering: John Wiley & Sons, 2008.
Yıl 2022, Cilt: 7 Sayı: 1, 7 - 11, 03.07.2022

Kaynakça

• [1] F. Akdeniz, Olasılık ve istatistik: Akademisyen Kitabevi, 2015.
• [2] S. Şahinler, "En küçük kareler yöntemi ile doğrusal regresyon modeli oluşturmanın temel prensipleri," Mustafa Kemal Üniversitesi Ziraat Fakültesi Dergisi, vol. 5, pp. 57-73, 2000.
• [3] D. C. Montgomery, E. A. Peck, and G. G. Vining, Introduction to linear regression analysis vol. 821: John Wiley & Sons, 2012.
• [4] R. Özkale, "Çoklu İç İlişki ile İlgili Yöntemler," Doktora, Fen Bilimleri Enstitüsü, Çukurova Üniversitesi, 2007.
• [5] I. Corp, "IBM SPSS statistics for windows, version 22.0," Armonk, NY: IBM Corp, 2013.
• [6] R. Alpar, Spor, sağlık ve eğitim bilimlerinden örneklerle uygulamalı istatistik ve geçerlik-güvenirlik: Detay Yayıncılık, 2010.
• [7] K. Özdamar, "SPSS ile Biyoistatistik. Kaan Kitabevi.. 4," Baskı. Eskişehir, pp. 315-368, 2001.
• [8] J. Neter, M. H. Kutner, C. J. Nachtsheim, and W. Wasserman, Applied linear statistical models vol. 4: Irwin Chicago, 1996.
• [9] S. Cankaya, "A comparative study of some estimation methods for parameters and effects of outliers in simple regression model for research on small ruminants," Tropical animal health and production, vol. 41, pp. 35-41, 2009.
• [10] W. W. Hines, D. C. Montgomery, and D. M. G. C. M. Borror, Probability and statistics in engineering: John Wiley & Sons, 2008.