Comparison of Ridge Regression and Least Squares Methods in the Presence of Multicollinearity for Body Measurements in Saanen Kids

Year 2020, Volume: 10 Issue: 2, 1429 - 1437, 01.06.2020

Cem Tırınk , Samet Hasan Abacı , Hasan Onder

https://doi.org/10.21597/jist.671662

Cited By: 2

Abstract

Least square (LS) method is a common method used to estimate the coefficients in multiple regression models. The least square multiple regression models produce biased regression coefficients when the multicollinearity is encountered in the studied data sets. Multicollinearity problem can be solved by using some methods. As one of the methods, Ridge Regression (RR) is a biased estimation method that enables to obtain models having more reliable coefficient of determination (R2). This study was conducted on 40 Saanen kids in order to determine some morphological measurements (withers height, rump height, body length, chest width, chest girth and chest depth) affecting body weight. In this study, usability of ridge regression method in the presence of multicollinearity was evaluated. Variance Inflation Factor (VIF) values higher than 10 were detected for withers height and rump height. Coefficient of determination (R2) was obtained as 0.88 from LS method and R2 was obtained 0.875 with k=0.0136 from RR method. As a result, the model obtained from RR is more reliable than that obtained from LS.

Keywords

Least squares, multiple regression, Multicollinearity, Ridge Regression

Thanks

This research was presented as an oral presentation at the International 8th Balkan Animal Science Conference (BALNIMALCON) held on 6-8 September 2017 in Prizren.

References

• Akcay A, Sariozkan A, 2015. Estimation of income with using Ridge Regression analysis in layer hen industry. Ankara Üniversitesi Veteriner Fakültesi Dergisi, 62, 69-74, 2015.
• Albayrak AS, 2005. An alternative bias estimation technique and an application of the least-squares technique in multiple linear connections. Zonguldak Karaelmas University Journal Social Sciences1: 105-126.
• Alpar R, 2011. Uygulamalı çok değişkenli istatistiksel yöntemler. 3. Baskı. Kızılay/Ankara. Detay Yayımcılık. ISBN:978-605-5437-42-8
• Anderson B, 1998. Scandinavian evidence on growth and age structure, ESPE 1997 Conference at Uppsala University.
• Ari A, Onder H, 2013. Regression Models Used for Different Data Structures. Anadolu Journal of Agricultural Sciences, 2013,28(3):168-174. doi: 10.7161/anajas.2013.28.3.168.
• Aytekin I, Eyduran E, Karadas K, Aksahan R, Keskin I, 2018. Prediction of Fattening Final Live Weight from some Body Measurements and Fattening Period in Young Bulls of Crossbred and Exotic Breeds using MARS Data Mining Algorithm. Pakistan Journal of Zoology, vol. 50(1), pp 189-195, 2018.
• Cankaya S, Eker S, Abaci, SH, 2019. Comparison of Least Squares, Ridge Regression and Principal Component Approaches in the Presence of Multicollinearity in Regression Analysis. Turkish Journal of Agriculture-Food Science and Technology, 7(8): 1166-1172. DOI: 10.24925/turjaf.v7i8.1166-1172.2515
• Ergüneş E, 2004. The examing least square method and Ridge regression method by comparison. Cukurova University Graduate School of Natural and Applied Sciences, Master Thesis (Printed).
• Eyduran E, Zaborski D, Waheed A, Celik S, Karadas K, Grzesiak W, 2017. Comparison of the predictive Capabilities of several data mining algorithms and multiple linear regression in the prediction of body weight by means of body measurements in the indigenous beetal goat of Pakistan. Pakistan Journal of Zoology, 49: 257-265. https://doi.org/10.17582/journal.pjz/2017.49.1.257.265.
• Gujarati DN, 1995. Basic econometrics, 3rd ed. McGraw-Hill, New York, USA.
• Hair JF Jr, Black WC, Babin BJ, Anderson RE, (2014). Multivariate Data Analysis (7th edition), pp. 200, ISBN 10: 1-292-02190-X, Pearson Education Limited -England.
• Hoerl AE, Kennard R, 1970. Ridge regression: Biased estimation for non-orthogonal problems. Technometrics, 12: 55-67. https://doi.org/10.1080/00401706.1970.10488635
• IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.
• Kurtuluş M, 2001. A study on ridge regression. Gazi University Graduate School of Natural and Applied Sciences, Master Thesis (Printed).
• Kutner MH, Nachtsheim CJ, Neter J, Li W, 2004. Applied Linear Statistical Models, 5th edition, pp. 15, McGraw‐Hill/Irwin.
• Lukuyu MN, Gipson JP, Savage DB, Duncan AJ, Mujibi FBN, Okeyo AM, 2016. Use of body linear measurements to estimate live weight of crossbred dairy cattle in small holder farms in Kenya. SpringerPlus, 5:3-14. https://doi.org/10.1186/s40064-016-1698-3.
• NCSS 11 Statistical Software trial version, 2016. NCSS, LLC. Kaysville, Utah, USA, ncss.com/software/ncss.
• Pagel MU, Lunneborg CE, 1985. Empirical evaluation of Ridge Regression. Psychological Bulletin, 97: 342-355. https://doi.org/10.1037/0033-2909.97.2.342.
• Rathert ÇT, Üçkardeş F, Narinç D, Aksoy T, 2011. Comparison of principal component regression with the least square method in prediction of internal egg quality characteristics in Japanese quails. Kafkas Universitesi Veteriner Fakültesi Dergisi, 17:687-692.
• Sahin M, Yavuz E, Uckardes F, 2018. Multicollinearity Problem and Bias Estimates in Japanese Quail. Pakistan Journal of Zoology, vol. 50(2), pp 757-761, 2018. DOI: 10.17582/journal.pjz/2018.50.2.757.761.
• Sarstedt M, Mooi E, 2014. A Concise Guide to Market Research, Springer Texts in Business and Economics: Chapter 7, Regression Analysis pp. 193-233. DOI 10.1007/978-3-642-53965-7_7, #Springer-Verlag Berlin Heidelberg.
• Topal M, Eyduran E, Yaganoglu AM, Sonmez AY, Keskin S, 2010. Use of Ridge and Principal Component Regression Analysis Methods in Multicollinearity. Journal of Agricultural Faculty of Atatürk University, 41 (1), 53-57. ISSN: 1300-9036.
• Uckardes F, Efe E, Narinc D, Aksoy T, 2012. Estimation of the egg albumen index in the Japanese quails with ridge regression method. Akademik Ziraat Dergisi 1(1): 11-20. ISSN: 2147-6403.
• Vupa O, Gurunlu Alma O, 2008. Investigation of Multicollinearity Problem in Small Samples Included Outlier Value in Linear Regression Analysis. Selcuk University Journal of Science Faculty. Vol.31, 97-107.