A Comparison on Performances of Differential Evolution Algorithm and Genetic Algorithm in Determining the Biasing Parameter k of Ridge Regression

Abstract

Ridge Regression is a very common way of the remedies for dealing with the “multicollinearity problem” in multiple regression analysis. Although it can provide much more consistent estimates than the ordinary least squares does, there is still a problematic issue in the use of Ridge Regression, which is the choice of biasing parameter k. In this study we propose the use of some Artificial Intelligence Algorithms, such as genetic and differential evolution, for choosing the optimal k value by not allowing to increase too much the mean absolute prediction error while reducing the variation inflation factors and condition number.

Keywords

• Ridge Regression; Genetic Algorithm
• Differential Evolution Algorithm
• MAPE
• Variance İnflation Factor
• Condition Number

Ridge Regresyonda Yanlılık Parametresi k’nın Belirlenmesinde Genetik ve Differansiyel Gelişim Algoritmalarının Performanslarına Dair Bir Karşılaştırması

Abstract

Çoklu regresyonda karşılaşılan “çoklubağlantı” problem için en yaygın olarak önerilen yaklaşım Ridge Regresyondur. Ridge regresyon en küçük kareler yönteminden daha tutarlı tahminler sağlamasına rağmen yanlılık partametresi k’nın belirlenmesi hala çözülmesi gereken bir meseledir. Bu çalışmada optimal k değerini bulmak için Yapay Zeka Tekniklerinden olan Genetik Algoritma ve Diferansiyel Gelişim Algoritması ‘nın kullanımı önerilmiştir. Bu yaklaşımların uygulanmasında varyans büyütme faktörü ile şartlı sayı gibi çoklubağlantı probleminin teşhisinde kulanılan göstergeler küçültülmeye çalışılırken ortalama mutlak yüzdelik hatanın çok büyümemesini kontrol altında tutarak algoritmalar geliştirilmiştir.

Keywords

• Ridge Regresyon; Genetik Algoritma
• Differansiyal Gelişim Algoritması
• Ortalama Mutlak Yüzdelik Hata
• Varyans Büyütme Faktörü
• Şartlı Sayı

References

1. Ahn, J.J., Byun, H.W., Oh, K.J., and Kim, T.Y., (2012). “Using ridge regression with genetic algorithm to enhance real estate appraisal forecasting”, Expert Systems with Applications, 39, 8369–8379.
2. Belsley, D. A., Kuh, E. and Welsch, R. E., “Regression Diagnostics: Identifying Influential Data and Sources of Collinearity”, New York: John Wiley and Sons, 1980.
3. Chatterjee, S. and Hadi, A., “Regression analysis by example”, 4th edition, New York, 2006.
4. Gibbons, D. G. (1981). “A simulation stady of some ridge estimators”, Journal of the American Statistical Association, 76, 131-139.
5. Hoerl, A. E., (1962). “Application of ridge analysis to regression problems”, Chemical Engineering Progress, 58, 54-59.
6. Hoerl, A.E., and Kennard, R.W. (1976). “Ridge regression: iterative estimation of the biasing parameter”, Communication in Statistics, Part A5, 77-88.
7. Hoerl, A.E., and Kennard, R.W. (1970b). “Ridge regression: applications to non-orthogonal problems”, Technometrics, 12, 69-82.
8. Hoerl, A.E., Kennard, R.W., and Baldwin, K.F. (1975). “Ridge regression: some simulation”, Communication in Statistics, 4, 105-123.

9. Hoerl, A.E., and Kennard, R.W. (1970a). “Ridge regression: biased estimation for non-orthogonal problems”, Technometrics, 12, 55-67.
10. Kibria, B.M.G. (2003). “Performance of Some New Ridge Regression Estimators”, Communications in Statistics - Theory and Methods, 32, 419-435.
11. Longley J.W., (1967). “An appraisal of least squrares programs fort the electronic computer from point of view of the user”, Journal of The American Statistical Association, 62, 819-841.
12. Mardikyan, S., Cetin, E. (2008). "Efficient Choice of Biasing Constant for Ridge Regression", International Journal of Contemporary Mathematical Sciences, Vol: 3, No: 11, pp. 527-536.
13. Marquardt, D.W., and Snee, R.D., (1975). “Ridge regression in practice”, The American Statisticians, 29, 3-20.
14. Muniz G., Kibria, B. M. G., Mansson, K. and Shukur, G., (2012). “On developing ridge regression parameters:a graphical investigation”, Sort-Statistics And Operations Research Transactions, Volume 36, Issue 2, pages 115-138.
15. Prago-Alejo, R.J., Torre-Trevino, L.M., and Pina-Monarrez M.R., (2008). “Optimal determination of k constant of ridge regression using a simple genetic algorithm”, Electronics robotics and Automotive Mechanics Conference.
16. Shukur, G. and Khalaf, G. (2005). “Choosing Ridge Parameters for. Regression Problems”, Communication in Statistics – Theory and Methods, 34, 1177-1182.
17. Uslu V. R., Eğrioğlu E. and Baş E., (2014). “Finding Optimal Value for the Shrinkage Parameter in Ridge Regression via Particle Swarm Optimization”, American Journal of Intelligent Systems, Volume 4, Number 4, pages 142-147.
18. Storn, R. and Price K., (1997). “Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces”, Journal of Global Optimization, 11: 341–359.
19. Wooldridge, J. M., “Introductory Econometrics: A Modern Approach”, South Western, 2000.
20. Vinod, H.D., (1976). “Application of ridge regression methods to a study of Bell System scale economics”, Journal of the American Statistical Association, 71, 835-841.
21. Wooldridge, J. M., (2000). “A framework for estimating dynamic, unobserved effects panel data models with possible feedback to future explanatory variables”, Economic letters, 68, 245-250.