AN ALTERNATIVE APPROACH TO SOLVE THE LAD-LASSO PROBLEM

Year 2016, Volume: 34 Issue: 3, 467 - 476, 01.09.2016

Esra Emiroğlu

Abstract

The least absolute deviation (LAD) regression is more robust alternative to the popular least squares (LS) regression whenever there are outliers in the response variable, or the errors follow a heavy-tailed distribution. The least absolute shrinkage and selection operator (LASSO) is a popular choice for shrinkage estimation and variable selection. By combining these two classical ideas, LAD-LASSO is an estimator which is able to perform shrinkage estimation while at the same time selecting the variables and is resistant to heavy-tailed distributions and outliers. The aim of this article is to reformulate LAD-LASSO problem to solve with the Simplex Algorithm, which is an area of Mathematical Programming.

Keywords

Regression, LAD, LASSO, LAD-LASSO, Mathematical Programming, Simplex Algorithm

References

• [1] Akaike, H. (1973) Information Theory and an Estimation of the Maximum Likelihood Principle. In 2nd International Symposium on Information Theory, eds. B. N. Petrov and F.Csaki, Budapest:Akademia Kiado, pp. 267-281.
• [2] Arslan, O. (2011) Weighted LAD-LASSO Method for Robust Parameter Estimation and Variable Selection in Regression. Computational Statistics and Data Analysis 56, 1952-1965.
• [3] Arthanari, T. S., Dodge, Y., (1993) Mathematical Programming in Statistics, John Wiley&Sons Inc., New York, USA.
• [4] Fan and Li, (2001), Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties, Journal of the American Statistical Association, 96, 1348-1360.
• [5] Friedman, J., Hastie, T., Tibshirani, R. (2001) The elements of statistical learning. New York: Springer Series in Statistics, 2001.
• [6] Hoerl, A. E. and Kennard, R. V. (1970) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67.
• [7] Huber, P. J. (1981) Robust Statistics. John Wiley & Sons Inc., New York, USA.
• [8] Knight, Fu, (2000), Asymptotic for Lasso-Type Estimators, The Annals of Statistics, 28, 1356-1378.
• [9] Montgomery, D. C., Peck, E. A., Vinig, G. G. (2001) Introduction to Linear Regression Analysis, 3th. Ed. John Wiley & Sons, Inc., USA.
• [10] Robert Tibshirani, (1996), Regression Shrinkage and Selection via the Lasso, Journal of the Royal Statistical Society, Series B Vol. 58, No. 1, pp 267-288.
• [11] Rosset, S., Zhu, J. (2004), Least Angle Regression: discussion. The Annals of Statistics 32, 469-475.
• [12] Schwarz, G. (1978), Estimating the Dimension of a Model. The Annals of Statistics, 6, 461-464.
• [13] Shi, P. and Tsai, C. L. (2002), Regression Model Selection a Residual Likelihood Approach. Journal of the Royal Statistical Society: Series B 64.2: 237-252.
• [14] Tibshirani, 1996, Regression Shrinkage and Selection via the LASSO, Journal of the Royal Statistical Society, Ser.B, 58, 923-941.
• [15] Wang, H., Li, G. and Jiang, G. (2007). Robust Regression Shrinkage and Consistent Variable Selection Through The LAD_LASSO. Journal of Business & Economic Statistics 25, 347-355.
• [16] Zhao, P. and Yu, B. (2006), On Model Selection Consistency Of Lasso. J. Mach. Learn. Res, 7, 2541-2563.