High-Dimensional Robust Estimation Methods in Contaminated Datasets and a Review of the Environmental Kuznets Curve Hypothesis in OECD Countries

Abstract

In high-dimensional datasets, it is a subject of investigation whether robust regression methods have high prediction and variable selection performance when vertical outliers and leverage points change from light-tailed errors to heavy-tailed errors. In this paper, we study the performance of six types of high-dimensional robust regression models (LAD-Lasso, Q-Lasso applied at the 0.25 and 0.75 quantile levels, Huber-Lasso, MTE-Lasso, RLARS, and Sparse-LTS) by creating both uncontaminated and contaminated control groups with different distributions of error terms. The methods are validated by comparing the estimation metrics at different scenarios through object-oriented simulations. Additionally, in this study, real data sets are examined and the performance of robust regression methods is evaluated to select the factors determining the CO2 emissions per capita of OECD countries under the EKC (Environmental Kuznets Curve) hypothesis. The results prove that, although simulations show that different methods perform well on different datasets with different contaminations, Spars LTS is superior to other methods. In addition, sparse LTS also performs better in terms of variable selection and prediction success in real OECD dataset with outliers and leverage points.

Keywords

• Outliers
• Leverage points
• High-dimensional robust regression
• Variable selection

References

1. Adomaityte, U., L. Defilippis, B. Loureiro and G. Sicuro (2024). High-Dimensional Robust Regression Under Heavy-Tailed Data: Asymptotics and Universality. Journal of Statistical Mechanics: Theory and Experiment, 2024(11), 114002.
2. Alfons, A. (2022). “Package‘perryExamples’: Examples for Integrating Prediction Error Estimation into Regression Models. https://cran.r-project.org/web/packages/perryExamples/refman/perryExamples.html
3. Alfons, A. (2022). Package ‘simFrame’: Simulation Framework. https://CRAN.R-project.org/package=simFrame
4. Alfons, A. and M. A. Alfons (2021). Package ‘robustHD’: An R Package for Robust Regression with High-Dimensional Data. Journal of Open-Source Software, 6, 1-61
5. Alfons, A., C. Croux and S. Gelper (2013). Sparse Least Trimmed Squares Regression for Analyzing High-Dimensional Large Data Sets. The Annals of Applied Statistics, 7(1), 226-248.
6. Amato, U., A. Antoniadis, I. De Feis, and I. Gijbels (2021). Penalised Robust Estimators for Sparse and High-Dimensional Linear Models. Statistical Methods & Applications, 30(1), 1-48.
7. Belloni, A., and V. Chernozhukov (2011). L1 -Penalized Quantile Regression in High-Dimensional Sparse Models. The Annals of Statistics, 39, 82-130.
8. Dalalyan, A. and P. Thompson (2019). Outlier-Robust Estimation of a Sparse Linear Model Using L1 Penalized Huber's M -Estimator. Advances in Neural Information Processing Systems, 32, 13208-13219.

9. Darwish, K. and A. H. Büyüklü (2015). Robust Linear Regression Using L1 -Penalized MM-Estimation for High Dimensional Data. American Journal of Theoretical and Applied Statistics, 4(3), 78-84.
10. Fan, J., Q. Li and Y. Wang (2014). Robust Estimation of High-Dimensional Mean Regression. arXiv:1410.2150
11. Fan, J., Q. Li and Y. Wang (2017). Estimation of High Dimensional Mean Regression in The Absence of Aymmetry and Light Tail Assumptions. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(1), 247–265.
12. Filzmoser, P. and M. Gschwandtner (2022). Package‘mvoutlier’: Multivariate Outlier Detection Based on Robust Methods.https://CRAN.R-project.org/package=mvoutlier
13. Genz, A., F. Bretz, T. Miwa, X. Mi, F. Leisch, F. Scheipl, B. Bornkamp and M. T. Hothorn (2020). Package ‘mvtnorm’. J. Comput. Graph. Stat., 11, 950-971.
14. Ghosh, A., M. Jaenada and L. Pardo (2020). Robust Adaptive Variable Selection in Ultra-High Dimensional Linear Regression Models. arXiv:2004.05470
15. Khorsand, B., J. Zahiri and A. Savadi (2016). Package ‘Evaluation Measures’: Collection of Model Evaluation Measure Functions. https://CRAN.Rproject.org/package=EvaluationMeasures
16. Li, S. (2022). Package ‘MTE’: Maximum Tangent Likelihood Estimation for Linear Regression. https://CRAN.R-project.org/package=perryExamples.
17. Luo, B. and X. Gao (2019). High-Dimensional Robust Approximated M-Estimators for Mean Regression with Asymmetric Data. arXiv:1910.09493
18. Mai, T. T. (2025a). Heavy Lasso: Sparse Penalized Regression Under Heavy-Tailed Noise via Data-Augmented Soft-Thresholding. arXiv preprint arXiv:2506.07790
19. Mai, T. T. (2025b). Exponential Lasso: Robust Sparse Penalization Under Heavy-Tailed Noise and Outliers with Exponential-Type Loss. arXiv preprint arXiv:2511.15332
20. Qin, Y., S. Li, Y. Li and Y. Yu (2017). Penalized Maximum Tangent Likelihood Estimation and Robust Variable Selection. arXiv:1708.05439
21. Smucler, E. and V. J. Yohai (2017). Robust and Sparse Estimators for Linear Regression Models. Computational Statistics & Data Analysis 111: 116-130.
22. Stefelova, N., A. Alfons, J. Palarea-Albaladejo, P. Filzmoser and K. Hron (2021). Robust Regression with Compositional Covariates Including Cellwise Outliers. Advances in Data Analysis and Classification 15: 869-909.
23. Sun Q, W. X. Zhou and J. Fan (2020). Adaptive Huber Regression. Journal of the American Statistical Association, 115, 254–265.
24. Toka, O., M. Çetin and O. Arslan (2021). Robust Regression Estimation and Variable Selection When Cellwise and Casewise Outliers are Present. Hacettepe Journal of Mathematics and Statistics, 50, 289-303.
25. Wang, Y. (2021). High-dimensional Generalized Robust Regression and Outlier Detection. PhD diss., Department of Mathematical and Statistical Sciences, University of Alberta.
26. Wei, Q. and Z. Zhao (2024, July). High-Dimensional Constrained Huber Regression. In 2024 IEEE 13rd Sensor Array and Multichannel Signal Processing Workshop (SAM) (pp. 1-5). IEEE.