A hard re-descending hybrid robust regression estimation technique using direct weights

Abstract

A hybrid approach of M and R estimators using an iterative procedure is proposed to detect outliers and estimation of regression parameters for linear models. We consider the deviation of each residual from its median to measure the likelihood of the corresponding data point to be an outlier. Also, the proposed work develops a reliable algorithm to estimate parameters of regression model that is unaffected by outliers. The significance of the proposed work is a novel hybrid approach of weighing the observations based on the order of residuals and is computationally simpler. Our proposal is illustrated using Monte Carlo simulation and analysed for few empirical benchmark data sets.

Keywords

• Robust regression
• hybrid estimator
• M-estimator
• R-estimator
• outlier detection
• Monte Carlo simulation

References

1. [1] D.F. Andrews, A robust method for multiple linear regression, Technometrics. 16 (4), 523-531, 1974.
2. [2] R. Baby, C.S. Kumar, K.K. George and A. Panda, Noise compensation in i-vector space using linear regression for robust speaker verification, 2017 International Conference on Multimedia, Signal Processing and Communication Technologies (IMPACT). 161-165, 2017.
3. [3] A.E. Beaton and J. WTukey, The fitting of power series, meaning polynomials, illustrated on band-spectroscopic data., Technometrics. 16 (2) 147-185, 1974.
4. [4] G.B. Begashaw and Y.B. Yohannes , Review of outlier detection and identifying using robust regression model, International Journal of Systems Science and Applied Mathematics. 5 (1), 4-11, 2020.
5. [5] D.Q.F. De Menezes, D.M. Prata, A.R. Secchi and J.C. Pinto, TA review on robust M-estimators for regression analysis, Comput. Chem. Eng. 147, 107254. 2021.
6. [6] F.Y. Edgeworth, On observations relating to several quantities, Hermathena. 6 (13), 279-285, 1887.
7. [7] L. Fu, Y.G. Wang and F. Cai, A working likelihood approach for robust regression, Stat. Methods Med. Res. 29 (12), 3641-3652, 2020.
8. [8] F.R. Hampel, The influence curve and its role in robust estimation, J. Am. Stat. Assoc. 69 (346), 383-393, 1974.

9. [9] D.M. Hawkins and D. Bradu, Location of several outliers in multiple-regression data using elemental sets, Technometrics. 26 (3), 197-208, 1984.
10. [10] S. Hekimolu and R.C. Erenoglu, A new GM-estimate with high breakdown point, Acta Geod. Geophys. 48, 419-437, 2013.
11. [11] P.W. Holland and R.E. Welsch, Robust regression using iteratively reweighted leastsquares, Commun. Stat. - Theory Methods 6 (9), 813-827, 1977.
12. [12] P.J. Huber and R.E. Welsch, Robust regression: Asymptotics, conjectures and monte carlo, Ann. Stat. 1 (5), 799-821, 1973.
13. [13] M. Hubert and M. Debruyne, Breakdown value , WIREs Comp Stat. 1, 296-302, 2009.
14. [14] L.A. Jaeckel, Estimating regression coefficients by minimizing the dispersion of the residuals, Ann. Math. Statist. 43 (5), 1449-1458, 1972.
15. [15] J. Jureckova, Nonparametric estimate of regression coefficients, Ann. Math. Statist. 42 (4), 1328-1338, 1971.
16. [16] J. Kalina, Regularized least weighted squares estimator in linear regression, Commun. Stat. - Simul. Comput. 2024.
17. [17] D.M. Khan, M. Ali, Z. Ahmad, S. Manzoor and S. Hussain, A new efficient redescending M-estimator for robust fitting of linear regression models in the presence of outliers, Math. Probl. Eng. 3090537, 2021.
18. [18] D.C. Montgomery, E.A. Peck and G.G. Vining, Introduction to Linear Regression Analysis, Fifth Edition, 2013.
19. [19] P.J. Rousseeuw and A.M. Leroy, Robust Regression and Outlier Detection, John wiley & sons. 2005.
20. [20] P.J. Rousseeuw and M. Hubert, Robust statistics for outlier detection, WIREs Data Mining Knowl Discov. 1, 73-79, 2011.
21. [21] P.J. Rousseeuw and M. Hubert, Anomaly detection by robust statistics, WIREs Data Mining Knowl Discov. 8, e1236, 2018.
22. [22] A. Shyna, C. UshadeviAmma, A. John, C. Kesavadas and B. Thomas, Deep-ASL enhancement technique in arterial spin labeling MRI - A novel approach for the error reduction of partial volume correction technique with linear regression algorithm, J. Comput. Sci. 58, 101546, 2022.
23. [23] R. Tibshirani, Regression shrinkage and selection via the lasso, J. R. Stat. Soc., B: Stat. Methodol. 58 (1), 267-288, 1996.
24. [24] Y.G. Wang, X. Lin, M. Zhu and Z. Bai, Robust estimation using the huber function with a data-dependent tuning constant, J. Comput. Graph. Stat. 16 (2), 468-481, 2007.
25. [25] J.W. Wisnowski, D.C. Montgomery and J.R. Simpson, A comparative analysis of multiple outlier detection procedures in the linear regression model, Comput. Stat. Data Anal. 36 (3), 351-382, 2001.
26. [26] C. Yu and W. Yao, Robust linear regression: A review and comparison, Commun. Stat. - Simul. Comput. 46 (8), 6261-6282, 2017.