Gini-weighted class of ridge estimators in linear regression with heteroscedastic errors

Abstract

The joint presence of multicollinearity and heteroscedasticity substantially deteriorates the performance of ordinary least squares and ridge estimators. To address this, a robust heteroscedasticity-adjusted ridge estimator is proposed that incorporates a data-dependent scaling factor based on the Gini index. The proposed scaling factor is integrated into the traditional ridge parameter to mitigate unjust penalization arising due to heteroscedasticity and severe multicollinearity. Monte Carlo simulation across varying sample sizes, dimensionalities, multicollinearity levels, and degrees of heteroscedasticity shows a substantial reduction in mean squared error, with the most notable improvements observed in scenarios where predictors are highly correlated and error variances exhibit extreme heteroscedasticity. Further, the proposed methodology is applied to the Auto-MPG and Pakistan Economic datasets, which demonstrate the challenging characteristics addressed in this study. The PRESS confirms that the Gini-adjusted ridge parameter achieves superior predictive accuracy and more stable coefficient estimates compared to existing approaches.

Keywords

• Gini index
• heteroscedasticity
• multicollinearity
• ridge regression
• scaling factor

References

1. [1] A. A. Al-Majali, The effect of democracy and income inequality (Gini index), Jordan. J. Law Polit. Sci. 15 (4), 2023.
2. [2] O. O. Alabi, K. Ayinde, O. E. Babalola, H. A. Bello and E. C. Okon, Effects of multicollinearity on type-I error of some methods of detecting heteroscedasticity in linear regression model, Open J. Stat. 10 (4), 664–677, 2020.
3. [3] Z. Y. Algamal, A new method for choosing the biasing parameter in ridge estimator for generalized linear model, Chemom. Intell. Lab. Syst. 183, 96–101, 2018.
4. [4] M. Alheety and G. B. Kibria, On the Liu and almost unbiased Liu estimators in the presence of multicollinearity with heteroscedastic or correlated error, Surv. Math. Appl. 4, 155–164, 2009.
5. [5] M. A. Alkhamisi, Ridge estimation in linear models with heteroskedastic errors, Sankhya B 74 (2), 302–314, 2012.
6. [6] M. A. Alkhamisi, G. Khalaf and G. Shukur, Modified generalized ridge regression estimators, Commun. Stat. - Theory Methods 35 (1), 105–118, 2006.
7. [7] D. M. Allen, The relationship between variable selection and data augmentation and a method for prediction, Technometrics 16 (1), 125–127, 1974.
8. [8] I. L. Amerise, Iterative weighted LAD estimation with homoskedasticity testing using the Gini concentration index, Comput. Stat., 2025.

9. [9] M. Aslam, Performance of Kibria’s method for the heteroscedastic ridge regression model: Some Monte Carlo evidence, Commun. Stat. - Simul. Comput. 43 (4), 673– 686, 2014.
10. [10] M. Aslam, Using heteroscedasticity-consistent standard errors for the linear regression model with correlated regressors, Commun. Stat. - Simul. Comput. 43 (10), 2353–2373, 2014.
11. [11] D. A. Belsley, E. Kuh and R. E. Welsch, Regression Diagnostics: Identifying Influential Data and Sources of Collinearity, Wiley Series in Probability and Statistics, John Wiley & Sons, Inc., Hoboken, New Jersey, 2005.
12. [12] C. Brown, Comparing health inequality measures, Int. J. Equity Health 3 (1), 19–27, 1994.
13. [13] I. S. Dar, S. Chand, M. Shabbir and B. G. Kibria, Condition-index based new ridge regression estimator for linear regression model with multicollinearity, Kuwait J. Sci. 50 (2), 91–96, 2023.
14. [14] A. V. Dorugade, Improved ridge estimator in linear regression with multicollinearity, heteroscedastic errors and outliers, J. Mod. Appl. Stat. Methods 15 (2), 362–381, 2016.
15. [15] D. E. Farrar and R. R. Glauber, Multicollinearity in regression analysis: The problem revisited, Rev. Econ. Stat. 49 (1), 92–107, 1967.
16. [16] Finance Division, Government of Pakistan, Pakistan Economic Survey 2020–21, Economic Adviser’s Wing, Finance Division, Islamabad, https://finance.gov.pk/ survey_2021.html, accessed: 2026-05-05.
17. [17] Finance Division, Government of Pakistan, Pakistan Economic Survey Archive, https://finance.gov.pk/survey_archieve.html, accessed: 2026-05-05.
18. [18] D. G. Gibbons, A simulation study of some ridge estimators, J. Am. Stat. Assoc. 76 (373), 131–139, 1981.
19. [19] C. Gini, Variabilità e mutabilità, contributo allo studio delle distribuzioni e relazioni statistiche, Studi Economico-Giuridici della R. Università di Cagliari 3, 1–10, 1912.
20. [20] G. H. Golub, M. Heath and G. Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics 21 (2), 215–223, 1979.
21. [21] W. H. Greene, Econometric Analysis, Prentice Hall, Upper Saddle River, New Jersey, 5th ed., 2003.
22. [22] D. N. Gujarati, Basic Econometrics, Tata McGraw-Hill Education, New Delhi, India, 2009.
23. [23] R. F. Gunst and J. T. Webster, Regression analysis and problems of multicollinearity, Commun. Stat. 4 (3), 277–292, 1975.
24. [24] A. E. Hoerl, R. W. Kennard and K. F. Baldwin, Ridge regression: Some simulations, Commun. Stat. - Simul. Comput. 4 (2), 105–123, 1975.
25. [25] A. E. Hoerl and R. W. Kennard, Ridge regression: Biased estimation for nonorthogonal problems, Technometrics 12 (1), 55–67, 1970.
26. [26] N. H. Jadhav and D. N. Kashid, A jackknifed ridge M-estimator for regression model with multicollinearity and outliers, J. Stat. Theory Pract. 5 (4), 659–673, 2011.
27. [27] G. Khalaf, K. Mansson and G. Shukur, Modified ridge regression estimators, Commun. Stat. - Theory Methods 42 (8), 1476–1487, 2013.
28. [28] G. Khalaf and G. Shukur, Choosing ridge parameter for regression problems, Commun. Stat. - Theory Methods 34 (5), 1177–1182, 2005.
29. [29] B. M. G. Kibria, Performance of some new ridge regression estimators, Commun. Stat. - Simul. Comput. 32 (2), 419–435, 2003.
30. [30] B. M. G. Kibria and A. F. Lukman, A new ridge-type estimator for the linear regression model: Simulations and applications, Scientifica 2020, 9758378, 2020.
31. [31] J. Lawless and P. Wang, A simulation study of ridge and other regression estimators, Commun. Stat. - Theory Methods 5 (4), 307–323, 1976.
32. [32] Y. Li and H. Yang, On the performance of the jackknifed modified ridge estimator in the linear regression model with correlated or heteroscedastic errors, Commun. Stat. - Theory Methods 40 (15), 2695–2708, 2011.
33. [33] J. S. Long and L. H. Ervin, Using heteroskedasticity consistent standard errors in the linear regression model, Am. Stat. 54 (3), 217–224, 2000.
34. [34] M. Lorenz, Methods of measuring the concentration of wealth, Publ. Am. Stat. Assoc. 9, 209–219, 1905.
35. [35] A. F. Lukman and A. Olatunji, Newly proposed biased ridge estimator: an application to the Nigerian economy, Pak. J. Stat. 34 (2), 91–98, 2018.
36. [36] G. C. McDonald and D. I. Galarneau, A Monte Carlo evaluation of some ridge-type estimators, J. Am. Stat. Assoc. 70 (350), 407–416, 1975.
37. [37] Pakistan Bureau of Statistics, National Accounts, https://www.pbs.gov.pk/ national-accounts/, accessed: 2026-05-05.
38. [38] D. Dua and C. Graff, UCI Machine Learning Repository: Auto MPG dataset, University of California, Irvine, School of Information and Computer Sciences, 2019, https://archive.ics.uci.edu/dataset/9/auto+mpg.
39. [39] R. Quinlan, Combining instance-based and model-based learning, in: Proceedings of the Tenth International Conference on Machine Learning, 236–243, Amherst, MA, Morgan Kaufmann, 1993.
40. [40] RStudio Team, RStudio: Integrated Development Environment for R, Posit Software, PBC, Boston, MA, 2023.
41. [41] A. M. E. Saleh, B. G. Kibria and F. George, Comparative study of lasso, ridge regression, preliminary test and Stein-type estimators for the sparse Gaussian regression model, Stat. Optim. Inf. Comput. 7 (4), 626–667, 2019.
42. [42] A. K. Sen, On Economic Inequality, Clarendon Press, Oxford, 1973.
43. [43] M. Shabbir, S. Chand and I. S. Dar, Heteroscedastic-adjusted standard error based estimation of ridge parameter in the linear regression model, Commun. Stat. - Theory Methods, 1–15, 2025.
44. [44] M. Shabbir, S. Chand and F. Iqbal, A new ridge estimator for linear regression model with some challenging behavior of error term, Commun. Stat. - Simul. Comput. 53 (11), 5442–5452, 2024.
45. [45] M. Suhail, S. Chand and B. M. G. Kibria, Quantile based estimation of biasing parameters in ridge regression model, Commun. Stat. - Simul. Comput. 49 (10), 2732–2744, 2020.
46. [46] G. Trenkler, On the performance of biased estimators in the linear regression model with correlated or heteroscedastic errors, J. Econom. 25 (1–2), 179–190, 1984.
47. [47] S. Türkan and G. Özel, A new modified jackknifed estimator for the Poisson regression model, J. Appl. Stat. 43 (10), 1892–1905, 2016.
48. [48] S. Türkan and G. Özel, A jackknifed estimators for the negative binomial regression model, Commun. Stat. - Simul. Comput. 47 (6), 1845–1865, 2018.
49. [49] H. White, A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity, Econometrica 48 (4), 817–838, 1980.
50. [50] S. Yasin, S. Salem, H. Ayed, S. Kamal, M. Suhail and Y. A. Khan, Modified robust ridge M-estimators in two-parameter ridge regression model, Math. Probl. Eng. 2021, 1–24, 2021.