ADAPTIVE BOOSTED ESTIMATION FOR SINGLE-INDEX QUANTILE REGRESSION

Year 2025, Volume: 15 Issue: 10, 2567 - 2583, 01.10.2025

Taha Alshaybawee Fadel Hamid Hadi Alhusseini Asaad Naser Hussein Mzedawee

Abstract

We propose a novel boosted estimation method for single-index quantile regression (SIQR) that combines the robustness of quantile regression with the flexibility of gradient boosting. By modeling the conditional quantile through a single linear index and a nonlinear link function, our method achieves effective dimension reduction while capturing complex relationships in the data. The procedure iteratively updates the index direction and fits base learners such as splines or regression trees to the pseudoresiduals from the quantile loss. This approach avoids multivariate smoothing, handles non-Gaussian errors, and adapts well to nonlinear structures. We establish theoretical guarantees, including consistency and optimal convergence rates under standard conditions. Extensive simulation studies and a real-data application demonstrate that the proposed method outperforms existing SIQR approaches in terms of accuracy and robustness.

Keywords

Quantile regression , Single-index model , Gradient boosting , semi-parametric quantile regression , Single-index quantile regression

References

• Reference1 Freund, Y.,Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1), 119-139.
• Reference2 Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of statistics, 1189-1232.
• Reference3 Koenker, R., Bassett Jr, G. (1978). Regression quantiles.Econometrica: journal of the Econometric Society, 33-50.
• Reference4 Ichimura, H. (1993). Semiparametric least squares (SLS) and weighted SLS estimation of single-index models. Journal of econometrics, 58(1-2), 71-120.
• Reference5 Hardle, W., Hall, P., Ichimura, H. (1993). Optimal smoothing in single-index models. The annals of Statistics,21(1), 157-178.
• Reference6 Yu, K., Jones, M. (1998). Local linear quantile regression. Journal of the American statistical Association, 93(441), 228-237.
• Reference7 Xia, Y., Tong, H., Li, W. K., Zhu, L. X. (2002). An adaptive estimation of dimension reduction space. Journal of the Royal Statistical Society Series B: Statistical Methodology, 64(3), 363-410.
• Reference8 Wu, Y., Liu, Y. (2009). Variable selection in quantile regression. Statistica Sinica, 801-817.
• Reference9 Buhlmann, P., Yu, B. (2003). Boosting with the L 2 loss: regression and classification. Journal of the American Statistical Association, 98(462), 324-339.
• Reference10 Fenske, N., Kneib, T., Hothorn, T. (2011). Identifying risk factors for severe childhood malnutrition by boosting additive quantile regression. Journal of the American Statistical Association, 106(494), 494-510.
• Reference11 Wu, Y., Yu, K., Yu, Y. (2010). Single-index quantile regression. Journal of Multivariate Analysis, 101(7), 1607–1621.
• Reference12 Kong, E., Linton, O., Xia, Y. (2010). Uniform Bahadur representation for local polynomial estimates of M-regression and its application to the additive model. Econometric Theory, 26(5), 1231–1266.
• Reference13 Buhlmann, P., Van De Geer, S. (2011). Statistics for high-dimensional data: methods, theory and applications. Springer Science Business Media.