Adaptive functional kernel kNN regression via distance-exponent weighting

Abstract

We propose a family of kernel estimators k-nearest neighbor (kNN) for scalar-on-function regression based on a distance-exponent weighting scheme within the kNN neighborhood. Using the random radius kNN as a data-adaptive local scale, the method extends the classical functional kernel-kNN regression by introducing an additional parameter that controls the weight distribution among neighboring observations. Under standard assumptions on the functional design and a mild annulus-mass condition, pointwise consistency and explicit bias–variance bounds are established. Under local Hölder smoothness and polynomial small-ball behavior, the estimator achieves the usual convergence rate, while the exponent parameter affects only the leading constants. A leading bias expansion is derived, making the exponent’s influence explicit via weighted moments of the limiting distribution of normalized distances. In particular, the analysis identifies situations in which increasing the exponent can reduce the leading bias constant without altering the order of the variance. Practical selection of the tuning parameters is discussed through cross-validation over finite grids. The finite-sample performance of the proposed estimator is illustrated through simulation experiments and an application to spectrometric data.

Keywords

• convergence rates
• cross-validation
• distance-exponent weighting
• functional data analysis
• kernel kNN regression
• small-ball probability

References

1. [1] S. Arlot and A. Celisse, A survey of cross-validation procedures for model selection, Stat. Surv. 4, 40–79, 2010.
2. [2] M.K. Attouch, A. Laksaci and F. Rafaa, Estimation locale linéaire de la régression non paramétrique fonctionnelle par la méthode des k plus proches voisins, C. R. Math. 355(7), 824–829, 2017.
3. [3] F. Belarbi, S. Chemikh and A. Laksaci, Local linear estimate of the nonparametric robust regression in functional data, Stat. Probab. Lett. 134, 128–133, 2018.
4. [4] N. Belatrach, W. Bouabsa, M.K. Attouch and O. Fetitah, M-regression estimation with the k nearest neighbor smoothing under quasi-associated data in functional statistics, Appl. Appl. Math. 17(2), 333–365, 2023.
5. [5] N.H. Bingham, C.M. Goldie and J.L. Teugels, Regular Variation, Cambridge University Press, Cambridge, 1987.
6. [6] F. Burba, F. Ferraty and P. Vieu, Convergence de l’estimateur noyau des k plus proches voisins en régression fonctionnelle non paramétrique, C. R. Math. 346(5–6), 339–342, 2008.
7. [7] F. Burba, F. Ferraty and P. Vieu, k-Nearest neighbour method in functional nonparametric regression, J. Nonparametr. Stat. 21(4), 453–469, 2009.
8. [8] A. Chouaf and A. Laksaci, On the functional local linear estimate for spatial regression, Stat. Risk Model. 29(3), 189–214, 2012.

9. [9] A. Cuevas, A partial overview of the theory of statistics with functional data, J. Stat. Plan. Inference 147, 1–23, 2014.
10. [10] H.A. David and H.N. Nagaraja, Order Statistics, 3rd ed., Wiley, Hoboken, NJ, 2003.
11. [11] L. de Haan and A. Ferreira, Extreme Value Theory: An Introduction, Springer, New York, 2006.
12. [12] J. Demongeot, A. Laksaci, F. Madani and M. Rachdi, Functional data: local linear estimation of the conditional density and its application, Statistics 47(1), 26–44, 2013.
13. [13] F. Ferraty and P. Vieu, Nonparametric Functional Data Analysis: Theory and Practice, Springer, New York, 2006.
14. [14] S. Guenani, W. Bouabsa, O. Fetitah, M.K. Attouch and S. Khardani, Some asymptotic results of a kNN conditional mode estimator for functional stationary ergodic data, Commun. Stat. Theory Methods 54(10), 3094–3113, 2025.
15. [15] F.R. Hampel, E.M. Ronchetti, P.J. Rousseeuw and W.A. Stahel, Robust Statistics: The Approach Based on Influence Functions, John Wiley & Sons, 1986.
16. [16] A. Laksaci, M. Rachdi and S. Rahmani, Spatial modelization: local linear estimation of the conditional distribution for functional data, Spat. Stat. 6, 1–23, 2013.
17. [17] P. Massart, Concentration Inequalities and Model Selection, Springer, Berlin, 2007.
18. [18] E. Masry, Nonparametric regression estimation for dependent functional data: Asymptotic normality, Stochastic Process. Appl. 115(1), 155–177, 2005.
19. [19] J.O. Ramsay and B.W. Silverman, Functional Data Analysis, 2nd ed., Springer, New York, 2005.
20. [20] M. Shin and Y. Jeon, Adaptive functional nearest-neighbor regression under complex dependence structures, J. Multivar. Anal. 198, 105214, 2023.
21. [21] H. Wang and L. Zhao, Robust functional nonparametric regression with applications to contaminated functional data, Stat. Comput. 34(2), 55, 2024.