A Review of Kernel Density Estimation with Applications to Econometrics

Abstract

Nonparametric density estimation is of great importance when econometricians want to model the probabilistic or stochastic structure of a data set. This comprehensive review summarizes the most important theoretical aspects of kernel density estimation and provides an extensive description of classical and modern data analytic methods to compute the smoothing parameter. Throughout the text, several references can be found to the most up-to-date and cut point research approaches in this area, while econometric data sets are analyzed as examples. Lastly, we present SIZer, a new approach introduced by Chaudhuri and Marron (2000), whose objective is to analyze the visible features representing important underlying structures for different bandwidths.

Keywords

• Nonparametric Density Estimation, SiZer, Plug-In Bandwidth Selectors, Cross- Validation, Smoothing Parameter.

References

1. Ahmad, I.A. and M. Amezziane (2007). A general and fast convergent bandwidth selection method of kernel estimator. Journal of Nonparametric Statistics, 19, 165˗187.
2. Altman, N. and C. Leger (1995). Bandwidth selection for kernel distribution function estimation. Journal of Statistical Planning and Inference, 46, 195˗214.
3. Berg, A. and D. Politis (2009). Cdf and survival function estimation with infinite order kernels. Electronic Journal of Statistics, 3, 1436˗1454.
4. Bhattacharya, P. (1967). Estimation of a probability density function and its derivatives, Sankhyii Ser. A, 29, 373˗382.
5. Bickel, P. J. and M. Rosenblatt (1973). On some global measures of the deviations of density function estimates. The Annals of Statistics, 1071˗1095.
6. Bierens, H.J. (1987). Kernel estimators of regression functions. In Advances in Econometrics: Fifth World Congress, Vol.I, Cambridge University Press 99˗144.
7. Bowman, A.W. (1984). An alternative method of cross-validation for the smoothing of density estimates. Biometrika, 71, 353˗360.
8. Bowman, A.W., P. Hall and T. Prvan (1998). Bandwidth selection for the smoothing of distribution function. Biometrika, 85, 799˗808.

9. Bowman, A.W., P. Hall and D.M. Titterington (1984). Cross-validation in nonparametric estimation of probabilities and probability densities. Biometrika, 71, 341˗351.
10. Breiman, L., W. Meisel and E. Purcell (1977). Variable kernel estimates of multivariate densities. Technometrics, 19, 135˗144.
11. Cai, Q., G. Rushton, and B. Bhaduri (2012). Validation tests of an improved kernel density estimation method for identifying disease clusters. Journal of Geographical Systems, 14 (3), 243˗264.
12. Cao, R., A. Cuevas and W. Gonzalez-Manteiga (1994). A comparative study of several smoothing methods in density estimation. Computational Statistics & Data Analysis, 17 (2), 153˗176.
13. Chan, N.-H., T.C. Lee and L. Peng (2010). On nonparametric local inference for density estimation. Computational Statistics & Data Analysis, 54, 509˗515.
14. Chaudhuri, P. and J.S. Marron (1999). Sizer for exploration of structures in curves. Journal of the American Statistical Association, 94, 807˗823.
15. Chaudhuri, P. and J.S. Marron (2000). Scale space view of curve estimation. The Annals of Statistics, 28, 402˗428.
16. Chen, S. (2000). Probability density function estimation using gamma kernels. Annals of the Institute of Statistical Mathematics, 52, 471˗480.
17. Cheng, M.-Y. (1997). A bandwidth selector for local linear density estimators. The Annals of Statistics, 25, 1001˗1013.
18. Cheng, M.-Y., E. Choi, J. Fan and P. Hall (2000). Skewing-methods for two parameter locally-parametric density estimation. Bernoulli, 6, 169˗182.
19. Choi, E. and P. Hall (1999). Data sharpening as prelude to density estimation. Biometrika, 86,
20. Choi, E., P. Hall and V. Roussan (2000). Data sharpening methods for bias reduction in nonparametric regression. Annals of Statistics, 28, 1339˗1355.
21. Chu, H.-J., C.-J. Liau, C.-H. Lin and B.-S. Su (2012). Integration of fuzzy cluster analysis and kernel density estimation for tracking typhoon trajectories in the Taiwan region. Expert Systems with Applications, 39, 9451˗9457.
22. Comte, F. and V. Genon-Catalot (2012). Convolution power kernels for density estimation. Journal of Statistical Planning and Inference, 142, 1698˗1715.
23. Delaigle, A. and I. Gijbels (2004). Bootstrap bandwidth selection in kernel density estimation from a contaminated sample. Annals of the Institute of Statistical Mathematics, 56 (1), 19˗47.
24. Devroye, L. and T. Wagner (1980). The strong uniform consistency of kernel density estimates. In Multivariate Analysis, Vol. V, ed. P.R. Krishnaiah, Amsterdam: North- Holland, 59˗77.
25. Duin, R.P.W. (1976). On the choice of smoothing parameters of parzen estimators of probability density functions. IEEE Transactions on Computers C-25, 1175˗1179.
26. Einmahl, U. and D.M. Mason (2005). Uniform in bandwidth consistency of kernel-type function estimators. The Annals of Statistics, 33, 1380˗1403.
27. Faraway, J. and M. Jhun (1990). Bootstrap choice of bandwidth for density estimation. Journal of the American Statistical Association, 85, 1119˗1122.
28. Gine, E. and A. Guillou (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Annales de l’Institut Henri Poincare (B) Probability and Statistics, 38, 907˗921.
29. Gine, E. and H. Sang (2010). Uniform asymptotics for kernel density estimators with variable bandwidths. Journal of Nonparametric Statistics, 22, 773˗795.
30. Golyandina, N., A. Pepelyshev and A. Steland (2012). New approaches to nonparametric density estimation and selection of smoothing parameters. Computational Statistics and Data Analysis, 56, 2206˗2218.
31. Habbema, J.D.F., J. Hermans and K. van den Broek (1974). A stepwise discrimination analysis program using density estimation. IN Proceedings in Computational Statistics. Vienna: Physica Verlag.
32. Hall, P. (1983). Large sample optimality of least squares cross-validation in density estimation. Annals of Statistics, 11, 1156˗1174.
33. Hall, P. (1990). Using the bootstrap to estimate mean squared error and select smoothing parameter in nonparametric problems. Journal of Multivariate Analysis, 32, 177˗203.
34. Hall, P. (1992). On global properties of variable bandwidth density estimators. The Annals of Statistics, 20, 762˗778.
35. Hall, P. and J.S. Marron (1987). Estimation of integrated squared density derivatives. Statistics & Probability Letters, 6, 109˗115.
36. Hall, P. and M. Minnotte (2002). High order data sharpening for density estimation. Journal of the Royal Statistical Society Series B, 64, 141˗157.
37. Hall, P. and M. Wand (1996). On the accuracy of binned kernel density estimators. Journal of Multivariate Analysis, 56, 165˗184.
38. Hall, P., S.J. Sheather, M.C. Jones and J.S. Marron (1991). On optimal data-based bandwidth selection in kernel density estimation. Biometrika, 78, 263˗269.
39. Hardle, W. (1991). Smoothing Techniques, With Implementations in S. New York: Springer.
40. Hazelton, M. (1996). Bandwidth selection for local density estimators. Scandinavian Journal of Statistics, 23, 221˗232.
41. Hazelton, M. (1999). An optimal local bandwidth selector for kernel density estimation. Journal of Statistical Planning and Inference, 77, 37˗50.
42. Hirukawa, M. (2010). Nonparametric multiplicative bias correction for kernel-type density estimation on the unit interval. Computational Statistics and Data Analysis, 54, 473˗495.
43. Holmstrom, L. (2000). The accuracy and the computational complexity of a multivariate binned kernel density estimator. Journal of Multivariate Analysis, 72, 264˗309.
44. Janssen, P., J. Swanepoel and N. Veraberbeke (2007). Modifying the kernel distribution function estimator towards reduced bias. Statistics, 41, 93˗103.
45. Jones, M.C. (1989). Discretized and interpolated kernel density estimates. Journal of the American Statistical Association, 84, 733˗741.
46. Jones, M., O. Linton and J. Nielsen (1995). A simple bias reduction method for density estimation. Biometrika, 82, 327˗328.
47. Liao, J., Y. Wu and Y. Lin (2010). Improving sheather and jones bandwidth selector for difficult densities in kernel density estimation. Journal of Nonparametric Statistics, 22, 105˗114.
48. Loader, C.R. (1999). Bandwidth selection: Classical or plug-in? The Annals of Statistics, 27 (2), 415˗438.
49. Loftsgaarden, D.O. and C.P. Quesenberry (1965). A nonparametric estimate of a multi- variate density function. The Annals of Mathematical Statistics, 36, 1049˗1051.
50. Loh, J.M. and W. Jang (2010). Estimating a cosmological mass bias parameter with bootstrap bandwidth selection. Journal of the Royal Statistical Society Series C, 59, 761˗779.
51. Marron, J.S. (1987). An asymptotically efficient solution to the bandwidth problem of kernel density estimation. The Annals of Statistics, 13, 1011˗1023.
52. Marron, J.S. (1989). Comments on a data based bandwidth selector. Computational Statistics & Data Analysis, 8, 155˗170.
53. Marron, J.S. and S.S. Chung (1997). Presentation of smoothers: the family approach. unpublished manuscript.
54. Matuszyk, T.I., M.J. Cardew-Hall and B.F. Rolfe (2010). The kernel density estimate/point distribution model (kde-pdm) for statistical shape modeling of automotive stampings and assemblies. Robotics and Computer-Integrated Manufacturing, 26, 370˗380.
55. Miao, X., A. Rahimi and Rao, R.P. (2012). Complementary kernel density estimation. Pattern Recognition Letters, 33, 1381˗1387.
56. Minnotte, M.C. (1999). Achieving higher-order convergence rates for density estimation with binned data. Journal of the American Statistical Association, 93, 663˗672.
57. Mnatsakanov, R. and F. Ruymgaart (2012). Moment-density estimation for positive random variables. Statistics, 46, 215˗230.
58. Mnatsakanov, R. and K. Sarkisian (2012). Varying kernel density estimation on +. Statistics and Probability Letters, 82, 1337˗1345.
59. Moore, D. and J. Yackel (1977). Consistency properties of nearest neighbour density function estimators. The Annals of Statistics, 5, 143˗154.
60. Nadaraya, E.A. (1964). On estimating regression. Theory of Probability & Its Applications, 9, 141˗142.
61. Nadaraya, E. A. (1965). On nonparametric estimates of density functions and regression curves, Theory Probab. Appl. 10, 186˗190.
62. Parzen, B.U. and J.S. Marron (1990). Comparison of data-driven bandwidth selectors. Journal of the American Statistical Association, 85, 66˗72.
63. Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065˗1076.
64. Pawlak, M. and U. Stadtmuller (1999). Kernel density estimation with generalized binning. Scandinavian Journal of Statistics, 26, 539˗561.
65. Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. Annals of Mathematical Statistics, 27, 832˗837.
66. Rudemo, M. (1982). Empirical choice of histograms and kernel density estimators. Scandinavian Journal of Statistics, 9, 65˗78.
67. Sarda, P. (1993). Smoothing parameter selection for smooth distribution functions. Journal of Statistical Planning and Inference, 35, 65˗75.
68. Savchuk, O., J. Hart and S. Sheather (2010). Indirect cross-validation for density estimation. Journal of the American Statistical Association, 105, 415˗423.
69. Scaillet, O. (2004). Density estimation using inverse and reciprocal inverse gaussian kernels. Journal of Nonparametric Statistics, 16, 217˗226.
70. Scott, D.W. (1981). Using computer-binned data for density estimation In Computer Science and Statistics: Proceedings of the 13th Symposium on the Interface. Ed. W.F. Eddy. New York: Springer-Velag, 292˗294.
71. Scott, D.W. (1992). Multivariate density estimation: Theory, practice, and visualization. John Wiley & Sons.
72. Scott, D.W. and G.R. Terrell (1987). Biased and unbiased cross-validation in density estimation. Journal of American Statistical Association, 82, 1131˗1146.
73. Scott, D.W., R.A. Tapia and J.R. Thompson (1977). Kernel density estimation revisited. Nonlinear Analysis, Theory, Methods and Applications, 1, 339˗372.
74. Sheather, S. and M. Jones (1991). A reliable data-based bandwidth selection method for kernel density estimation. Journal of the Royal Statistical Society – B, 53, 683˗690.
75. Silverman, B.W. (1978). Weak and strong uniform consistency of the kernel estimate of a density and its derivatives. The Annals of Statistics, 6, 177˗184.
76. Silverman, B.W. (1981). Using kernel density estimates to investigate multimodality. Journal of the Royal Statistical Society – B, 43, 97˗99.
77. Silverman, B.W. (1982). Kernel density estimation using the fast fourier transform. Applied Statistics, 31, 93˗97.
78. Silverman, B.W. (1986). Density Estimation for Statistics and Data Analysis. London: Chapman & Hall.
79. Stone, C.J. (1984). An asymptotically optimal window selection rule for kernel density estimates. Annals of Statistics, 12, 1285˗1297.
80. Stute, W. (1982). A law of the logarithm for kernel density estimators. The Annals of Probability, 10 (2), 414˗422.
81. Taron, M., N. Paragios and M.P. Jolly (2005). Modeling shapes with uncertainties: Higher order polynomials, variable bandwidth kernels and non parametric density estimation. 10th IEEE International Conference on Computer Vision, 1659˗1666.
82. Tenreiro, C. (2006). Asymptotic behavior of multistage plug-in bandwidth selections for kernel distribution function estimators. Journal of Nonparametric Statistics, 18,
83. Turlach, B.A. (1993). Bandwidth selection in kernel density estimation: A review. CORE and Institut de Statistique.
84. Van Ryzin, J. (1969). On strong consistency of density estimates. The Annals of Mathematical Statistics, 40 (486), 1765˗1772.
85. Wu, T.-J., C.-F. Chen and H.-Y. Chen (2007). A variable bandwidth selector in multivariate kernel density estimation. Statistics & Probability Letters, 77 (4), 462˗467.
86. Ziegler, K. (2006). On local bootstrap bandwidth choice in kernel density estimation. Statistics & Decisions, 24, 291˗301.