Probability density function estimation using Multi-layer perceptron

Year 2015, Volume: 5 Issue: 2, 54 - 63, 23.07.2016

Touba Mostefa Mohamed Abdenacer Titaouine Touba Sonia Ouafae Bennis

Abstract

The problem of estimating a probability density function (pdf) can easily be encountered in many areas of experimental physics (high energy, spectroscopy, etc.) and other fields. The standard procedure is to bin the space and approximate the pdf by the ratio between the number of events falling inside each bin over the total and normalized to the bin volume. In this paper we estimate the univariate pdf using an MLP (Multi-Layer Perceptron) where the inputs are based on the exponential model. The proposed method is very effective and estimated densities are too close to some theoretical pdfs. Our method has been integrated in the famous steepest descent algorithm for marginal score functions estimation where two linearly mixed sources were successfully separated

Keywords

Probability density estimation, Neural networks, Multilayer perceptron, BSS, Score function

References

• Silverman, B.W. (1986), “ Density Estimation for Statistics and Data Analysis ”, Chapman & Hall.
• Vogt, J. (2007), “ Basic Analysis Techniques & Multi-Spacecraft Data ”, 6th COSPAR Capacity Building Workshop (pp.4-16)., Sinaia.
• Hwang, J. N., Lay, S. R., & Lippman A. , (1993), “Unsupervised learning for multivariate probability density estimation: Radial basis and projection pursuit,” IEEE Int. Conf Neural Networks (pp. 1486-1491), SanFrancisco, CA.
• Popat, K. & Picard, R. W. (1993), “Novel cluster-based probability model for texture synthesis, classification,
• and compression,” in Proc. SPIE Visual Commun. Image Processing’93, Boston, MA.
• Popat, K. & Picard, R. W. (1994), “Cluster-based probability model applied to image restoration and compression,” in Proc. ICASSP, Adelaide, Australia.
• Rabiner, L. R. (1989), “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, vol. 77, no. 2 (pp. 257-286).
• Moody, J. & Darken, C. J. (1989), “Fast learning in networks of locally tuned processing units,” Neural Computation, vol. 1, no. 3 (pp. 281-294).
• Svozil, D., Kvasnicka, V. & Pospichal, J. (1997), “ Introduction to Multi-Layer Feed-Forward Neural Networks ”, Chemometrics and Intelligent Laboratory Systems Vol.39 ( pp.43-62).
• Modha, D.S. & Fainman, Y. (1994), “A learning law for density estimation,” IEEE Trans. On neural networks, Vol.5, no.3 (pp.519-523).
• White, H. (1992), “Mathematical perspectives on Neural Networks”, M. Moser, D. Rumelhart (Eds).
• Likas, A. (2001), “Probability density estimation using neural networks,” Computer Physics Communications, Vol. 135 (pp. 167-175).
• Ould Mohamed, M.S. (2012), “Contribution à la separation aveugle de sources par utilisation des divergences entre densités de probabilité : application à l’analyse vibratoire,’’ thèse de doctorat de l’université de Reims Champagne – Ardenne.
• Hyvärinen, A., Karhunen, J., & Oja, E. (2001), “Independent Component Analysis.” John Wiely & Sons.
• Jutten, C. & Comon, P. (2007), "Séparation de sources – Tome2 : au-delà de l’aveugle et applications", chapitre 13 par Y. Deville. Collection "Traité IC2, Information - Commande -Communication", Hermès - Lavoisier, Paris.
• Taleb, A. & Jutten, C. (1999), “Source separation in post nonlinear mixtures.” IEEE Transactions on Signal Processing, vol. 47, no. 10 (pp. 2807–2820).