New Information Inequalities in Terms of Variational Distance and its Application

Year 2016, Volume: 5 Issue: 11, 30 - 40, 31.08.2016

K. C. Jain Praphull Chhabra

Abstract

In this work, new information inequalities are obtained and characterized on new generalized f- divergence (introduced by Jain and Saraswat (2012)) in terms of the Variational distance and these inequalities have been taken for evaluating some new relations among well known divergences. These new relations have been verified numerically by considering two discrete probability distributions: Binomial and Poisson. Asymptotic approximation on new generalized f- divergence is done as well.

Keywords

Variational distance, new information inequalities, bounded variation, convex and normalized function, bounds of divergences, numerical veri- fication, asymptotic approximation.

References

• M.B. Bassat, f- Entropies, probability of error and feature selection, Inform. Control, vol. 39, 1978, pp: 227-242.
• H.C. Chen, Statistical pattern recognition, Hoyderc Book Co., Rocelle Park, New York, 1973.
• C.K. Chow and C.N. Lin, Approximating discrete probability distributions with dependence trees, IEEE Trans. Inform. Theory, vol. 14, 1968, no 3, pp: 462-467.
• D. Dacunha- Castelle, Ecole d’Ete de Probabilites de, Saint-Flour VII-1977, Berlin, Heidelberg, New York: Springer, 1978.
• S.S. Dragomir, On the Ostrowski’s integral inequality for mappings of bounded variation and applications, RGMIA Res. Rep. Coll. 2 (1999) No.1, 73?80.
• S.S. Dragomir, V. Gluscevic, and C.E.M. Pearce, Approximation for the Csiszar f-divergence via midpoint inequalities, in inequality theory and applications - Y.J.Cho, J.K. Kim, and S.S. Dragomir (Eds.), Nova Science Publishers, Inc., Huntington, New York, Vol. 1, 2001, pp. 139-154.
• D.V. Gokhale and S. Kullback, Information in contingency Tables, New York, Marcel Dekker, 1978.
• K.C. Jain and P. Chhabra, Series of new information divergences, properties and corresponding series of metric spaces, International Journal of Innovative Research in Science, Engineering and Technology, vol. 3- no.5 (2014), pp: 12124- 12132.
• K.C. Jain and R.N. Saraswat, Some new information inequalities and its applications in information theory, International Journal of Mathematics Research, vol. 4, Number 3 (2012), pp. 295-307.
• L. Jones and C. Byrne, General entropy criteria for inverse problems with applications to data compression, pattern classification and cluster analysis, IEEE Trans. Inform. Theory, vol. 36, 1990, pp: 23-30.
• T. Kailath, The divergence and Bhattacharyya distance measures in signal selection, IEEE Trans. Comm. Technology, vol. COM-15, 1967, pp: 52-60.
• T.T. Kadota and L.A. Shepp, On the best finite set of linear observables for discriminating two Gaussian signals, IEEE Trans. Inform. Theory, vol. 13, 1967, pp: 288-294.
• D. Kazakos and T. Cotsidas, A decision theory approach to the approximation of discrete probability densities, IEEE Trans. Perform. Anal. Machine Intell., vol. 1, 1980, pp: 61-67.
• A.N. Kolmogorov, On the approximation of distributions of sums of independent summands by infinitely divisible distributions, Sankhya, 25, 159-174, 1963.
• F. Nielsen and S. Boltz, The Burbea-Rao and Bhattacharyya centroids, Apr. 2010, arxiv.
• K. Pearson, On the Criterion that a given system of deviations from the probable in the case of correlated system of variables is such that it can be reasonable supposed to have arisen from random sampling, Phil. Mag., 50(1900), 157-172.
• E.C. Pielou, Ecological diversity, New York, Wiley, 1975.
• R. Santos-Rodriguez, D. Garcia-Garcia, and J. Cid-Sueiro, Cost-sensitive classification based on Bregman divergences for medical diagnosis, In M. A. Wani, editor, Proceedings of the 8th International Conference on Machine Learning and Applications (ICMLA’09), Miami Beach, Fl., USA, December 13-15, 2009, pp: 551- 556, 2009.
• R. Sibson, Information radius, Z. Wahrs. Undverw. Geb., (14) (1969),149-160.
• I.J. Taneja, New developments in generalized information measures, Chapter in: Advances in Imaging and Electron Physics, Ed. P.W. Hawkes, 91(1995), 37-135.
• B. Taskar, S. Lacoste-Julien, and M.I. Jordan, Structured prediction, dual extra gradient and Bregman projections, Journal of Machine Learning Research, 7, pp:1627- 1653, July 2006.
• H. Theil, Economics and information theory, Amsterdam, North-Holland, 1967.
• H. Theil, Statistical decomposition analysis, Amsterdam, North-Holland, 1972.
• B. Vemuri, M. Liu, S. Amari, and F. Nielsen, Total Bregman divergence and its applications to DTI analysis, IEEE Transactions on Medical Imaging, 2010.