Generating function for generalized Fisher information measure and its application to finite mixture models

Abstract

In this work, we consider generating function for generalized Fisher information measure and use it to develop some results for this measure. Next, we study generalized Fisher information for the mixing parameter vector of a finite mixture density function and develop some results for this model. Further, we propose a Jensen-type divergence measure, namely, Jensen-generalized Fisher information (JGFI), and establish some properties for this measure and its generating function. Finally, for illustrative purposes, we examine a real example from image processing and provide some numerical results in terms of JGFI measure. 

Keywords

• Fisher information
• generalized Fisher information
• generating function
• Jensen-Fisher information
• finite mixture distributions

References

1. [1] N. Balakrishnan and A. Stepanov, On the Fisher information in record data, Statist. Probab. Lett. 76 (5), 537-545, 2006.
2. [2] P.K. Bhattacharya, Estimation of a probability density function and its derivatives, Sankhya A 29 (4), 373-382, 1967.
3. [3] J.F. Bercher, Some properties of generalized Fisher information in the context of nonextensive thermostatistics, Phys. A 392 (15), 3140-3154, 2013.
4. [4] S.G. Bobkov, Moments of the scores, IEEE Trans. Inf. Theory 65 (9), 5294-5301, 2019.
5. [5] T. Duong, M. Wand, J. Chacon and A. Gramacki, Package “ks”, R package version: 1.13.5, 2022.
6. [6] R.A. Fisher, Tests of significance in harmonic analysis, Proc. R. Soc. A 125 (796), 54-59, 1929.
7. [7] S. Golomb, The information generating function of a probability distribution (corresp.), IEEE Trans. Inf. Theory 12 (1), 75-77, 1966.
8. [8] S. Guiasu and C. Reischer, The relative information generating function, Inform. Sci. 35 (3), 235-241, 1985.

9. [9] O. Kharazmi and N. Balakrishnan, Jensen-information generating function and its connections to some well-known information measures, Statist. Probab. Lett. 170, 1-10, 2021.
10. [10] F. Nielsen and R. Nock, On the chi square and higher-order chi distances for approximating f-divergences, IEEE Signal Process. Lett. 21 (1), 10-13, 2014.
11. [11] T. Papaioannou, K. Ferentinos and C. Tsairidis, Some information theoretic ideas useful in statistical inference, Methodol. Comput. Appl. Probab. 9 (2), 307-323, 2007.
12. [12] G. Pau, F. Fuchs, O. Sklyar, M. Boutros and W. Huber, EBImage-an R package for image processing with applications to cellular phenotypes, Bioinformatics 26 (7), 979-981, 2010.
13. [13] P. Sánchez-Moreno, A. Zarzo and J.S. Dehesa, Jensen divergence based on Fisher’s information, J. Phys. A Math. 45 (12), 125305, 2012.
14. [14] C.E. Shannon, A mathematical theory of communication, Bell Labs Tech. J. 27 (3), 379-423, 1948.
15. [15] P. Zegers, Fisher information properties, Entropy 17 (7), 4918-4939, 2015.