The Inference of Complicated Networks by Mutual Information

Abstract

Unsupervised machine learning affords a general idea about complicated data using a graphical representation of networks by nodes and edges to provide a better and easier understanding of the data. The existence of an edge between two entire nodes is determined by their relationship in terms of any kind of dependence i.e., conditional dependence, linear and non-linear, directed or undirected. This study tries to show the accuracy of a non-parametric approach i.e., mutual information (MI) on a real data set named by the Rochdale data that is composed of eight factors that affected women’s economic activity by comparing with some methods such as reversible jump MCMC and birth-death MCMC those tried to detect the conditional dependence between the variables. As a result, MI is not only a very simple but also a very accurate method in the inference of data with complexities.

Keywords

• Conditional dependence
• Gaussian Copula
• Sahnnon Entropy
• Mutual information.

References

1. Abegaz, F., & Wit, E. (2013). Sparse time series chain graphical models for reconstructing genetic networks. Biostatistics, 14(3), 586-599. https://doi.org/10.1093/biostatistics/kxt001.
2. Borowiecki, M. (1947). On the problems of isomorphism and construction of oriented graphs. In Colloquium Mathematicum (Vol. 1, p. 1). Editions Scientifiques de Pologne. https://doi.org/10.4064/cm-1-1-37-50.
3. Dobra, A., & Lenkoski, A. (2011). Copula Gaussian graphical models and their application to modeling functional disability data. The Annals of Applied Statistics, 5(3), 969-993. https://doi.org/10.1214/10-AOAS439
4. Farnoudkia, H., Purutçuoğlu, V. (2020). Application of r-vine copula method in Istanbul stock market data: A case study for the construction sector. Journal of Turkish Operations Management, 4:509-518. https://dergipark.org.tr/tr/pub/jtom/issue/59336/851947.
5. Harary, F., & Norman, R. Z. (1953). Graph theory as a mathematical model in social science (No. 2). Ann Arbor: the University of Michigan, Institute for Social Research. https://doi.org/10.1017/s1373971900075089
6. Kojadinovic, I., & Yan, J. (2010). Modeling multivariate distributions with continuous margins using the copula R package. Journal of Statistical Software, 34, 1-20. https://doi.org/10.18637/jss.v034.i09.
7. Kong, N. (2007). An entropy‐based measure of dependence between two groups of random variables. ETS Research Report Series, 1, i-18. https://files.eric.ed.gov/fulltext/EJ1111559.pdf.
8. McGill, W. (1954). Multivariate information transmission. Transactions of the IRE Professional Group on Information Theory, 4(4), 93-111. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1057469.

9. Mohammadi, A., & Wit, E. C. (2015). Bayesian structure learning in sparse Gaussian graphical models. https://doi.org/10.1007/s11222-014-9523-7
10. Mohammadi, R., & Wit, E. C. (2017). An Introduction to the BDgraph for Bayesian Graphical Models. https://pure.uva.nl/ws/files/25409351/1712_Crop_.pdf. Schreiber, T. (2000). Measuring information transfer. Physical review letters, 85(2), 461. https://doi.org/10.1103/PhysRevLett.85.461.
11. Shannon, C. E., & Weaver, W. (1949). A mathematical model of communication. Urbana, IL: University of Illinois Press, 11, 11-20. https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf.
12. Whittaker, J. (2009). Graphical models in applied multivariate statistics. Wiley Publishing. https://doi.org/10.1002/9780470744639.