Understanding the mathematical background of Generative Adversarial Networks (GANs)
Abstract
Generative Adversarial Networks (GANs) have gained widespread attention since their introduction, leading to numerous extensions and applications of the original GAN idea. A thorough understanding of GANs' mathematical foundations is necessary to use and build upon these techniques. However, most studies on GANs are presented from a computer science or engineering perspective, which can be challenging for beginners to understand fully. Therefore, this paper aims to provide an overview of the mathematical background of GANs, including detailed proofs of optimal solutions for vanilla GANs and boundaries for $f$-GANs that minimize a variational approximation of the $f$-divergence between two distributions. These contributions will enhance the understanding of GANs for those with a mathematical background and pave the way for future research.
Keywords
Generative adversarial networks, unsupervised learning, qualitative analysis
Supporting Institution
German Ministry of Education Research (BMBF)
Project Number
05M20UKA
References
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S. et al. Generative adversarial nets. Advances in Neural Information Processing Systems, 27, (2014).
- de Meer Pardo, F. Enriching financial datasets with generative adversarial networks. MS thesis, Delft University of Technology, The Netherlands, (2019).
- Wang, Y. A mathematical introduction to generative adversarial nets (GAN). ArXiv Prints, ArXiv:2009.00169, (2020).
- Arjovsky, M. and Léon, B. Towards principled methods for training generative adversarial Networks. ArXiv Prints, arXiv:1701.04862, (2017).
- Syed A.M. and Samuel, D.S. A general class of coefficients of divergence of one distribution from another. Journal of the Royal Statistical Society: Series B (Methodological), 28(1), 131-142, (1996).
- Nguyen, X., Wainwright, M.J. and Jordan M.I. Estimating divergence functionals and the likelihood ratio by convex risk minimization. In Proceedings, IEEE Transactions on Information Theory, 56(11), pp. 5847-5861, (2010, October).
- Nowozin, S., Cseke, B. and Tomioka, R. f-GAN: Training generative neural samplers using variational divergence minimization. In Proceedings, Advances in Neural Information Processing Systems 29 (NIPS), (2016, December).
- Arjovsky, M., Chintala, S. and Bottou, L. Wasserstein generative adversarial networks. In Proceedings Proceedings of the 34th International Conference on Machine Learning (PMLR), pp. 214-223, (2017, July).
- Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. and Courville, A.C. Improved training of Wasserstein GANs. In Proceedings Advances in neural information processing systems 30 (NIPS), (2017, December).
- Ni, H., Szpruch, L., Wiese, M., Liao, S. and Xiao, B. Conditional sig-wasserstein gans for time series generation. ArXiv Preprint, arXiv:2006.05421, (2020).