Gauss Log-Derivative Factorization Lemma: Kernel Synthesis, Analytic Hessian, and Bayesian Inference

Abstract

In this study, a factorization lemma defined on positive functions is proposed, and its capabilities in differential algebra, kernel synthesis, and Bayesian inference are demonstrated. Through the log-derivative operator L_w=D-(logw)^', the lemma decomposes the pre-factor (P) of functions multiplied by a positive weight function w from the envelope (w); this mechanism is a direct consequence of the product rule within differential operator algebra. Under the Gauss weight, the lemma enables the investigation of four distinct function families under a common template ψ = P•w through two structural paths: (i) the universal equation (L_w^2+ γ)[P•w]= 0, which yields the polynomial, exponential tilting, and Gabor families depending on the sign of γ, and (ii) high-order iteration of L_w, which yields a family derived from Hermite polynomials. Using the function P(z)=1+αz selected from the polynomial family, ψ_α (z)=(1+αz)e^(-z^2/2) is obtained; this function is employed as a random feature activation, and the corresponding closed-form kernel k_α (x,y)=Δ^(-3/2) (Δ+α^2 r) is derived and proven to be positive semi-definite for all α∈R. By utilizing the differential identity provided by the lemma, the Hessian matrix is obtained without requiring separate computation of second derivatives and is verified at machine epsilon precision. Experimental results indicate that the use of the ψ_α function as both a random feature activation and a corresponding closed-form kernel yields competitive performance with Gauss RBF and Matérn kernels in classification and regression tasks. The factorization structure revealed by the lemma under the Gauss weight points to the potential for establishing a bidirectional relationship between extended Gauss-weighted probability distributions and kernel functions.

Keywords

• Kernel functions
• Activation functions
• Hessian
• Bayesian inference
• Gaussian process
• Random features

Ethical Statement

Ethics committee approval was not required for this study because there was no study on animals or humans. All experiments were conducted using publicly available benchmark datasets from the UCI Machine Learning Repository

Thanks

The source code of this study is available upon request for the purpose of reproducibility.

References

1. Ahlfors, L. V. (1979). Complex analysis (3rd ed.). McGraw-Hill. https://mccuan.math.gatech.edu/courses/6321/lars-ahlfors-complex-analysis-third-edition-mcgraw-hill-science_engineering_math-1979.pdf
2. Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of the American Mathematical Society, 68(3), 337–404. https://doi.org/10.1090/S0002-9947-1950-0051437-7
3. Bochner, S. (1932). Vorlesungen über Fourierische Integrale. Akademische Verlagsgesellschaft.
4. Dua, D., & Graff, C. (2019). UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. https://archive.ics.uci.edu/
5. Gabor, D. (1946). Theory of communication. Part 3: Frequency compression and expansion. Journal of the Institution of Electrical Engineers - Part III: Radio and Communication Engineering, 93(26), 445–457. https://doi.org/10.1049/ji-3-2.1946.0076
6. Ince, E. L. (1956). Ordinary differential equations. Dover Publications.
7. Jacot, A., Gabriel, F., & Hongler, C. (2018). Neural tangent kernel: Convergence and generalization in neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, & R. Garnett (Eds.), Advances in neural information processing systems (Vol. 31, pp. 8571–8580). Curran Associates, Inc. https://papers.nips.cc/paper/2018/hash/5a4be1fa34e62bb8a6ec6b91d2462f5a-Abstract.html
8. Lee, J., Xiao, L., Schoenholz, S. S., Bahri, Y., Novak, R., Sohl-Dickstein, J., & Pennington, J. (2018). Deep neural networks as Gaussian processes. In 7th International Conference on Learning Representations. OpenReview. https://openreview.net/forum?id=B1EA-M-0Z

9. David J. C. MacKay; A Practical Bayesian Framework for Backpropagation Networks. Neural Comput 1992; 4 (3): 448–472. doi: https://doi.org/10.1162/neco.1992.4.3.448
10. Petersen, K. B., & Pedersen, M. S. (2012). The matrix cookbook. Technical University of Denmark.
11. Rahimi, A., & Recht, B. (2007). Random features for large-scale kernel machines. In J. Platt, D. Koller, Y. Singer, & S. Roweis (Eds.), Advances in neural information processing systems (Vol. 20, pp. 1177–1184). Curran Associates, Inc.
12. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. MIT Press. https://doi.org/10.7551/mitpress/3206.001.0001
13. Schölkopf, B., & Smola, A. J. (2002). Learning with kernels: Support vector machines, regularization, optimization, and beyond. MIT Press. https://doi.org/10.7551/mitpress/4175.001.0001
14. Szegő, G. (1939). Orthogonal polynomials. American Mathematical Society.
15. van Hoeij, M. (1997). Factorization of differential operators with rational functions coefficients. Journal of Symbolic Computation, 24(5), 537–561. https://doi.org/10.1006/jsco.1997.0151
16. Vapnik, V. N. (1998). Statistical learning theory. John Wiley & Sons.