Orthogonal Embedding-Based Artificial Neural Network Solutions to Ordinary Differential Equations

Year 2025, Volume: 25 Issue: 3, 489 - 496, 10.06.2025

Tolga Recep Uçar , Hasan Halit Tali

https://doi.org/10.35414/akufemubid.1558289

Abstract

Providing numerical solutions to differential equations in cases where analytical solutions are not available is of great importance. Recently, obtaining more accurate numerical solutions with artificial neural network-based machine learning methods are seen as promising developments for numerical solutions of differential equations. In this paper, a low-cost, orthogonal embedding-based network with fast training by simple gradient descent algorithm is proposed to obtain numerical solutions of differential equations. This architecture is essentially a two-layer neural network that takes orthogonal polynomials as input. The efficiency and accuracy of the method used in this paper are demonstrated in various problems and comparisons are made with other methods. It is observed that the proposed method stands out especially when compared with high-cost solutions.

Keywords

non-linear ordinary differential equations , numerical approximation , artificial neural networks , orthogonal polynomials

Adi Diferansiyel Denklemlerin Ortogonal Gömme Tabanlı Yapay Sinir Ağı Çözümleri

Abstract

Analitik çözümlerin mevcut olmadığı durumlarda diferansiyel denklemler için nümerik çözümler elde etmek büyük önem taşımaktadır. Son zamanlarda, yapay sinir ağı tabanlı makine öğrenmesi yöntemleriyle daha tutarlı nümerik çözümlerin elde edilmesi diferansiyel denklemlerin nümerik çözümleri için ümit verici gelişmeler olarak görülmektedir. Bu makalede, diferansiyel denklemlerin nümerik çözümlerini elde etmek için basit gradyan düşüm algoritması ile hızlı eğime sahip düşük maliyetli bir ortogonal gömme tabanlı ağ önerilmektedir. Bu mimari, temelde, ortogonal polinomları girdi olarak alan iki katmanlı bir sinir ağıdır. Bu makalede kullanılan yöntemin verimliliği ve tutarlılığı, çeşitli problemlerde gösterilmiş ve diğer yöntemlerle karşılaştırmalar yapılmıştır. Kullanılan yöntemin, özellikle yüksek maliyetli çözümlerle karşılaştırıldığında öne çıktığı görülmüştür.

Keywords

Doğrusal olmayan adi diferansiyel denklemler , nümerik yaklaşım , yapay sinir ağları , ortogonal polinomlar

References

• Chakraverty, S. and Mall, S., 2017. Artificial Neural Networks for Engineers and Scientists. CRC Press. https://doi.org/10.1201/9781315155265
• Cybenko, G., 1989. Approximation by superpositions of a sigmoidal function. Math. Control Signal Systems, 2, 303-314. https://doi.org/10.1007/BF02551274
• Gülsü, M. and Sezer, M., 2006. On the solution of the Riccati equation by the Taylor matrix method. Applied Mathematics and Computation, 176, 414-421. https://doi.org/10.1016/j.amc.2005.09.030
• Günel, K. and Gör, I., 2022. Solving Dirichlet boundary problems for ODEs via swarm intelligence. Mathematical Sciences, 16, 325-341. https://doi.org/10.1007/s40096-021-00424-2
• Hornik, K., Maxwell, S. and Halbert, W., 1989. Multilayer feedforward networks are universal approximators. Neural Network, 2, 359-366. https://doi.org/10.1016/0893-6080(89)90020-8
• Knoke, T. and Wick, T., 2021. Solving differential equations via artificial neural networks: Findings and failures in a model problem. Examples and Counterexamples, 1. https:/doi.org/10.1016/j.exco.2021.100035
• Kumar, M. and Yadav, N., 2011. Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: A survey. Computers & Mathematics with Applications, 10, 3796-3811. https://doi.org/10.1016/j.camwa.2011.09.028
• Lagaris, I.E., Likas, A. and Fotiadis, D.I., 1998. Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. on Neural Netw., 9, 987-1000. https://doi.org/10.1109/72.712178
• Lebedev, N.N., 1965. Special Functions and Their Applications. Prentice-Hall.
• Malek, A. and Beidokhti, R.S., 2006. Numerical solution for high order differential equations using a hybrid neural network—Optimization method. Applied Mathematics and Computation, 1, 260-271. https://doi.org/10.1016/j.amc.2006.05.068
• Mall, S. and Chakraverty, S., 2014. Chebyshev Neural Network based model for solving Lane–Emden type equations. Applied Mathematics and Computation, 247, 100-114. https://doi.org/10.1016/j.amc.2014.08.085
• Mall, S. and Chakraverty, S., 2016. Application of Legendre Neural Network for solving ordinary differential equations. Applied Soft Computing, 43, 347-356. https://doi.org/10.1016/j.asoc.2015.10.069
• Meade, A.J. and Fernandez, A.A., 1994. The numerical solution of linear ordinary differential equations by feedforward neural networks. Mathematical and Computer Modelling, 19, 1-25. https://doi.org/10.1016/0895-7177(94)90095-7
• Mehrkanoon, S., Falck, T. and Suykens, J.A.K., 2012. Approximate Solutions to Ordinary Differential Equations Using Least Squares Support Vector Machines. IEEE Trans. Neural Netw. Learn. Syst., 23, 1356-1367. https://doi.org/10.1109/TNNLS.2012.2202126
• Parand, K., Aghaei, A.A., Kiani, S., Ilkhas Zadeh, T. and Khosravi, Z., 2024. A neural network approach for solving nonlinear differential equations of Lane–Emden type. Engineering with Computers, 40, 953-969. https://doi.org/10.1007/s00366-023-01836-5
• Pinkus, A., 1999. Approximation theory of the MLP model in neural networks. Acta Numerica, 8, 143-195. https://doi.org/10.1017/S0962492900002919
• Schiassi, E., Furfaro, R., Leake, C., De Florio, M., Johnston, H., Mortari, D., 2021. Extreme theory of functional connections: A fast physics-informed neural network method for solving ordinary and partial differential equations. Neurocomputing, 457, 334-356. https://doi.org/10.1016/j.neucom.2021.06.015
• Snyder, M.A., 1966. Chebyshev Methods in Numerical Approximation. Prentice-Hall.
• Strang, G., 2007. Computational Science and Engineering. Wellesley-Cambridge Press. https://doi.org/10.1137/1.9780961408817
• Weidmann, J., 1980. Linear Operators in Hilbert Spaces. Springer-Verlag.
• Wen, Y., Chaolu, T. and Wang, X., 1980. Solving the initial value problem of ordinary differential equations by Lie group based neural network method. PLOS ONE, 17(4). https://doi.org/10.1371/journal.pone.0265992