ADVANCED APPLICATIONS OF PHYSICS-INFORMED NEURAL NETWORKS (PINNS) IN R FOR SOLVING DIFFERENTIAL EQUATIONS

Year 2024, Volume: 25 Issue: 4, 530 - 541, 27.12.2024

Melih Agraz

https://doi.org/10.18038/estubtda.1470050

Abstract

Deep learning, a powerful machine learning technique leveraging artificial neural networks, excels in identifying complex patterns and relationships within data. Among its innovations is the emergence of Physics-Informed Neural Networks (PINNs), which have revolutionized the field of applied mathematics by enabling the solution and discovery of differential equations through neural networks. PINNs address two key challenges: data-driven solutions, where the model approximates the hidden solutions of differential equations with fixed parameters, and data-driven discovery, where the network learns parameters that best describe observed data. This study explores the implementation of PINNs within the R programming environment to solve two differential equations: one with boundary conditions y^'-y=0 with y(0)=0 and y(e)=1 boundaries and the Burgers’ Equation. The research utilizes R libraries, including reticulate for Python integration and torch for neural network operations, to demonstrate the versatility and efficacy of PINNs in addressing both data-centric solutions and parameter discovery. The results showcase the ability of PINNs to handle complex, high-dimensional problems, offering a promising alternative to traditional numerical methods for solving differential equations.

Keywords

Physics-Informed Neural Networks (PINNs), Differential Equations, R-programming language, Burgers’ Equation

Thanks

I would like to thanks to Brown University, Department of Applied Math and Prof.Dr. George Karniadakis.

References

• [1] Lagaris I, Likas A, Fotiadis D. Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 1998; 9(5): 987–1000. doi: 10.1109/72.712178.
• [2] Cheng T, Lewis FL, Abu-Khalaf M. A neural network solution for fixed-final time optimal control of nonlinear systems. Automatica, 2007; 43(3): 482–490. doi: https://doi.org/10.1016/j.automatica.2006.09.021. URL https://www.sciencedirect.com/science/article/pii/S0005109806004250.
• [3] Raissi M, Perdikaris P, Karniadaksi GM. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019; pages 686–707.
• [4] Raissi M, Perdikaris P, Karniadaksi GM. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint. arXiv:1711.10561. 2017a.
• [5] Raissi M, Perdikaris P, Karniadaksi GM. Physics informed deep learning (part ii): Data-driven discovery of nonlinear partial differential equations. arXiv preprint. preprint arXiv:1711.10566. 2017b.
• [6] Soetaert K, Petzoldt T, Setzer RW. Solving differential equations in R: Package deSolve. Journal of Statistical Software, 2010; 33(9): 1–25. doi: 10.18637/jss.v033.i09.
• [7] Soetaert K, Meysman F. R-package reactran : Reactive transport modelling in r. 2010.
• [8] Soetaert K. rootSolve: Nonlinear root finding, equilibrium and steady-state analysis of ordinary differential equations. R package 1.6. 2009.
• [9] Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. Pytorch: An imperative style, high-performance deep learning library. In: Wallach H, Larochelle H, Beygelzimer A, d'Alché-Buc F, Fox E, Garnett R, editors. Advances in Neural Information Processing Systems 32. Curran Associates, Inc.; 2019. pages 8024–8035.
• [10] Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. URL https://www.tensorflow.org/. Software available from tensorflow.org.
• [11] Markidis S. The old and the new: Can physics-informed deep-learning replace traditional linear solvers? Frontiers in big Data, 2021; 4: 669097.
• [12] Cai S, Wang Z, Wang S, Perdikaris P, Karniadakis G. Physics-informed neural networks (pinns) for heat transfer problems. Journal of Heat Transfer, 2021; 143. doi: 10.1115/1.4050542.
• [13] Baydin A, Pearlmutter B, Radul A, Siskind J. Automatic differentiation in machine learning: A survey. Journal of Machine Learning Research, 2018; 18: 1–43.
• [14] Kingma D, Ba J. Adam: A method for stochastic optimization. International Conference on Learning Representations, 2014.
• [15] Zhang D, Lu L, Guo L, Karniadakis G. Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems. Journal of Computational Physics, 2019; 397. doi: 10.1016/j.jcp.2019.07.048.
• [16] Pang G, Lu L, Karniadakis G. fpinns: Fractional physics-informed neural networks. SIAM Journal on Scientific Computing, 2019; 41: A2603–A2626. doi: 10.1137/18M1229845.
• [17] Jagtap A, Kharazmi E, Karniadakis G. Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 2020; 365: 113028. doi: 10.1016/j.cma.2020.113028.
• [18] Meng X, Li Z, Zhang D, Karniadakis G. Ppinn: Parareal physics-informed neural network for time-dependent pdes. Computer Methods in Applied Mechanics and Engineering, 2020; 370: 113250. doi: 10.1016/j.cma.2020.113250.
• [19] Jagtap A, Karniadakis G. Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics, 2020; 28: 2002–2041. doi: 10.4208/cicp.OA2020-0164.
• [20] Pang G, D’Elia M, Parks M, Karniadakis G. npinns: Nonlocal physics-informed neural networks for a parametrized nonlocal universal laplacian operator. algorithms and applications. Journal of Computational Physics, 2020; 422: 109760. doi: 10.1016/j.jcp.2020.109760.
• [21] Kharazmi E, Zhang Z, Karniadakis GE. hp-vpinns: Variational physics-informed neural networks with domain decomposition. Computer Methods in Applied Mechanics and Engineering, 2021; 374.
• [22] Shukla K, Jagtap AD, Karniadakis GE. Parallel physics-informed neural networks via domain decomposition. Journal of Computational Physics, 2021; 447: 110683.
• [23] Yang L, Meng X, Karniadakis GE. B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data. Journal of Computational Physics, 2021; 425: 109913.
• [24] Lu L, Jin P, Pang G, Zang H, Karniadakis G. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 2021; 3: 218–229. doi: 10.1038/s42256-021-00302-5.
• [25] Raissi M, Karniadakis GE. Hidden physics models: Machine learning of nonlinear partial differential equations. Journal of Computational Physics, 2018; 357: 125–141.
• [26] Basdevant C, Deville M, Haldenwang P, Lacroix J, Ouazzani J, Peyret R, et al. Spectral and finite difference solutions of the burgers equation. Computational Fluid Dynamics. 1986; pages 23–41.
• [27] Ushey K, Allaire J, Tang Y. reticulate: Interface to ’Python’, 2022. URL https://rstudio.github.io/reticulate/, https://github.com/rstudio/reticulate.
• [28] Wickham H, Chang W. Ggplot2: Create Elegant Data Visualisations Using the Grammar of Graphics, 2016. URL URLhttps://CRAN.R-project.org/package=ggplot2.
• [29] Falbel D, Luraschi J. torch: Tensors and Neural Networks with ’GPU’ Acceleration, 2022. URL https://torch.mlverse.org/docs,https://github.com/mlverse/torch.
• [30] Liu DC, Nocedal J. On the limited memory bfgs method for large scale optimization. Mathematical Programming, 1989; 45: 503–528.
• [31] Ruder S. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747. 2016.
• [32] Agraz, M. Evaluating single multiplicative neuron models in physics-informed neural networks for differential equations. Scientific Reports, 2024, 14(1), 19073.