A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems with Shapley Value Interpretability

Mustafa Can Bingöl

doi:10.28979/jarnas.1949124

A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems with Shapley Value Interpretability

Abstract

This study introduces the Proposed Optimization Algorithm (POA), a swarmbased hybrid combining the Gorilla Troops Optimizer (GTO) and the Artificial Bee Colony (ABC) algorithm, to enhance reward maximization in control tasks. Neural networks were trained for both simple (pendulum) and complex (bipedal walker) environments. The POA algorithm was run 10 times, and based on the resulting median values, the best reward scores achieved were −117.771 in the pendulum environment and 30.936 in the bipedal walker environment. These reward values indicate a 0.244 improvement for the pendulum environment and a 39.569 improvement for the bipedal walker compared to the closest competitors (GTO). While there was no statistically significant difference between GTO and POA in the pendulum task, POA performed significantly better than all other algorithms in the bipedal walker environment (p < 0.05). To address the “black box” nature of reinforcement learning, the study integrated Shapley Value Theory for post-training analysis. This explainable AI (XAI) approach identified angular velocity as the primary driver of torque in the pendulum task and quantified the importance of observation parameters for the bipedal walker. The results provide both a high-performing optimization framework and a robust method for interpreting neural network decision-making in robotic control systems.

Keywords

Ethical Statement

No approval from the Board of Ethics is required.

References

M. L. Puterman, Markov decision processes: Discrete stochastic dynamic programming, John Wiley & Sons, 2014.
R. Ozalp, A. Ucar, C. Guzelis, Advancements in deep reinforcement learning and inverse reinforcement learning for robotic manipulation: Toward trustworthy, interpretable, and explainable artificial intelligence, IEEE Access 12 (2024) 51840–51858.
N. Justesen, P. Bontrager, J. Togelius, S. Risi, Deep learning for video game playing, IEEE Transactions on Games 12 (1) (2020) 1–20.
A. A. Abdellatif, N. Mhaisen, A. Mohamed, A. Erbad, M. Guizani, Reinforcement learning for intelligent healthcare systems: A review of challenges, applications, and open research issues, IEEE Internet of Things Journal 10 (24) (2023) 21982–22007.
B. R. Kiran, I. Sobh, V. Talpaert, P. Mannion, A. A. Al Sallab, S. Yogamani, P. PÅLerez, Deep reinforcement learning for autonomous driving: A survey, IEEE Transactions on Intelligent Transportation Systems 23 (6) (2022) 4909–4926.
I. Saleh, N. Borhan, A. Yunus, W. Rahiman, Comprehensive technical review of recent bio-inspired population-based optimization (BPO) algorithms for mobile robot path planning, IEEE Access 12 (2024) 20942–20961.
L. Deng, S. Liu, Advancing photovoltaic system design: An enhanced social learning swarm optimizer with guaranteed stability, Computers in Industry 164 (2025) 104209.
S. Xing, Z. Shao, W. Shao, J. Chen, D. Pi, Joint scheduling of hybrid flow-shop with limited automatic guided vehicles: A hierarchical learning-based swarm optimizer, Computers & Industrial Engineering 198 (2024) 110686.

C. Wu, C. Wang, J.-W. Kim, Welding sequence optimization to reduce welding distortion based on coupled artificial neural network and swarm intelligence algorithm, Engineering Applications of Artificial Intelligence 114 (2022) 105142.
L. Hou, G. Bi, Q. Guo, An improved sparrow search algorithm optimized LightGBM approach for credit risk prediction of SMEs in supply chain finance, Journal of Computational and Applied Mathematics 454 (2025) 116197.
D. Wei, Z. Zhang, W. Zhang, Y. Yang, Z. Yang, Optimization of multi-energy complementary power generation system configuration based on particle swarm optimization, Energy Reports 12 (2024) 2257–2269.
M. Wang, K. Zheng, Y. Yang, X. Wang, An explainable machine learning framework for intrusion detection systems, IEEE Access 8 (2020) 73127–73141.
N. A. Wani, R. Kumar, Mamta, J. Bedi, I. Rida, Explainable AI-driven IoMT fusion: Unravelling techniques, opportunities, and challenges with explainable AI in healthcare, Information Fusion 110 (2024) 102472.
J. Collenette, K. Atkinson, T. Bench-Capon, Explainable AI tools for legal reasoning about cases: A study on the European Court of Human Rights, Artificial Intelligence 317 (2023) 103861.
J. Baek, C. Lee, Y. S. Lee, S. Jeon, S. Han, Reinforcement learning to achieve real-time control of triple inverted pendulum, Engineering Applications of Artificial Intelligence 128 (2024) 107518.
M. Safeea, P. Neto, A Q-learning approach to the continuous control problem of robot inverted pendulum balancing, Intelligent Systems with Applications 21 (2024) 200313.
O. Aydogmus, M. Yilmaz, Comparative analysis of reinforcement learning algorithms for bipedal robot locomotion, IEEE Access 12 (2024) 7490–7499.
C. Kaymak, A. U.car, C. GÅNuzeli.s, Development of a new robust stable walking algorithm for a humanoid robot using deep reinforcement learning with multi-sensor data fusion, Electronics 12 (3) (2023) 568. J. Zhang, K. Chen, Research on carbon asset trading strategy based on PSO-VMD and deep reinforcement learning, Journal of Cleaner Production 435 (2024) 140322.
X. Ma, J. Fan, J. Wang, J. Huang, Y. Ke, GWO-ANFIS-RD3PG: A reinforcement learning approach with dynamic adjustment and dual replay mechanism for building energy forecasting, Journal of Building Engineering 97 (2024) 110726.
T. G. Ritto, S. Beregi, D. A. W. Barton, Reinforcement learning and approximate Bayesian computation (RL-ABC) for model selection and parameter calibration of time-varying systems, Mechanical Systems and Signal Processing 200 (2023) 110458.
P. Zhang, M. Wang, G. Zhang, P. Zheng, M. Jin, J. Zhang, Multi-robot multi-station welding flow shop closed-loop rescheduling with deep reinforcement learning and improved artificial bee colony algorithm, Computers & Industrial Engineering 193 (2024) 110295.
R. Zhang, S. Bu, M. Zhou, G. Li, B. Zhan, Z. Zhang, Deep reinforcement learning based interpretable photovoltaic power prediction framework, Sustainable Energy Technologies and Assessments 67 (2024) 103830.
T. Hickling, N. Aouf, P. Spencer, Robust adversarial attacks detection based on explainable deep reinforcement learning for UAV guidance and planning, IEEE Transactions on Intelligent Vehicles 8 (10) (2023) 4381–4394.
D. Karaboga, B. Akay, A comparative study of artificial bee colony algorithm, Applied Mathematics and Computation 214 (1) (2009) 108–132. D. Karaboga, B. Akay, Artificial bee colony (ABC), harmony search and bees algorithms on numerical optimization, Innovative Production Machines and Systems Virtual Conference, 2009.
B. Abdollahzadeh, F. Soleimanian Gharehchopogh, S. Mirjalili, Artificial gorilla troops optimizer: A new nature-inspired metaheuristic algorithm for global optimization problems, International Journal of Intelligent Systems 36 (10) (2021) 5887–5958.
F. F. Dzekashu, A. A. Yusuf, K. Takemoto, M. K. Peters, H. M. G. Lattorff, I. Steffan-Dewenter, C. W. Pirk, Network resilience of plant-bee interactions in the Eastern Afromontane Biodiversity Hotspot, Ecological Indicators 166 (2024) 112415.
B. Mechri, M. Tekaya, A. Guesmi, N. B. Hamadi, L. Khezami, T. Soltani, F. Attia, H. Chehab, Enhancing olive tree (Olea europaea) rhizosphere dynamics: Co-inoculation effects of arbuscular mycorrhizal fungi and plant growth-promoting rhizobacteria in field experiments, Applied Soil Ecology 202 (2024) 105596.
L. S. Shapley, A value for n-person games, Contributions to the Theory of Games, Vol. 28 of Annals of Mathematics Studies, Princeton University Press, 1953, pp. 307–318.
S. M. Lundberg, S.-I. Lee, A unified approach to interpreting model predictions, Advances in Neural Information Processing Systems 30, Long Beach, 2017, pp. 4765–4774. J. Kennedy, R. Eberhart, Particle swarm optimization, Proceedings of ICNN’95–International Conference on Neural Networks, Vol. 4, Perth, 1995, pp. 1942–1948.
M. Abdel-Basset, R. Mohamed, M. Jameel, M. Abouhawwash, Nutcracker optimizer: A novel nature-inspired metaheuristic algorithm for global optimization and engineering design problems, Knowledge-Based Systems 262 (2023) 110248.
S. Mirjalili, S. M. Mirjalili, A. Lewis, Grey wolf optimizer, Advances in Engineering Software 69 (2014) 46–61.
S. Mirjalili, A. H. Gandomi, S. Z. Mirjalili, S. Saremi, H. Faris, S. M. Mirjalili, Salp swarm algorithm: A bio-inspired optimizer for engineering design problems, Advances in Engineering Software 114 (2017) 163–191.
W. Zhao, L. Wang, S. Mirjalili, Artificial hummingbird algorithm: A new bio-inspired optimizer with its engineering applications, Computer Methods in Applied Mechanics and Engineering 388 (2022) 114194.

Details

Primary Language

English

Subjects

Control Theoryand Applications

Journal Section

Research Article

Authors

Mustafa Can Bingöl ^*
0000-0001-5448-8281
Türkiye

Publication Date

September 30, 2026

Submission Date

May 11, 2026

Acceptance Date

July 23, 2026

Published in Issue

Year 2026 Volume: 12 Number: 3

DOI

https://doi.org/10.28979/jarnas.1949124

IZ

https://izlik.org/JA52PC47UF

Cite

RIS / Bibtex

APA

Bingöl, M. C. (2026). A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems with Shapley Value Interpretability. Journal of Advanced Research in Natural and Applied Sciences, 12(3), 264-281. https://doi.org/10.28979/jarnas.1949124

AMA

1.Bingöl MC. A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems with Shapley Value Interpretability. JARNAS. 2026;12(3):264-281. doi:10.28979/jarnas.1949124

Chicago

Bingöl, Mustafa Can. 2026. “A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems With Shapley Value Interpretability”. Journal of Advanced Research in Natural and Applied Sciences 12 (3): 264-81. https://doi.org/10.28979/jarnas.1949124.

EndNote

Bingöl MC (September 1, 2026) A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems with Shapley Value Interpretability. Journal of Advanced Research in Natural and Applied Sciences 12 3 264–281.

IEEE

[1]M. C. Bingöl, “A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems with Shapley Value Interpretability”, JARNAS, vol. 12, no. 3, pp. 264–281, Sept. 2026, doi: 10.28979/jarnas.1949124.

ISNAD

Bingöl, Mustafa Can. “A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems With Shapley Value Interpretability”. Journal of Advanced Research in Natural and Applied Sciences 12/3 (September 1, 2026): 264-281. https://doi.org/10.28979/jarnas.1949124.

JAMA

1.Bingöl MC. A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems with Shapley Value Interpretability. JARNAS. 2026;12:264–281.

MLA

Bingöl, Mustafa Can. “A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems With Shapley Value Interpretability”. Journal of Advanced Research in Natural and Applied Sciences, vol. 12, no. 3, Sept. 2026, pp. 264-81, doi:10.28979/jarnas.1949124.

Vancouver

1.Mustafa Can Bingöl. A Swarm Optimization-Based Deep Reinforcement Learning Approach for Nonlinear Control Systems with Shapley Value Interpretability. JARNAS. 2026 Sep. 1;12(3):264-81. doi:10.28979/jarnas.1949124

DOAJ	EBSCO
Scilit	SOBİAD