Derin Q Ağları Tabanlı Parçacık Sürü Optimizasyonu

Öz

Günümüzde, yapay zekâ ve makine öğrenmesi teknolojilerindeki hızlı gelişmeler, optimizasyon problemlerinin çözümüne farklı ve yenilikçi yaklaşımlar getirmiştir. Bu yöntemler, problem çözümünde klasik optimizasyon tekniklerine alternatif yaklaşımlar sunmaktadırlar. Optimizasyon problemlerinin çözümünde sıklıkla kullanılan metasezgisel algoritmaları makine öğrenmesi teknikleriyle birlikte kullanmak güçlü bir potansiyel sunmaktadır. Bu çalışmada doğa esinli bir metasezgisel algoritma olan parçacık sürü optimizasyonu ile bir makine öğrenmesi yöntemi olan pekiştirmeli öğrenmeyi birlikte kullanan bir model önerilmiştir. Önerilen model 9 tane kıyaslama problemi kullanılarak 50 ve 100 boyut için test edilmiştir. Elde edilen sonuçlar pekiştirmeli öğrenmenin PSO’nun yakınsama ve küresel keşif yeteneklerini geliştirmek için büyük bir potansiyel sunduğunu göstermektedir.

Anahtar Kelimeler

• Pekiştirmeli öğrenme
• DQN
• metasezgisel optimizasyon
• parçacık sürü optimizasyonu

Deep Q Networks Based Particle Swarm Optimization

Öz

Today, rapid developments in artificial intelligence and machine learning technologies have brought different and innovative approaches to the solution of optimization problems. These methods offer alternative approaches to classical optimization techniques in problem solving. Using metaheuristic algorithms, which are frequently used in solving optimization problems, together with machine learning techniques offers a strong potential. In this study, a model that uses particle swarm optimization, which is a nature-inspired metaheuristic algorithm, and reinforcement learning, which is a machine learning method, is proposed. The proposed model is tested for 50 and 100 dimensions using 9 comparison problems. The results show that reinforcement learning offers great potential to enhance the convergence and global exploration capabilities of PSO.

Anahtar Kelimeler

• Reinforcement learning
• DQN
• metaheuristic optimization
• particle swarm optimization.

Kaynakça

1. Calafiore G ve Ghaoui L E. Optimization Models. Cambridge University Press, 2014.
2. Seyyedabbasi A, Aliyev R, Kiani F, Gulle M U, Basyildiz H ve Shah M A. Hybrid Algorithms Based on Combining Reinforcement Learning and Metaheuristic Methods to Solve Global Optimization Problems. Knowledge-Based Systems 2021; 223: 1-20.
3. Kennedy J ve Eberhart R C. Particle Swarm Optimization. International Conference on Neural Networks; 1995; Perth, WA, Australia.
4. Xu G. An Adaptive Parameter Tuning of Particle Swarm Optimization Algorithm. Applied Mathematics and Computation 2013; 219(9): 4560-4569.
5. Zhang W, Ma D, Wei J ve Liang H. A Parameter Selection Strategy for Particle Swarm Optimization Based on Particle Positions. Expert Systems with Applications 2014; 41(7): 3576-3584.
6. Pedersen M ve Chipperfield A. Simplifying Particle Swarm Optimization. Applied Soft Computing 2010; 10(2): 618-628.
7. Garg H. A Hybrid PSO-GA Algorithm for Constrained Optimization Problems. Applied Mathematics and Computation 2016; 274: 292-305.
8. Kamboj V K. A Novel Hybrid PSO–GWO Approach for Unit Commitment Problem. Neural Computing and Applications 2016; 27: 1643-1655.

9. F. A. Şenel, F. Gökçe, A. S. Yüksel ve T. Yiğit, «A Novel Hybrid PSO–GWO Algorithm for Optimization Problems,» Engineering with Computers , cilt 35, pp. 1359-1373, 2018.
10. Premalatha K ve Natarajan A. Hybrid PSO and GA for Global Maximization. International Journal of Open Problems in Computer Science and Mathematics 2009; 2(4): 597-608.
11. Chegini S N, Bagheri A ve Najafi F. PSOSCALF: A New Hybrid PSO Based on Sine Cosine Algorithm and Levy Flight for Solving Optimization Problems. Applied Soft Computing 2018; 73: 697-726.
12. Hayat I, Tariq A, Shahzad W, Masud M, Ahmed S, Ali M U ve Zafar A. Hybridization of Particle Swarm Optimization with Variable Neighborhood Search and Simulated Annealing for Improved Handling of the Permutation Flow-Shop Scheduling Problem. Systems 2023; 11(5): 1-17.
13. Khaloo A ve Amirahmadi A. Performance Assessment of Steel Cantilever Beams Based on Connection Behaviour Using DIC Technique and İmproved Hybrid PSO Algorithm. Structural Health Monitoring 2023.
14. Sutton R S ve Barto A G. Reinforcement Learning: An Introduction. Londra: The MIT Press, 2015.
15. Yin S, Jin M, Lu H, Gong G, Mao W, Chen G ve Li W. Reinforcement-learning-based Parameter Adaptation Method for Particle Swarm Optimization. Complex & Intelligent Systems 2023.
16. Meng X, Li H ve Chen A. Multi-strategy Self-learning Particle Swarm Optimization Algorithm Based on Reinforcement Learning. Mathematical Biosciences and Engineering 2023; 20(5): 8498-8530.
17. Wang F, Wang X ve Sun S. A Reinforcement Learning Level-based Particle Swarm Optimization Algorithm for Large-scale Optimization. Information Sciences 2022; 602: 298-312.
18. Lu L, Zheng H, Jie J, Zhang M ve Dai R. Reinforcement Learning-based Particle Swarm Optimization for Sewage Treatment Control. Complex & Intelligent Systems 2021; 7: 2199-2210.
19. Wu D ve Wang G G. Employing Reinforcement Learning to Enhance Particle Swarm Optimization Methods. Engineering Optimization 2022; 54(2): 329-348.
20. Liu W ve Wang X. Dynamic Decision Model in Evolutionary Games Based on Reinforcement Learning. Systems Engineering - Theory & Practice 2009; 29(3): 28-33.
21. Zai A ve Brown B. Deep Reinforcement Learning in Action. Manning, 2020.
22. Abeyrathna K D ve Jeenanunta C. Escape Local Minima with Improved Particle Swarm Optimization Algorithm. In Norsk IKT-konferanse for Forskning Og Utdanning; 2019.
23. Çomak E. A Particle Swarm Optimizer with Modified Velocity Update and Adaptive Diversity Regulation. Expert Systems 2018; 36(1).
24. Freitas D, Lopes L G ve Morgado-Dias F. Particle Swarm Optimisation: A Historical Review Up to the Current Developments. Entropy 2020; 22(3).
25. He Y, Ma W J ve Zhang J P. The Parameters Selection of PSO Algorithm influencing on Performance of Fault Diagnosis. MATEC Web of Conferences; 2016; Amsterdam, Netherlands.
26. Piotrowski A P, Napiorkowski J J ve Piotrowska A E. Population size in Particle Swarm Optimization. Swarm and Evolutionary Computation 2020; 58.
27. Plevris V ve Solorzano G. A Collection of 30 Multidimensional Functions for Global Optimization Benchmarking. Data 2022; 7(4).