A Multi-Stage Metaheuristic Framework for Reliable WGAN-GP Hyperparameter Optimization

Öz

Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP) improve adversarial training stability, but performance remains highly sensitive to hyperparameters governing generator–critic balance, learning rates, optimizer momentum, regularization, and update frequency. This study presents a progressive three-stage metaheuristic framework for WGAN-GP hyperparameter optimization. Stage 1 performs low-cost preliminary exploration on MNIST. Stage 2 refines the FFHQ 64 × 64 search space using a baseline and LSHADE-cnEpSin. Stage 3 compares Particle Swarm Optimization (PSO), Genetic Algorithm (GA), Differential Evolution (DE), and LSHADE-cnEpSin under equal function-evaluation budgets with real Fréchet Inception Distance (FID) as the primary metric. The main result is a search-validation gap: PSO achieves the best short-horizon search-phase FID, whereas GA achieves the best long-horizon final-validation FID. GA is therefore interpreted as showing delayed exploitation in the reported setting, not universal superiority. LSHADE-cnEpSin obtains the lowest search-phase mean FID and stable candidate-level behavior, showing that population-level consistency and final best-candidate quality are distinct evaluation dimensions. These results support staged, fair-budget validation as a more reliable protocol for WGAN-GP hyperparameter optimization.

Anahtar Kelimeler

• Generative adversarial networks
• WGAN-GP
• Hyperparameter optimization
• Metaheuristic optimization
• Fréchet Inception Distance
• LSHADE-cnEpSin

Güvenilir WGAN-GP Hiperparametre Optimizasyonu için Çok Aşamalı Metasezgisel Bir Çerçeve

Öz

Gradyan cezalı Wasserstein çekişmeli üretici ağlar (WGAN-GP), çekişmeli eğitim kararlılığını artırır; ancak performans, üretici–eleştirmen dengesi, öğrenme oranları, optimize edici momentum parametreleri, düzenlileştirme ve güncelleme sıklığını yöneten hiperparametrelere karşı hâlâ oldukça duyarlıdır. Bu çalışma, WGAN-GP hiperparametre optimizasyonu için kademeli üç aşamalı bir metasezgisel çerçeve sunmaktadır. Birinci aşama, MNIST üzerinde düşük maliyetli ön keşif gerçekleştirir. İkinci aşama, temel yapılandırma ve LSHADE-cnEpSin kullanarak FFHQ 64 × 64 arama uzayını iyileştirir. Üçüncü aşama, Parçacık Sürü Optimizasyonu (PSO), Genetik Algoritma (GA), Diferansiyel Evrim (DE) ve LSHADE-cnEpSin yöntemlerini, birincil metrik olarak gerçek Fréchet Inception Distance (FID) kullanarak eşit fonksiyon değerlendirme bütçeleri altında karşılaştırır. Temel sonuç, bir arama–doğrulama farkıdır: PSO kısa ufuklu arama aşamasında en iyi FID değerine ulaşırken, GA uzun ufuklu nihai doğrulamada en iyi FID değerine ulaşmaktadır. Bu nedenle GA, evrensel bir üstünlük olarak değil, raporlanan deneysel ortamda gecikmeli sömürü davranışı göstermesiyle yorumlanmaktadır. LSHADE-cnEpSin, en düşük arama aşaması ortalama FID değerini ve kararlı aday düzeyi davranışı elde ederek, popülasyon düzeyi tutarlılık ile nihai en iyi aday kalitesinin birbirinden ayrı değerlendirme boyutları olduğunu göstermektedir. Bu sonuçlar, WGAN-GP hiperparametre optimizasyonu için kademeli ve adil bütçeli doğrulamayı daha güvenilir bir protokol olarak desteklemektedir.

Anahtar Kelimeler

• Çekişmeli üretici ağlar
• WGAN-GP
• Hiperparametre optimizasyonu
• Metasezgisel optimizasyon
• Fréchet Inception Distance
• LSHADE-cnEpSin

Kaynakça

1. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio,“Generative adversarial nets,” in Advances in Neural Information Processing Systems, 2014.
2. A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional gen- erative adversarial networks,” in Proceedings of the International Conference on Learning Representations, 2016.
3. M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” in Proceedings of the International Conference on Machine Learning, 2017.
4. I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, “Improved training of Wasserstein GANs,” in Advances in Neural Information Processing Systems, 2017.
5. T. Karras, S. Laine, and T. Aila,“Astyle-based generator architecture for generative adversarial networks,”in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4401–4410.
6. T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training GANs,” in Advances in Neural Information Processing Systems, 2016.
7. M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “GANs trained by a two time-scale update rule converge to a local Nash equilibrium,” in Advances in Neural Information Processing Systems, 2017.
8. A. Borji, “Pros and cons of GAN evaluation measures,” Computer Vision and Image Understanding, vol. 179, pp. 41–65, 2019.

9. M. Lucic, K. Kurach, M. Michalski, S. Gelly, and O. Bousquet, “Are GANs created equal? A large-scale study,” in Advances in Neural Information Processing Systems, 2018.
10. G. Parmar, R. Zhang, and J.-Y. Zhu,“Onaliased resizing and surprising subtleties in GAN evaluation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 11410–11420.
11. M. S. M. Sajjadi, O. Bachem, M. Lucic, O. Bousquet, and S. Gelly, “Assessing generative models via precision and recall,” in Advances in Neural Information Processing Systems, 2018.
12. T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila, “Improved precision and recall metric for assessing generative models,” in Advances in Neural Information Processing Systems, 2019.
13. J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization,” Journal of Machine Learning Research, vol. 13, pp. 281–305, 2012.
14. J. Snoek, H. Larochelle, and R. P. Adams, “Practical Bayesian optimization of machine learning algorithms,”in Advances in Neural Information Processing Systems, 2012.
15. L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar, “Hyperband: A novel bandit-based approach to hyperparameter optimization,” Journal of Machine Learning Research, vol. 18, no. 185, pp.1–52, 2018.
16. L. Yang and A. Shami,“Onhyperparameter optimization of machine learning algorithms: Theory and prac- tice,” Neurocomputing, vol. 415, pp. 295–316, 2020.
17. J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of the IEEE International Con- ference on Neural Networks, 1995.
18. M. Clerc and J. Kennedy, “The particle swarm: Explosion, stability, and convergence in a multidimensional complex space,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 1, pp. 58–73, 2002.
19. R. Storn and K. Price, “Differential evolution: A simple and efficient heuristic for global optimization over continuous spaces,” Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997.
20. R. Tanabe and A. Fukunaga, “Improving the search performance of SHADE using linear population size reduction,” in Proceedings of the IEEE Congress on Evolutionary Computation, 2014.
21. N. H. Awad, M. Z. Ali, and P. N. Suganthan, “Ensemble sinusoidal differential covariance matrix adapta- tion with Euclidean neighborhood for solving CEC2017 benchmark problems,” in 2017 IEEE Congress on Evolutionary Computation (CEC), Donostia, Spain, 2017, pp. 372–379, doi: 10.1109/CEC.2017.7969336.
22. A. A. Mohamed, Y. N. M. Saleh, and A. A. Abdel-Hamid, “Fisher-PSO latent optimizer: Metaheuristic- based optimization for privacy-utility tradeoff in WGAN,” in 2024 International Conference on Computer and Applications (ICCA), 2024, pp. 1–6, doi: 10.1109/ICCA62237.2024.10927965.
23. A. Alqushaibi, M. H. Hasan, S. J. Abdulkadir, K. U. Danyaro, M. G. Ragab, S. M. Al-Selwi, E. H. Sumiea, and H. Alhussian, “Enhanced colon cancer segmentation and image synthesis through advanced generative adversarial networks based-sine cosine algorithm,” IEEE Access, vol. 12, pp. 105354–105369, 2024, doi: 10.1109/ACCESS.2024.3402262.
24. B. He and E. Kita, “GA-based optimization of generative adversarial networks on stock price prediction,” in 2021 International Conference on Computational Science and Computational Intelligence (CSCI), 2021, pp. 199–202, doi: 10.1109/CSCI54926.2021.00106.