A Parallel PSO Approach for Hyperparameter Optimization in Machine Learning Models

Abstract

Hyperparameter tuning is crucial for improving the performance of machine learning models, especially in high-dimensional and complex parameter spaces where traditional methods (Grid Search(GS) and Random Search (RS) fall short. This work introduces a parallelized Particle Swarm Optimization(P-PSO) approach for hyperparameter optimization, which is evaluated on three benchmark datasets (Iris, Breast Cancer, Red Wine Quality) across three models (Logistic Regression (LR), Random Forest (RF), and Support Vector Classifier (SVC)). Experimental results show that P-PSO achieves superior weighted F1-scores in most cases; for example, it reaches 0.96 on the Iris dataset across all models, 0.88 for RF on Breast Cancer, and 0.69 for RF on the particularly challenging Red Wine Quality dataset, outperforming other optimization techniques by margins of up to 0.02-0.05. Despite longer execution times, especially on complex models (up to 43 seconds for RF on Red Wine Quality), P-PSO offers more consistency and higher accuracy. These results confirm that P-PSO is an effective, scalable, and robust alternative for hyperparameter tuning, especially in cases where maximizing model performance rather than computational cost is prioritized.

Keywords

• Hyperparameter optimization
• Parallel particle swarm optimization
• Optimization

Makine Öğrenimi Modellerinde Hiperparametre Optimizasyonu için Paralel PSO Yaklaşımı

Abstract

Hiperparametre ayarı, özellikle geleneksel yöntemlerin (Grid Search(GS) ve Random Search(RS)) yetersiz kaldığı yüksek boyutlu ve karmaşık parametre alanlarında, makine öğrenimi modellerinin performansını artırmak için çok önemlidir. Bu çalışma, üç modelde (Lojistik Regresyon (LR), Random Forest (RF) ve Destek Vektör Sınıflandırıcısı (SVM) ) üç veri kümesi (Iris, Meme Kanseri (Breast Cancer), Kırmızı Şarap Kalitesi (Red Wine Quality)) üzerinde değerlendirilen hiperparametre optimizasyonu için paralelleştirilmiş Parçacık Sürüsü Optimizasyonu (P-PSO) yaklaşımını tanıtmaktadır. Deneysel sonuçlar, P-PSO'nun çoğu durumda üstün ağırlıklı F1 puanları elde ettiğini göstermektedir; örneğin, tüm modellerde Iris veri kümesinde 0,96'ya, Breast Cancer'da RF için 0,88'e ve özellikle zorlu Red Wine Quality veri kümesinde RF için 0,69'a ulaşarak diğer optimizasyon tekniklerini 0,02-0,05'e varan marjlarla geride bırakmaktadır. Özellikle karmaşık modellerde daha uzun yürütme sürelerine rağmen, P-PSO daha tutarlı ve daha yüksek doğruluk sunmaktadır. Bu sonuçlar, P-PSO'nun, özellikle hesaplama maliyetinden ziyade model performansının en üst düzeye çıkarılmasının önceliklendirildiği durumlarda, hiperparametre ayarlaması için etkili, ölçeklenebilir ve sağlam bir alternatif olduğunu doğrulamaktadır.

References

1. C. Barut, G. Yildirim, and Y. Tatar, “An intelligent and interpretable rule-based metaheuristic approach to task scheduling in cloud systems,” KNOWLEDGE-BASED Syst., vol. 284, 2024.
2. M. Das, O. Pektezel, C. Barut, G. Yildirim, and B. Alatas, “Case Studies in Thermal Engineering Explainable artificial intelligence for energy efficiency in experimental refrigeration Systems : Advanced cutting-edge sunflower optimization,” Case Stud. Therm. Eng., vol. 73, no. May, p. 106581, 2025.
3. C. Barut, S. Yildirim, B. Alatas, and G. Yildirim, “Innovative multi objective optimization based automatic fake news detection,” pp. 1–39, 2025.
4. Risvanli, A., Tanyeri, B., Yildirim, G., Tatar, Y., Gedikpinar, M., Kalender, H., ... & Kilinc, M. A.. Interpretable Artificial Intelligence for Analysing Changes in Gases in the Uterine Environment of Cows According to Physiological Structures in the Ovary. Veterinary Medicine and Science, 2025;11(2), e70252.
5. H. Ulutas, R. B. Günay, and M. E. Sahin, “Detecting diabetes in an ensemble model using a unique PSO-GWO hybrid approach to hyperparameter optimization,” Neural Comput. Appl., vol. 36, no. 29, pp. 18313–18341, 2024.
6. A. Pranolo, Y. Mao, A. P. Wibawa, A. B. P. Utama, and F. A. Dwiyanto, “Optimized Three Deep Learning Models Based-PSO Hyperparameters for Beijing PM2.5 Prediction,” Knowl. Eng. Data Sci., vol. 5, no. 1, p. 53, 2022.
7. M. Açıkkar and Y. Altunkol, “A novel hybrid PSO- and GS-based hyperparameter optimization algorithm for support vector regression,” Neural Comput. Appl., vol. 35, no. 27, pp. 19961–19977, 2023.
8. K. Aguerchi, Y. Jabrane, M. Habba, and A. H. El Hassani, “A CNN Hyperparameters Optimization Based on Particle Swarm Optimization for Mammography Breast Cancer Classification,” J. Imaging, vol. 10, no. 2, 2024.

9. H. S. Salem, M. A. Mead, and G. S. El-Taweel, “Particle Swarm Optimization-Based Hyperparameters Tuning of Machine Learning Models for Big COVID-19 Data Analysis,” J. Comput. Commun., vol. 12, no. 03, pp. 160–183, 2024.
10. Z. Fouad, M. Alfonse, M. Roushdy, and A. B. M. Salem, “Hyper-parameter optimization of convolutional neural network based on particle swarm optimization algorithm,” Bull. Electr. Eng. Informatics, vol. 10, no. 6, pp. 3377–3384, 2021.
11. Y. Li and Y. Zhang, “Hyper-parameter estimation method with particle swarm optimization,” 2020.
12. L. Tani and C. Veelken, “Comparison of Bayesian and particle swarm algorithms for hyperparameter optimisation in machine learning applications in high energy physics,” Comput. Phys. Commun., vol. 294, no. October 2023, p. 108955, 2024.
13. R. Valarmathi and T. Sheela, “Heart disease prediction using hyper parameter optimization (HPO) tuning,” Biomed. Signal Process. Control, vol. 70, no. July, 2021.
14. J. Yang et al., “IoT-Driven Skin Cancer Detection: Active Learning and Hyperparameter Optimization for Enhanced Accuracy,” IEEE J. Biomed. Heal. Informatics, pp. 1–11, 2025.
15. F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” J. Mach. Learn. Res., vol. 12, 2012.