A novel hybrid algorithm based on Stochastic Fractal Search Algorithm and CMA-ES

Abstract

In this study, a novel hybridization approach, which is called CMASFS and is based on the covariance matrix adaptation evolution strategy (CMA-ES) and the stochastic fractal search (SFS) algorithms. To make the proposed algorithm dynamic, Gaussian walk equations involved in the diffusion process of SFS have been updated and the algorithm decide to use which the Gaussian walk equations. The effectiveness of the proposed algorithm is tested using CEC2017 benchmark functions having unimodal, multimodal, hybrid, and composition functions in 10, 30, 50, and 100 dimensions. The performance of the CMASFS algorithm is compared with 17 metaheuristic algorithms given in the literature over the CEC2017 benchmark functions. According to the results, it is seen that CMASFS is generally obtained better mean error values. Moreover, to show the superiority of the proposed algorithm, Friedman analysis and the Wilcoxon rank-sum test are applied to the test results of the algorithms. The results of the Wilcoxon signed-rank test show that the improvement with the CMASFS algorithm is statistically significant on the majority of the CEC2017. The results of Friedman test verify that the CMASFS is obtained the best rank compared to both the original SFS and other compared algorithms.

Keywords

• Optimization algorithm
• Meta-heuristic
• Covariance matrix adaptation evolution strategy
• Stochastic fractal search
• CEC 2017 benchmark problems

Stokastik Fraktal Arama Algoritması ve CMA-ES tabanlı yeni bir hibrit algoritma

Abstract

Bu çalışmada, kovaryans matris uyarlaması ile evrim stratejisi (CMA-ES) ve stokastik fraktal arama (SFA) algoritmalarına dayanan CMASFA adı verilen yeni bir hibritleştirme yaklaşımı geliştirilmiştir. Önerilen algoritmayı dinamik hale getirmek için, SFS'nin yayılım sürecinde yer alan Gauss yürüyüş eşitlikleri güncellenmiş ve hangi Gauss yürüyüş eşitliğinin kullanılacağına algoritmanın karar vermesi sağlanmıştır. Önerilen algoritmanın etkinliği, 10, 30, 50 ve 100 boyutlu tekmodlu, çokmodlu, melez ve komposizyon fonksiyonlarına sahip CEC2017 benchmark fonksiyonları kullanılarak test edilmiştir. CEC2017 benchmark fonksiyonları kullanılarak CMASFS algoritmasının performansı, literatürde verilen 17 metasezgisel algoritma ile karşılaştırılmıştır. Elde edilen sonuçlara göre, CMASFS'nin daha düşük bir ortalama hata değerleri elde ettiği görülmüştür. Ayrıca önerilen algoritmanın üstünlüğünü göstermek için algoritmaların elde ettiği sonuçlar üzerinde Friedman analizi ve Wilcoxon işaretli sıra testi uygulanmıştır. Wilcoxon işaretli sıra testinin sonuçlarına göre, CMASFA algoritmasıyla yapılan iyileştirmenin CEC2017 içerisindeki fonksiyonların büyük bir çoğunluğunda istatistiksel olarak anlamlı farklılık oluşturduğu ve daha uygun sonuçlar elde ettiği sonucuna ulaşılmıştır. Friedman testinin sonuçlarına göre de CMASFA'nin hem orijinal SFA’ya hem de diğer karşılaştırılan algoritmalara kıyasla en iyi sıralamayı elde ettiği görülmektedir.

Keywords

• Optimizasyon algoritması
• Meta-sezgisel
• Kovaryans matrisi adaptasyon evrim stratejisi
• Stokastik fraktal arama
• CEC 2017 benchmark problemleri

References

1. Karaboğa, D., Yapay zekâ optimizasyon algoritmaları. Nobel Yayın Dağıtım, Ankara, 2011.
2. Reeves, C.R., Modern heuristic techniques for combinatorial problems. Advanced topics in computer science, 1995.
3. Holland, J.H., Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, MI, 1975.
4. Eberhart, R., Kennedy, J., A new optimizer using particle swarm theory. In MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science, 1995, 39-43, DOI: 10.1109/MHS.1995.494215.
5. Storn, R., Price, K., Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization, 11(4), 1997, pp. 341-359, DOI: 10.1023/A:1008202821328.
6. Dorigo, M., Di Caro, G., Ant colony optimization: a new meta-heuristic. In Proceedings of the 1999 congress on evolutionary computation-CEC99, 2, 1999, pp. 1470-1477, DOI: 10.1109/CEC.1999.782657.
7. Karaboga, D., An idea based on honey bee swarm for numerical optimization, Technical report-tr06, Erciyes University, Engineering faculty, Computer engineering department, 2005.
8. Yang, X.S., Firefly algorithm. Nature-inspired metaheuristic algorithms, 20, 2008, pp. 79-90.

9. Rashedi, E., Nezamabadi-Pour, H., Saryazdi, S., GSA: a gravitational search algorithm. Information sciences, 179(13), 2009, pp. 2232-2248. DOI: 10.1016/j.ins.2009.03.004.
10. Yang, X.S., Deb, S., Cuckoo search via Lévy flights. In 2009 World Congress on Nature and Biologically Inspired Computing (NaBIC), 2009, pp. 210-214, DOI: 10.1109/NABIC.2009.5393690.
11. Mirjalili, S., Mirjalili, S.M., Lewis, A., Grey wolf optimizer. Advances in engineering software, 69, 2014, pp. 46-61, DOI: 10.1016/j.advengsoft.2013.12.007.
12. Civicioglu, P., Transforming geocentric cartesian coordinates to geodetic coordinates by using differential search algorithm. Computers and Geosciences, 46, 2012, pp. 229-247, DOI: 10.1016/j.cageo.2011.12.011.
13. Civicioglu, P., Backtracking search optimization algorithm for numerical optimization problems. Applied Mathematics and Computation, 219(15), 2013, pp. 8121-8144, DOI: 10.1016/j.amc.2013.02.017.
14. Cheng, M.Y., Prayogo, D., Symbiotic organisms search: a new metaheuristic optimization algorithm. Computers and Structures, 139, 2014, pp. 98-112, DOI: 10.1016/j.compstruc.2014.03.007.
15. Mirjalili, S., Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm. Knowledge-Based Systems, 89, 2015, pp. 228-249, DOI: 10.1016/j.knosys.2015.07.006.
16. Shareef, H., Ibrahim, A.A., Mutlag, A.H., Lightning search algorithm. Applied Soft Computing, 36, 2015, pp. 315-333, DOI: 10.1016/j.asoc.2015.07.028.
17. Askarzadeh, A., A novel metaheuristic method for solving constrained engineering optimization problems: Crow search algorithm. Computers and Structures, 169, 2016, pp. 1-12 DOI: 10.1016/j.compstruc.2016.03.001.
18. Mirjalili, S., SCA: a sine cosine algorithm for solving optimization problems. Knowledge-Based Systems, 96, 2016, pp. 120-133, DOI: 10.1016/j.knosys.2015.12.022.
19. Mirjalili, S., Lewis, A., 2016. The whale optimization algorithm. Advances in engineering software, 95, 51-67. DOI: 10.1016/j.advengsoft.2016.01.008
20. Mirjalili, S., Gandomi, A.H., Mirjalili, S.Z., Saremi, S., Faris, H., Mirjalili, S.M., 2017. Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems. Advances in Engineering Software, 114, 163-191. DOI: 10.1016/j.advengsoft.2017.07.002
21. Pierezan, J., Coelho, L.D.S., Coyote optimization algorithm: a new metaheuristic for global optimization problems. In 2018 IEEE Congress on Evolutionary Computation (CEC), 2018, pp. 1-8, DOI: 10.1109/CEC.2018.8477769.
22. Arora, S., Singh, S., Butterfly optimization algorithm: a novel approach for global optimization. Soft Computing, 23(3), 2019, pp. 715-734, DOI: 10.1007/s00500-018-3102-4.
23. Zhao, W., Wang, L., Zhang, Z., Atom search optimization and its application to solve a hydrogeologic parameter estimation problem. Knowledge-Based Systems, 163, 2019, pp. 283-304, DOI: 10.1016/j.knosys.2018.08.030.
24. Salimi, H., Stochastic fractal search: a powerful metaheuristic algorithm. Knowledge-Based Systems, 75, 2015, pp. 1-18, DOI: 10.1016/j.knosys.2014.07.025.
25. Rahman, T.A. Parameters optimization of an SVM-classifier using stochastic fractal search algorithm for monitoring an aerospace structure. International Journal of Fluids and Heat Transfer, 1(1), 2016, pp. 69-79.
26. Mosbah, H., El-Hawary, M., Power system tracking state estimation based on stochastic fractal search technique under sudden load changing conditions. In 2016 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), 2016, pp. 1-6.
27. Chuan, S.U.N., WEI, Z.Q., ZHOU, C.J., Bin, W.A.N.G., Stochastic fractal search algorithm for 3d protein structure prediction. DEStech Transactions on Computer Science and Engineering. 2016, DOI: 10.12783/dtcse/aics2016/8189.
28. Luo, Q., Zhang, S., Zhou, Y., Stochastic Fractal Search Algorithm for Template Matching with Lateral Inhibition. Scientific Programming. 2017, DOI: 10.1155/2017/1803934.
29. Çelik, E., Incorporation of stochastic fractal search algorithm into efficient design of PID controller for an automatic voltage regulator system. Neural Computing and Applications, 30(6), 2018, pp. 1991-2002, DOI: 10.1007/s00521-017-3335-7.
30. Hinojosa, S., Dhal, K.G., Elaziz, M.A., Oliva, D., Cuevas, E., Entropy-based imagery segmentation for breast histology using the Stochastic Fractal Search. Neurocomputing, 321, 2018, pp. 201-215, DOI: 10.1016/j.neucom.2018.09.034.
31. Saha, D., Saikia, L.C., Automatic generation control of an interconnected CCGT‐thermal system using stochastic fractal search optimized classical controllers. International Transactions on Electrical Energy Systems, 28(5), 2018, pp. 2533. DOI: 10.1002/etep.2533.
32. Bingöl, O., Paçacı, S., Pişirir, O.M., Özkaya, B., Stochastic Fractal Search Algorithm for ANFIS Training, International Conference on Science and Technology (ICONST 2018), 2018, pp. 422-428.
33. Çelik, E., Gör, H., Enhanced speed control of a DC servo system using PI+ DF controller tuned by stochastic fractal search technique. Journal of the Franklin Institute, 356(3), 2019, pp. 1333-1359, DOI: 10.1016/j.jfranklin.2018.11.020.
34. Bhatt, R., Parmar, G., Gupta, R., Sikander, A., Application of stochastic fractal search in approximation and control of LTI systems. Microsystem Technologies, 25(1), 2019, pp. 105-114, DOI: 10.1007/s00542-018-3939-6.
35. Betka, A., Terki, N., Toumi, A., Hamiane, M., Ourchani, A., A new block matching algorithm based on stochastic fractal search. Applied Intelligence, 49(3), 2019, pp. 1146-1160, DOI: 10.1007/s10489-018-1312-1.
36. Mellal, M.A., Zio, E., A penalty guided stochastic fractal search approach for system reliability optimization. Reliability Engineering and System Safety, 152, 2016, pp. 213-227, DOI: 10.1016/j.ress.2016.03.019.
37. Awad, N.H., Ali, M.Z., Suganthan, P.N., Jaser, E., Differential evolution with stochastic fractal search algorithm for global numerical optimization. In 2016 IEEE Congress on Evolutionary Computation (CEC), 2016, pp. 3154-3161, DOI: 10.1109/CEC.2016.7744188.
38. Awad, N.H., Ali, M.Z., Suganthan, P.N., Jaser, E., A decremental stochastic fractal differential evolution for global numerical optimization. Information Sciences, 372, 2016, pp. 470-491, DOI: 10.1016/j.ins.2016.08.032.
39. Rahman, T.A., Tokhi, M.O., Enhanced stochastic fractal search algorithm with chaos. In 2016 7th IEEE Control and System Graduate Research Colloquium (ICSGRC), 2016, pp. 22-27, DOI: 10.1109/ICSGRC.2016.7813295.
40. Zhou, C., Sun, C., Wang, B., Wang, X., An improved stochastic fractal search algorithm for 3D protein structure prediction. Journal of molecular modeling, 24(6), 2018, pp. 125, DOI: 10.1007/s00894-018-3644-5.
41. Lin, J., Wang, Z.J., Multi-area economic dispatch using an improved stochastic fractal search algorithm. Energy, 166, 2019, pp. 47-58, DOI: 10.1016/j.energy.2018.10.065.
42. Bingöl, O., Güvenç, U., Duman, S., Paçacı, S., Stochastic fractal search with chaos. In 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), 2017, pp. 1-6. DOI: 10.1109/IDAP.2017.8090231.
43. Rahman, T.A., Jalil, N.A., As’Arry, A., Ahmad, R.R., Chaos-enhanced Stochastic Fractal Search algorithm for Global Optimization with Application to Fault Diagnosis. In IOP Conference Series: Materials Science and Engineering, 210(1), 2017.
44. Wang, L., Pan, Q.K., Suganthan, P.N., Wang, W.H., Wang, Y.M., A novel hybrid discrete differential evolution algorithm for blocking flow shop scheduling problems. Computers and Operations Research, 37(3), 2010, pp. 509-520, DOI: 10.1016/j.cor.2008.12.004.
45. Li, J.Q., Pan, Q.K., Suganthan, P.N., Chua, T.J., A hybrid tabu search algorithm with an efficient neighborhood structure for the flexible job shop scheduling problem. The international journal of advanced manufacturing technology, 52(5-8), 2011, pp. 683-697, DOI: 10.1007/s00170-010-2743-y.
46. Ali, M.Z., Awad, N.H., Suganthan, P.N., Duwairi, R.M., Reynolds, R.G., A novel hybrid Cultural Algorithms framework with trajectory-based search for global numerical optimization. Information Sciences, 334, 2016, pp. 219-249, DOI: 10.1016/j.ins.2015.11.032.
47. Jayabarathi, T., Raghunathan, T., Adarsh, B.R., Suganthan, P.N., Economic dispatch using hybrid grey wolf optimizer. Energy, 111, 2016, pp. 630-641, DOI: 10.1016/j.energy.2016.05.105.
48. Awad, N.H., Ali, M.Z., Suganthan, P.N., Reynolds, R.G., CADE: a hybridization of cultural algorithm and differential evolution for numerical optimization. Information Sciences, 378, 2017, pp. 215-241, DOI: 10.1016/j.ins.2016.10.039.
49. Sundar, S., Suganthan, P.N., Jin, C.T., Xiang, C.T., Soon, C.C., A hybrid artificial bee colony algorithm for the job-shop scheduling problem with no-wait constraint. Soft Computing, 21(5), 2017, pp. 1193-1202, DOI: 10.1007/s00500-015-1852-9
50. Barraza, J., Rodríguez, L., Castillo, O., Melin, P., Valdez, F., A new hybridization approach between the fireworks algorithm and grey wolf optimizer algorithm. Journal of Optimization, 2018, DOI: 10.1155/2018/6495362.
51. Zhang, X., Kang, Q., Cheng, J., Wang, X., A novel hybrid algorithm based on biogeography-based optimization and grey wolf optimizer. Applied Soft Computing, 67, 2018, pp. 197-214, DOI: 10.1016/j.asoc.2018.02.049.
52. Majumder, A., Laha, D., Suganthan, P.N., A hybrid cuckoo search algorithm in parallel batch processing machines with unequal job ready times. Computers and Industrial Engineering, 124, 2018, pp. 65-76, DOI: 10.1016/j.cie.2018.07.001.
53. Jiang, C., Wan, Z., Peng, Z., A new efficient hybrid algorithm for large scale multiple traveling salesman problems. Expert Systems with Applications, 139, 2020, DOI: 10.1016/j.eswa.2019.112867.
54. Ouyang, H. B., Gao, L. Q., Kong, X. Y., Li, S., Zou, D. X., “Hybrid harmony search particle swarm optimization with global dimension selection”. Information Sciences, 346, 2016, pp. 318-337, DOI: 10.1016/j.ins.2016.02.007.
55. Hansen, N., Ostermeier, A., Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In Proceedings of IEEE international conference on evolutionary computation, 1996, pp. 312-317, DOI: 10.1109/ICEC.1996.542381.
56. Ampellio, E., Vassio, L., A hybrid ABC for expensive optimizations: CEC 2016 competition benchmark. In 2016 IEEE congress on evolutionary computation (CEC), 2016, pp. 1157-1164, DOI: 10.1109/CEC.2016.7743918.
57. Biedrzycki, R., A version of IPOP-CMA-ES algorithm with midpoint for CEC 2017 single objective bound constrained problems. In 2017 IEEE Congress on Evolutionary Computation (CEC), 2017, pp. 1489-1494, DOI: 10.1109/CEC.2017.7969479.
58. Mohamed, A.W., Hadi, A.A., Fattouh, A.M., Jambi, K.M., LSHADE with semi-parameter adaptation hybrid with CMA-ES for solving CEC 2017 benchmark problems. In 2017 IEEE Congress on Evolutionary Computation (CEC), 2017, pp. 145-152, DOI: 10.1109/CEC.2017.7969307.
59. Kumar, A., Misra, R.K., Singh, D., Improving the local search capability of effective butterfly optimizer using covariance matrix adapted retreat phase. In 2017 IEEE Congress on Evolutionary Computation (CEC), 2017, pp. 1835-1842, DOI: 10.1109/CEC.2017.7969524.
60. Zhao, Y.T., Li, W.G., Liu, A., Improved grey wolf optimization based on the two-stage search of hybrid CMA-ES. Soft Computing, 2019, pp. 1-19, DOI: 10.1007/s00500-019-03948-x.
61. Xu, P., Luo, W., Lin, X., Qiao, Y., Zhu, T., Hybrid of PSO and CMA-ES for Global Optimization. In 2019 IEEE Congress on Evolutionary Computation (CEC), 2019, pp. 27-33. IEEE. DOI: 10.1109/CEC.2019.8789912.
62. Chen, X., and Xu, B., Teaching-learning-based artificial bee colony. In International Conference on Swarm Intelligence, 2018, pp. 166-178, Springer, Cham. DOI: 10.1007/978-3-319-93815-8_17.
63. Mirjalili, S., and Gandomi, A.H., Chaotic gravitational constants for the gravitational search algorithm, Applied soft computing, 53, 2017, pp: 407-419, DOI: 10.1016/j.asoc.2017.01.008.
64. Hansen, N., Ostermeier, A., 2001. Completely derandomized self-adaptation in evolution strategies. Evolutionary computation, 9(2), 2001, pp: 159-195. DOI: 10.1162/106365601750190398.
65. Hansen, N., Müller, S.D., Koumoutsakos, P., Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary computation, 11(1), 2003, pp. 1-18, DOI: 10.1162/106365603321828970.
66. Hansen, N., The CMA evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772, 2016.
67. N. H. Awad, M. Z. Ali, J. J. Liang, B. Y. Qu and P. N. Suganthan, “Problem Definitions and Evaluation Criteria for the CEC 2017 Special Session and Competition on Single Objective Bound Constrained Real-Parameter Numerical Optimization”, Technical Report, Nanyang Technological University, Singapore, November 2016.
68. Paçacı, S., Bingöl, O., Güvenç, U., Investigation of SFS Algorithm Parameters and the Determination of Optimum Values. International Journal of Technological Science, 11(2), 2019, pp. 81-93 (In Turkish).