PTGNG: An Evolutionary Approach for Parameter Optimization in the Growing Neural Gas Algorithm

Öz

Growing Neural Gas (GNG) algorithm is an unsupervised learning algorithm which belongs to the competitive learning family. Since then, GNG has been a subject to vaious developments and implementations found in the literatures for two main reasons: first, the number of neurons (i.e., nodes) is adaptive. Meaning, it is periodically changed through adding new neurons and removing old neurons accordingly in order to find the best network which captures the topological structure of the given data, and to reduce the overall error in that representation. Second, GNG algorithm has no restrictions when compared to other competitive learning algorithms, as it is both free in the space and the number of the neurons. In this paper, we propose and implement an evolutionary based approach, namely PTGNG, to tune GNG algorithm parameters for dealing with data in multiple dimensional space, namely, 2D, 3D, and 4D. The idea basically relies on finding the optimum set of parameter values for any given problem to be solved using GNG algorithm. The evolutionary algorithm by its nature searches a vast space of applicable solutions and evaluates each solution individually. When we implemented our approach of parameters tuning, we can note that GNG captured datasets topological structure with a smaller number of neurons and with a better accuracy. It also showed that the same results appeared when working on datasets with three and four dimensions.

Anahtar Kelimeler

• Growing Neural Gas
• Parameter tuning
• Evolutionary algorithm

Kaynakça

1. [1] Fritzke, B. (1994). A growing neural gas network learns topologies. Advances in neural information processing systems, 7.
2. [2] Fritzke, B. (1997). Some competitive learning methods. Artificial Intelligence Institute, Dresden University of Technology, 100.
3. [3] Fritzke, B. (1994). Growing cell structures—a self-organizing network for unsupervised and supervised learning. Neural networks, 7(9), 1441-1460. DOI:10.1016/0893-6080(94)90091-4
4. [4] Martinetz, T. and Schulten, K. (1991), “A" neural-gas" network learns topologies,” Artif. Neural Networks, pp. 397–402.
5. [5] Qin, A. K., & Suganthan, P. N. (2004). Robust growing neural gas algorithm with application in cluster analysis. Neural networks, 17(8-9), 1135-1148. DOI:10.1016/s0893-6080(04)00166-2
6. [6] Andreakis, A., Hoyningen-Huene, N. V., & Beetz, M. (2009). Incremental unsupervised time series analysis using merge growing neural gas. In Advances in Self-Organizing Maps: 7th International Workshop, WSOM 2009, St. Augustine, FL, USA, June 8-10, 2009. Proceedings 7 (pp. 10-18). Springer Berlin Heidelberg. DOI:10.1007/978-3-642-02397-2_2
7. [7] Kohonen, T. (1997, June). Exploration of very large databases by self-organizing maps. In Proceedings of international conference on neural networks (icnn'97) (Vol. 1, pp. PL1-PL6). IEEE. DOI:10.1109/icnn.1997.611622
8. [8] Qin, A. K., & Suganthan, P. N. (2005). Enhanced neural gas network for prototype-based clustering. Pattern recognition, 38(8), 1275-1288. DOI:10.1016/j.patcog.2004.12.007

9. [9] Strickert, M., & Hammer, B. (2005). Merge SOM for temporal data. Neurocomputing, 64, 39-71. DOI:10.1016/j.neucom.2004.11.014
10. [10] Lomp, O. (2008). Finding Optimal Parameters for Neural Gas Networks Using Evolutionary Algorithms.
11. [11] Al Shehabi, S., & Lamirel, J. C. (2005, July). Multi-Topographic Neural Network Communication and Generalization for Multi-Viewpoint Analysis. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. (Vol. 3, pp. 1564-1569). DOI:10.1109/ijcnn.2005.1556111
12. [12] Holland, J. H. (1992). Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press. DOI:10.7551/mitpress/ 1090.003.0007
13. [13] Nannen, V., & Eiben, A. E. (2007, September). Efficient relevance estimation and value calibration of evolutionary algorithm parameters. In 2007 IEEE congress on evolutionary computation (pp. 103-110). IEEE. DOI:10.1109/cec.2007.4424460
14. [14] Maron, O., & Moore, A. W. (1997). The racing algorithm: Model selection for lazy learners. Artificial Intelligence Review, 11, 193-225. DOI:10.1007/978-94-017-2053-3_8
15. [15] Dobslaw, F. (2010). A parameter tuning framework for metaheuristics based on design of experiments and artificial neural networks. In International conference on computer mathematics and natural computing. WASET.
16. [16] Goldberg, D. E. (1988). Holland, JH. Genetic Algorithms in Search. Optimization, and Machine Learning. Mach. Learn, 3, 95-99. DOI:10.1023/a:1022602019183
17. [17] TAN, R. K., & Şebnem, B. O. R. A. (2017). Parameter tuning algorithms in modeling and simulation. International Journal of Engineering Science and Application, 1(2), 58-66. DOI:10.1109/cicn.2017.8319375
18. [18] Ventocilla, E., Martins, R. M., Paulovich, F., & Riveiro, M. (2021). Scaling the growing neural gas for visual cluster analysis. Big Data Research, 26, 100254. DOI:10.1016/j.bdr.2021.100254
19. [19] Mendes, C. A. T., Gattass, M., & Lopes, H. (2014). FGNG: A fast multi-dimensional growing neural gas implementation. Neurocomputing, 128, 328-340. DOI:10.1016/j.neucom.2013.08.033
20. [20] Fišer, D., Faigl, J., & Kulich, M. (2013). Growing neural gas efficiently. Neurocomputing, 104, 72-82. DOI:10.1016/j.neucom.2012.10.004
21. [21] García-Rodríguez, J., Angelopoulou, A., García-Chamizo, J. M., Psarrou, A., Escolano, S. O., & Giménez, V. M. (2012). Autonomous growing neural gas for applications with time constraint: optimal parameter estimation. Neural Networks, 32, 196-208. DOI:10.1016/j.neunet.2012.02.032
22. [22] Donatti, G. S., Lomp, O., & Würtz, R. P. (2010, July). Evolutionary optimization of growing neural gas parameters for object categorization and recognition. In The 2010 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE. DOI:10.1109/ijcnn.2010.5596682
23. [23] Wall, M. (1996). GAlib: A C++ library of genetic algorithm components. Mechanical Engineering Department, Massachusetts Institute of Technology, 87, 54.
24. [24] Fritzke, B. (1994). Fast learning with incremental RBF networks. Neural Process. Lett., 1(1), 2-5. DOI:10.1007/bf02312392
25. [25] Fritzke, B. (1995). Growing grid—a self-organizing network with constant neighborhood range and adaptation strength. Neural processing letters, 2, 9-13. DOI:10.1007/bf02332159
26. [26] Zaki, M. J., & Meira, W. (2014). Data mining and analysis: fundamental concepts and algorithms. Cambridge University Press. DOI:10.1017/cbo9780511810114
27. [27] Lamirel, J. C., & Al Shehabi, S. (2015). Feature maximization based clustering quality evaluation: a promising approach. In Trends and Applications in Knowledge Discovery and Data Mining: PAKDD 2015 Workshops: BigPMA, VLSP, QIMIE, DAEBH, Ho Chi Minh City, Vietnam, May 19-21, 2015. Revised Selected Papers (pp. 210-222). Springer International Publishing. DOI:10.1007/978-3-319-25660-3_18