PTGNG: An Evolutionary Approach for Parameter Optimization in the Growing Neural Gas Algorithm

Year 2023, , 91 - 101, 30.06.2023

Mohanad Dhari Jassam Alalkawı , Shadi Al Shehabı , Meltem Yıldırım Imamoglu

https://doi.org/10.22399/ijcesen.1282146

Abstract

Growing Neural Gas (GNG) algorithm is an unsupervised learning algorithm which belongs to the competitive learning family. Since then, GNG has been a subject to vaious developments and implementations found in the literatures for two main reasons: first, the number of neurons (i.e., nodes) is adaptive. Meaning, it is periodically changed through adding new neurons and removing old neurons accordingly in order to find the best network which captures the topological structure of the given data, and to reduce the overall error in that representation. Second, GNG algorithm has no restrictions when compared to other competitive learning algorithms, as it is both free in the space and the number of the neurons. In this paper, we propose and implement an evolutionary based approach, namely PTGNG, to tune GNG algorithm parameters for dealing with data in multiple dimensional space, namely, 2D, 3D, and 4D. The idea basically relies on finding the optimum set of parameter values for any given problem to be solved using GNG algorithm. The evolutionary algorithm by its nature searches a vast space of applicable solutions and evaluates each solution individually. When we implemented our approach of parameters tuning, we can note that GNG captured datasets topological structure with a smaller number of neurons and with a better accuracy. It also showed that the same results appeared when working on datasets with three and four dimensions.

Keywords

Growing Neural Gas, Parameter tuning, Evolutionary algorithm

References

• [1] Fritzke, B. (1994). A growing neural gas network learns topologies. Advances in neural information processing systems, 7.
• [2] Fritzke, B. (1997). Some competitive learning methods. Artificial Intelligence Institute, Dresden University of Technology, 100.
• [3] Fritzke, B. (1994). Growing cell structures—a self-organizing network for unsupervised and supervised learning. Neural networks, 7(9), 1441-1460. DOI:10.1016/0893-6080(94)90091-4
• [4] Martinetz, T. and Schulten, K. (1991), “A" neural-gas" network learns topologies,” Artif. Neural Networks, pp. 397–402.
• [5] Qin, A. K., & Suganthan, P. N. (2004). Robust growing neural gas algorithm with application in cluster analysis. Neural networks, 17(8-9), 1135-1148. DOI:10.1016/s0893-6080(04)00166-2
• [6] Andreakis, A., Hoyningen-Huene, N. V., & Beetz, M. (2009). Incremental unsupervised time series analysis using merge growing neural gas. In Advances in Self-Organizing Maps: 7th International Workshop, WSOM 2009, St. Augustine, FL, USA, June 8-10, 2009. Proceedings 7 (pp. 10-18). Springer Berlin Heidelberg. DOI:10.1007/978-3-642-02397-2_2
• [7] Kohonen, T. (1997, June). Exploration of very large databases by self-organizing maps. In Proceedings of international conference on neural networks (icnn'97) (Vol. 1, pp. PL1-PL6). IEEE. DOI:10.1109/icnn.1997.611622
• [8] Qin, A. K., & Suganthan, P. N. (2005). Enhanced neural gas network for prototype-based clustering. Pattern recognition, 38(8), 1275-1288. DOI:10.1016/j.patcog.2004.12.007
• [9] Strickert, M., & Hammer, B. (2005). Merge SOM for temporal data. Neurocomputing, 64, 39-71. DOI:10.1016/j.neucom.2004.11.014
• [10] Lomp, O. (2008). Finding Optimal Parameters for Neural Gas Networks Using Evolutionary Algorithms.
• [11] Al Shehabi, S., & Lamirel, J. C. (2005, July). Multi-Topographic Neural Network Communication and Generalization for Multi-Viewpoint Analysis. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. (Vol. 3, pp. 1564-1569). DOI:10.1109/ijcnn.2005.1556111
• [12] Holland, J. H. (1992). Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press. DOI:10.7551/mitpress/ 1090.003.0007
• [13] Nannen, V., & Eiben, A. E. (2007, September). Efficient relevance estimation and value calibration of evolutionary algorithm parameters. In 2007 IEEE congress on evolutionary computation (pp. 103-110). IEEE. DOI:10.1109/cec.2007.4424460
• [14] Maron, O., & Moore, A. W. (1997). The racing algorithm: Model selection for lazy learners. Artificial Intelligence Review, 11, 193-225. DOI:10.1007/978-94-017-2053-3_8
• [15] Dobslaw, F. (2010). A parameter tuning framework for metaheuristics based on design of experiments and artificial neural networks. In International conference on computer mathematics and natural computing. WASET.
• [16] Goldberg, D. E. (1988). Holland, JH. Genetic Algorithms in Search. Optimization, and Machine Learning. Mach. Learn, 3, 95-99. DOI:10.1023/a:1022602019183
• [17] TAN, R. K., & Şebnem, B. O. R. A. (2017). Parameter tuning algorithms in modeling and simulation. International Journal of Engineering Science and Application, 1(2), 58-66. DOI:10.1109/cicn.2017.8319375
• [18] Ventocilla, E., Martins, R. M., Paulovich, F., & Riveiro, M. (2021). Scaling the growing neural gas for visual cluster analysis. Big Data Research, 26, 100254. DOI:10.1016/j.bdr.2021.100254
• [19] Mendes, C. A. T., Gattass, M., & Lopes, H. (2014). FGNG: A fast multi-dimensional growing neural gas implementation. Neurocomputing, 128, 328-340. DOI:10.1016/j.neucom.2013.08.033
• [20] Fišer, D., Faigl, J., & Kulich, M. (2013). Growing neural gas efficiently. Neurocomputing, 104, 72-82. DOI:10.1016/j.neucom.2012.10.004
• [21] García-Rodríguez, J., Angelopoulou, A., García-Chamizo, J. M., Psarrou, A., Escolano, S. O., & Giménez, V. M. (2012). Autonomous growing neural gas for applications with time constraint: optimal parameter estimation. Neural Networks, 32, 196-208. DOI:10.1016/j.neunet.2012.02.032
• [22] Donatti, G. S., Lomp, O., & Würtz, R. P. (2010, July). Evolutionary optimization of growing neural gas parameters for object categorization and recognition. In The 2010 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE. DOI:10.1109/ijcnn.2010.5596682
• [23] Wall, M. (1996). GAlib: A C++ library of genetic algorithm components. Mechanical Engineering Department, Massachusetts Institute of Technology, 87, 54.
• [24] Fritzke, B. (1994). Fast learning with incremental RBF networks. Neural Process. Lett., 1(1), 2-5. DOI:10.1007/bf02312392
• [25] Fritzke, B. (1995). Growing grid—a self-organizing network with constant neighborhood range and adaptation strength. Neural processing letters, 2, 9-13. DOI:10.1007/bf02332159
• [26] Zaki, M. J., & Meira, W. (2014). Data mining and analysis: fundamental concepts and algorithms. Cambridge University Press. DOI:10.1017/cbo9780511810114
• [27] Lamirel, J. C., & Al Shehabi, S. (2015). Feature maximization based clustering quality evaluation: a promising approach. In Trends and Applications in Knowledge Discovery and Data Mining: PAKDD 2015 Workshops: BigPMA, VLSP, QIMIE, DAEBH, Ho Chi Minh City, Vietnam, May 19-21, 2015. Revised Selected Papers (pp. 210-222). Springer International Publishing. DOI:10.1007/978-3-319-25660-3_18