A Hybrid Applied Optimization Algorithm for Training Multi-Layer Neural Networks in Data Classification

Abstract

Backpropagation algorithm is classical technique used in the training of the artificial neural networks. Since this algorithm has many disadvantages, the training of the neural networks has been implemented with various optimization methods. In this paper, a hybrid intelligent model, i.e., hybridGSA, is developed to training artificial neural networks (ANN) and undertaking data classification problems. The hybrid intelligent system aims to exploit the advantages of genetic and simulated annealing algorithms and, at the same time, alleviate their limitations. To evaluate the effectiveness of the hybridGSA method, three benchmark data sets, i.e., Breast Cancer Wisconsin, Pima Indians Diabetes, and Liver Disorders from the UCI Repository of Machine Learning, and a simulation experiment are used for evaluation. A comparative analysis on the real data sets and simulation data shows that the hybridGSA algorithm may offer efficient alternative to traditional training methods for the classification problem.

Keywords

• Artificial neural networks
• data classification
• training of neural networks
• genetic algorithm
• simulated annealing

References

1. [1] Fisher, R.A. “The use of multiple measurements in taxonomy problems” Annals of Eugenics, 7, 179–188 (1936).
2. [2] Xu, G. & Papageorgiou, L.G. “A mixed integer optimization for data classification” Computers & Industrial Engineering 56(4), 1205-1215 (2009).
3. [3] Fred, N. & Glover, F. “A linear programming approach to discriminant problem” Decision Sciences, 12, 68–74 (1981).
4. [4] Fred, N. & Glover, F. “Simple but powerful goal programming models for discriminant problems” European Journal of Operational Research, 7, 44–60 (1981).
5. [5] Lam, K.F., Choo, E.U., Moy, J.W. “Minimizing deviations from the group mean: A new linear programming approach for the two-group classification problem” European Journal of Operational Research, 88, 358–367 (1996).
6. [6] Lam, K.F. & Moy, J.W. “Improved linear programming formulations for the multi-group discriminant problem” Journal of Operational Research Society, 47, 1526–1529 (1996).