A New Conjugate Gradient Method for Learning Fuzzy Neural Networks

Year 2020, Volume: 3 Issue: 2, 57 - 69, 25.03.2021

Hisham Mohammed , Khalil K. Abbo

Abstract

In this paper, we suggest a conjugate gradient method, which belongs to the op-timization methods for learning a fuzzy neural network model that is based on Takagi Sugeno. Where we developed a new algorithm based on the Polak–Ribière–Polak (PRP) method, The technique developed is converging by assum-ing a certain hypothesis. The numerical results indicate the efficacy of the method developed for classifying data as shown in the table as the new method was supe-rior to the Polak–Ribière–Polak (PRP) and Liu-Storey (LS) methods in average training time, Average training accuracy, Average test accuracy, Average train-ing MSE, and Average test MSE. As for the figures, we showed the superiority of the new algorithm in The average training accuracy and The average training error Compared to Polak–Ribière–Polak (PRP) and Liu-Storey (LS) methods, in 100 No. of training iteration.

Keywords

algorithm, classification, fuzzy neural networks, techniques, optimization

References

• S. Chakraborty, A. Konar, A. Ralescu, and N. R. Pal, A fast algorithm to compute precise type-2 centroids for real-time control applications, IEEE Trans. Cybern., 45(2) 2014, 340-353.
• M. Sugeno and G. T. Kang, Structure identification of fuzzy model, Fuzzy Sets Syst., 28 (1), 1988, 15–33.
• T. Takagi and M. Sugeno, Fuzzy identification of systems and its applications to modeling and control, IEEE Trans. Syst. Man. Cybern., 1 1985, 116–132.
• J. C. Bezdek, J. Keller, R. Krisnapuram, and N. Pal, Fuzzy models and algorithms for pattern recognition and image processing, Springer Science & Business Media, vol. 4. 1999.
• J. C. Bezdek, Objective function clustering, in Pattern recognition with fuzzy objective function algorithms, Springer, 1981, 43–93.
• X. Gu and S. Wang, Bayesian Takagi–Sugeno–Kang fuzzy model and its joint learning of structure identification and parameter estimation, IEEE Trans. Ind. Informatics, 14(12) 2018, 5327–5337.
• R. R. Yager and D. P. Filev, Generation of fuzzy rules by mountain clustering, J. Intell. Fuzzy Syst., 2(3) 1994, 209–219.
• R. Krishnapuram and J. M. Keller, A possibilistic approach to clustering, IEEE Trans. fuzzy Syst., 1(2) 1993, 98–110.
• N. R. Pal, K. Pal, J. M. Keller, and J. C. Bezdek, A possibilistic fuzzy c-means clustering algorithm, IEEE Trans. Fuzzy Syst., 13(4) 2005, 517–530.
• C.-F. Juang and C.-T. Lin, An online self-constructing neural fuzzy inference network and its applications, IEEE Trans. Fuzzy Syst., 6(1) 1998, 12–32.
• H. Shahparast, E. G. Mansoori, and M. Z. Jahromi, AFCGD: an adaptive fuzzy classifier based on gradient descent, Soft Comput., 23(12) 2019, 4557–4571.
• X. Gu, F.-L. Chung, H. Ishibuchi, and S. Wang, Imbalanced TSK fuzzy classifier by cross-class Bayesian fuzzy clustering and imbalance learning, IEEE Trans. Syst. Man, Cybern. Syst., 47(8) 2016, 2005–2020.
• X. Gu, F.-L. Chung, and S. Wang, Bayesian Takagi–Sugeno–Kang fuzzy classifier, IEEE Trans. Fuzzy Syst., 25(6) 2016, 1655–1671.
• H. Ichihashi and I. B. Türksen, A neuro-fuzzy approach to data analysis of pairwise comparisons, Int. J. Approx. Reason., 9(3) 1993, 227–248.
• J. M. Mendel, General type-2 fuzzy logic systems made simple: a tutorial, IEEE Trans. Fuzzy Syst., 22(5) 2013, 1162–1182.
• W. Wu, L. Li, J. Yang, and Y. Liu, A modified gradient-based neuro-fuzzy learning algorithm and its convergence, Inf. Sci. (Ny)., 2010, doi: 10.1016/j.ins.2009.12.030.
• H. Ahmad, T. A. Khan, P. S. Stanimirović, Y. M. Chu, and I. Ahmad, Modified variational iteration algorithm-II: Convergence and applications to diffusion models, Complexity, 2020, doi: 10.1155/2020/8841718.
• A. Ghosh, N. R. Pal, and J. Das, A fuzzy rule based approach to cloud cover estimation, Remote Sens. Environ., 100(4) 2006, 531–549.
• N. R. Pal and S. Saha, Simultaneous structure identification and fuzzy rule generation for Takagi–Sugeno models, IEEE Trans. Syst. Man, Cybern. Part B, 38( 6) 2008, 1626–1638.
• H. Ahmad, A. Akgül, T. A. Khan, P. S. Stanimirović, and Y. M. Chu, New perspective on the conventional solutions of the nonlinear time-fractional partial differential equations, Complexity, 2020, doi: 10.1155/2020/8829017.
• H. Ahmad, T. A. Khan, and C. Cesarano, Numerical solutions of coupled burgers’ equations, Axioms, 2019, doi: 10.3390/axioms8040119.
• J. Wang, W. Wu, and J. M. Zurada, Deterministic convergence of conjugate gradient method for feedforward neural networks, Neurocomputing, 74(14–15) 2011, 2368–2376.
• M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, 49( 1). NBS Washington, DC, 1952.
• R. Fletcher and C. M. Reeves, Function minimization by conjugate gradients, Comput. J., 7(2) 1964, 149–154, doi: 10.1093/comjnl/7.2.149.
• E. Polak and G. Ribiere, Note sur la convergence de méthodes de directions conjuguées, ESAIM Math. Model. Numer. Anal. Mathématique Anal. Numérique, 3(1) 1969, 35–43.
• E. P. de Aguiar et al., EANN 2014: a fuzzy logic system trained by conjugate gradient methods for fault classification in a switch machine, Neural Comput. Appl., 27(5) 2016, 1175–1189.
• T. Gao, J. Wang, B. Zhang, H. Zhang, P. Ren, and N. R. Pal, A polak-ribière-polyak conjugate gradient-based neuro-fuzzy network and its convergence, IEEE Access, 2018, doi: 10.1109/ACCESS.2018.2848117.
• H. Ahmad, A. R. Seadawy, and T. A. Khan, Study on numerical solution of dispersive water wave phenomena by using a reliable modification of variational iteration algorithm, Math. Comput. Simul., 2020, doi: 10.1016/j.matcom.2020.04.005.
• I. Del Campo, J. Echanobe, G. Bosque, and J. M. Tarela, Efficient hardware/software implementation of an adaptive neuro-fuzzy system, IEEE Trans. Fuzzy Syst., 16(3) 2008, 761–778.
• K. T. Chaturvedi, M. Pandit, and L. Srivastava, Modified neo-fuzzy neuron-based approach for economic and environmental optimal power dispatch, Appl. Soft Comput., 8(4) 2008, 1428–1438.
• H. Ichihashi, Iterative fuzzy modelling and a hierarchical network, 1991.
• C.-J. Lin and W.-H. Ho, An asymmetry-similarity-measure-based neural fuzzy inference system, Fuzzy Sets Syst., 152(3) 2005, 535–551.
• M. Tang, K. jun Wang, and Y. Zhang, A research on chaotic recurrent fuzzy neural network and its convergence, in 2007 International Conference on Mechatronics and Automation, 2007, 682–687.
• J.-S. Jang, ANFIS: adaptive-network-based fuzzy inference system, IEEE Trans. Syst. Man. Cybern., 23(3) 1993, 665–685.
• X. G. Luo, D. Liu, and B. W. Wan, An adaptive fuzzy neural inferring network, Fuzzy Syst. Math., 12(4) 1998, 26–33.
• C.-F. Juang and J.-S. Chen, Water bath temperature control by a recurrent fuzzy controller and its FPGA implementation, IEEE Trans. Ind. Electron., 53(3) 2006, 941–949.
• A. Sahiner , N. Yilmaz and S. A. Ibrahem, Smoothing approximations to non-smooth functions, J. Multidiscip. Model. Optim., 1(2) 2018, 69-74.
• A. S. Ahmed, Optimization Methods For Learning Artificial Neural Networks, University of Mosul, 2018.
• A. Sahiner and S. A. Ibrahem, A new global optimization technique by auxiliary function method in a directional search, Optim. Lett., 2019, doi: 10.1007/s11590-018-1315-1.
• Y. Liu and C. Storey, Efficient generalized conjugate gradient algorithms, part 1: theory, J. Optim. Theory Appl., 69(1) 1991, 129–137.
• K. K. Abbo and H. M. Khudhur, New A hybrid Hestenes-Stiefel and Dai-Yuan conjugate gradient algorithms for unconstrained optimization, Tikrit J. Pure Sci., 21(1) 2015, 118–123.
• Y. A. Laylani, K. K. Abbo, and H. M. Khudhur, Training feed forward neural network with modified Fletcher-Reeves method, J. Multidiscip. Model. Optim., 1(1) 2018, 14–22.
• K. K. Abbo, Y. A. Laylani, and H. M. Khudhur, Proposed new Scaled conjugate gradient algorithm for unconstrained optimization, Int. J. Enhanc. Res. Sci. Technol. Eng., 5(7) 2016.
• Z. M. Abdullah, M. Hameed, M. K. Hisham, and M. A. Khaleel, Modified new conjugate gradient method for Unconstrained Optimization, Tikrit J. Pure Sci., 24(5) 2019, 86–90.
• H. M. Khudhur, Numerical and analytical study of some descent algorithms to solve unconstrained Optimization problems, University of Mosul, 2015.
• K. K. Abbo, Y. A. Laylani, and H. M. Khudhur, A new spectral conjugate gradient algorithm for unconstrained optimization, Int. J. Math. Comput. Appl. Res., 8 2018, 1–9.
• M. Al-Baali, Descent property and global convergence of the Fletcher—Reeves method with inexact line search, IMA J. Numer. Anal., 5(1) 1985, 121–124.
• L. Zhang and W. Zhou, Two descent hybrid conjugate gradient methods for optimization, J. Comput. Appl. Math., 216(1) 2008, 251–264.
• K. K. Abbo and H. M. Khudhur, New A hybrid conjugate gradient Fletcher-Reeves and Polak-Ribiere algorithm for unconstrained optimization, Tikrit J. Pure Sci., 21(1) 2015,124–129.
• H. N. Jabbar, K. K. Abbo, and H. M. Khudhur, Four--Term Conjugate Gradient (CG) Method Based on Pure Conjugacy Condition for Unconstrained Optimization, Kirkuk Univ. J. Sci. Stud., 13(2) 2018, 101–113.
• T. Gao, Z. Zhang, Q. Chang, X. Xie, P. Ren, and J. Wang, Conjugate gradient-based Takagi-Sugeno fuzzy neural network parameter identification and its convergence analysis, Neurocomputing, 2019, doi: 10.1016/j.neucom.2019.07.035.