A study on chaotic dynamics of deep artificial neural network activated by biological neuron model

Abstract

This paper analyzes the effects of the chaotic signals used by the brain to perform some cognitive functions on the Spiking Neural Network (SNN), defined as the third-generation Artificial Neural Network (ANN) that best represents the biological neuron. In the first phase of the paper, neural networks with different layers are designed to perform classifications like ANN and SNN. Classification performances of these deep networks using the Rectified Linear Unit activation function in ANN mode and the Izhikevich Neuron model in SNN mode are presented comparatively. It is observed that SNNs perform at least as well as ANNs under normal conditions. In the second stage of the study, the classification performances of these deep networks in the SNN mode were analyzed in different chaotic environments, and the findings were reported. In light of the findings, it is seen that SNNs can exhibit a classification success similar to ANNs and maintain this success rate up to a certain chaotic current intensity. Moreover, some levels of chaotic current contribute to the network's classification performance. This is the first study to investigate the chaotic environment behavior of SNNs.

Keywords

• SNN
• Izhikevich neuron
• Chaotic environment
• EEG-Image Classification

References

1. [1]. Rosenblatt, F., 1958. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review 65, 386.
2. [2]. Abdel-rahman M., George D., Geoffrey H., et al. 2009 Deep belief networks for phone recognition. In Nips workshop on deep learning for speech recognition and related applications, volume 1, page 39.
3. [3]. Krizhevsky A., Sutskever I, and Hinton G.X. 2012 Imagenet classification with deep convolutional neural networks,. Curran Associates, Inc., volume 25.
4. [4]. M. Alfaro-Ponce, A. Arg¨uelles, and I. Chairez. 2016 Pattern recognition for electroencephalographic signals based on continuous neural networks. Neural Networks, 79(11):88–96.
5. [5]. Maass W. 1997 Networks of spiking neurons: the third generation of neural network models. Neural networks, 10(9):1659–1671.
6. [6]. Abbott L.F. 1999 Lapicque’s introduction of the integrate-and-fire model neuron (1907). Brain research bulletin, 50(5-6):303–304.
7. [7]. Hodgkin A.L. and Huxley A.F. 1952 A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of physiology, 117(4):500.
8. [8]. Izhikevich. E.M. 2004 Which model to use for cortical spiking neurons? IEEE Transactions on Neural Networks, 15(5):1063–1070.

9. [9]. FitzHugh R. 1961 Impulses and physiological states in theoretical models of nerve membrane. Biophysical journal, 1(6):445–466.
10. [10]. Koch C. and Segev I. 1998 Methods in neuronal modeling: from ions to networks. MIT press.
11. [11]. Ghosh-Dastidar S. and Adeli H. 2009 Spiking neural networks. International journal of neural systems, 19(04):295–308.
12. [12]. Kasabov N., Dhoble K., Nuntalid N., and Indiveri G. 2013 Dynamic evolving spiking neural networks for on-line spatio-and spectro-temporal pattern recognition. Neural Networks, 41:188–201.
13. [13]. Dermot Kerr, Sonya Coleman, and Thomas Martin McGinnity. 2018 Biologically inspired intensity and depth image edge extraction. IEEE Transactions on Neural Networks and Learning Systems, 29(11):5356–5365
14. [14]. Carlos D. Virgilio G., Juan H. Sossa A., Javier M. Antelis, and Luis E. Falc´on. 2020 Spiking neural networks applied to the classification of motor tasks in eeg signals. Neural networks, 122:130–143.
15. [15]. Sanders P.J., Doborjeh Z.G., Doborjeh M.G., Kasabov N.K., and Searchfield G.D. 2021 Prediction of acoustic residual inhibition of tinnitus using a brain-inspired spiking neural network model. Brain Sciences, 11(1):52.
16. [16]. Luo Y., Fu Q., Xie J., Qin J., Wu G., Liu J., Jiang F., Cao Y., and Ding X. 2020 Eeg-based emotion classification using spiking neural networks. IEEE Access, 8:46007–46016.
17. [17]. Wu J., Chua Y., and Li H. A biologically plausible speech recognition framework based on spiking neural networks. International Joint Conference on Neural Networks (IJCNN), 2018, pages 1–8. IEEE.
18. [18]. Wu J., Yılmaz E., Zhang M., Li H., and. Tan. C. K 2020 Deep spiking neural networks for large vocabulary automatic speech recognition. Frontiers in neuroscience, 14:199.
19. [19]. Schmidhuber J. 2015 Deep learning in neural networks: An overview. Neural networks, 61:85–117.
20. [20]. Tavanaei A., Ghodrati M., KheradpishehS.R., Masquelier, and Anthony T. 2019 Deep learning in spiking neural networks. Neural networks, 111:47–63.
21. [21]. Stiefel K.M., Englitz B., and Sejnowski T.J. 2013 Origin of intrinsic irregular firing in cortical interneurons. Proceedings of the National Academy of Sciences, 110(19):7886–7891.
22. [22]. Rasmussen R., Jensen M.H., and Heltberg M.L. 2017 Chaotic dynamics mediate brain state transitions, driven by changes in extracellular ion concentrations. Cell systems, 5(6):591–603.
23. [23]. Hayashi H., Ishizuka S., and Hirakawa K. 1985 Chaotic response of the pacemaker neuron. Journal of the Physical Society of Japan, 54(6):2337–2346.
24. [24]. Yao Y., Ma J., Gui R, and Cheng G. 2021 Enhanced logical chaotic resonance. Chaos: An Interdisciplinary Journal of Nonlinear Science, 31(2):023103.
25. [25]. Yao Y. and Ma J. 2020 Logical chaotic resonance in a bistable system. International Journal of Bifurcation and Chaos, 30(13):2050196.
26. [26]. Lal T., Hinterberger G., Widman T., Schrder G., Hill J., Rosenstiel W., Elger C., Schlkopf B., and Birbaumer B. 2005 Methods Towards Invasive Human Brain Computer Interfaces. The MIT Press,.
27. [27]. LeCun Y., Bottou L., Bengio Y., and Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998 86(11):2278–2324.
28. [28]. Peng Z., Chu F., and He Y. 2002 Vibration signal analysis and feature extraction based on reassigned wavelet scalogram. Journal of Sound and Vibration, 253(5):1087–1100.
29. [29]. Ma J., Ying H., and Pu Z. 2005 An anti-control scheme for spiral under lorenz chaotic signals. Chinese Physics Letters, 22(5):1065–1068.
30. [30]. Beppu K., Kubo N., and Matsui K. 2021 Glial amplification of synaptic signals. The Journal of Physiology, 599(7):2085–2102.
31. [31]. Erkan E. and Kurnaz I. 2017 A study on the effect of psychophysiological signal features on classification methods. Measurement, 101:45.-52.
32. [32]. Hansel D. and Mato G. 2013 Short-term plasticity explains irregular persistent activity in working memory tasks. Journal of Neuroscience, 33(1):133–149.