İĞNECİKLİ SİNİR AĞLARININ FPGA TABANLI YENİDEN YAPILANDIRILABİLİR UYGULAMALARI: MİNİ DERLEME

Year 2022, Volume: 2 Issue: 2, 152 - 161, 21.06.2022

Oğuzhan Yıldırım , Özden Niyaz , Burcu Erkmen

Abstract

Günümüzde, günlük hayatımızda sistemlerin artan veri kapasitesi, düşük güç tüketimi ve yüksek hızlı veri işleme beklentileri nedeniyle Von Neumann darboğazı geçmişe göre daha önemli bir sorun haline gelmiştir. Bu nedenle, geleneksel bilgisayar mimarileri artık günümüzün gereksinimlerini tam olarak karşılayamamaktadır. Nöromorfik tasarımlar, düşük güç tüketimi ile büyük miktarda veriyi hızlı bir şekilde işleme açısından insan beynini taklit edebildiklerinden, alternatif bir çözüm olarak görülmektedir. Geleneksel Yapay Sinir Ağı yöntemlerinin başarısı tatmin edici olsa da, biyolojik sistemler güç tüketimi açısından hala çok daha avantajlıdır. Biyolojik olarak en gerçekçi ve üçüncü nesil sinir ağları olarak anılan İğnecikli Sinir Ağlarına dayalı nöromorfik donanım mimarileri, Von Neumann darboğazını aşarak akıllı sistemler için daha uygun bir donanım yapısı sağlamaktadır. Nöromorfik mimarilerin uygulanması için yeniden yapılandırılabilir donanımın kullanılması, entegre devreler ve hesaplama yaklaşımlarından daha hızlı ve güncellenebilir bir araştırma alanı yaratmaktadır. Bu sebeple, bu çalışma kapsamında, literatürdeki İğnecikli Sinir Ağlarının FPGA tabanlı yeniden yapılandırılabilir uygulamaları gözden geçirilmiş ve bu çalışmalar güç tüketimi, öğrenme yeteneği, kaynak tüketimi ve doğruluk oranları açısından karşılaştırılmıştır.

Keywords

İğnecikli Sinir Ağları, Yeniden Yapılandırılabilir Uygulamalar, FPGA, Nöromorfik

FPGA BASED RECONFIGURABLE IMPLEMENTATIONS OF SPIKING NEURAL NETWORKS: A MINI REVIEW

Abstract

Due to the increasing data capacity, low power consumption, and high-speed data processing expectations of systems in our daily lives today, the Von Neumann bottleneck has become a more important problem than in the past. For these reasons, conventional computer architectures can no longer fully meet today's requirements. Neuromorphic designs have been considered as an alternative solution to all, as they are able to mimic the human brain in terms of processing large amounts of data quickly with low power consumption. Although the success of traditional Artificial Neural Network methods is satisfactory, biological systems are still much more advantageous in terms of power consumption. Neuromorphic hardware architectures based on spiking neural networks, which are the most biologically plausible and are referred to as third-generation neural networks, overcome the Von Neumann bottleneck and provide a more suitable hardware structure for intelligent systems. The use of reconfigurable hardware for the implementation of neuromorphic architectures creates a faster and updatable research field than integrated circuits and computational approaches. Therefore, this study has reviewed FPGA-based reconfigurable implementations of Spiking Neural Networks in the literature and compared these studies in terms of power consumption, learning capability, resource consumption, and accuracy.

Keywords

Spiking Neural Networks, Reconfigurable Implementations, FPGA, Neuromorphic

References

• Akbarzadeh-Sherbaf, K., Safari, S., & Vahabie, A.-H. (2020). A digital hardware implementation of spiking neural networks with binary FORCE training. Neurocomputing, 412, 129–142. https://doi.org/10.1016/j.neucom.2020.05.044
• Ambroise, M., Levi, T., Bornat, Y., & Saïghi, S. (2013). Biorealistic spiking neural network on FPGA. 2013 47th Annual Conference on Information Sciences and Systems (CISS). https://doi.org/10.1109/ciss.2013.6616689
• Cerezuela-Escudero, E., Jimenez-Fernandez, A., Paz-Vicente, R., Dominguez-Morales, M., Linares-Barranco, A., & Jimenez-Moreno, G. (2015). Musical notes classification with neuromorphic auditory system using FPGA and a convolutional spiking network. 2015 International Joint Conference on Neural Networks (IJCNN). https://doi.org/10.1109/ijcnn.2015.7280619
• Chen, G. K., Kumar, R., Sumbul, H. E., Knag, P. C., & Krishnamurthy, R. K. (2018). A 4096-Neuron 1M-Synapse 3.8PJ/SOP Spiking Neural Network with On-Chip STDP Learning and Sparse Weights in 10NM FinFET CMOS. 2018 IEEE Symposium on VLSI Circuits. https://doi.org/10.1109/vlsic.2018.8502423
• Diehl, P. U., & Cook, M. (2015). Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in Computational Neuroscience, 9. https://doi.org/10.3389/fncom.2015.00099
• Fang, H., Mei, Z., Shrestha, A., Zhao, Z., Li, Y., & Qiu, Q. (2020). Encoding, model, and architecture. Proceedings of the 39th International Conference on Computer-Aided Design. https://doi.org/10.1145/3400302.3415608
• Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014). Part II Generalized Integrate-and-Fire Neurons | Neuronal Dynamics online book. Retrieved April 20, 2022, from Epfl.ch website: https://neuronaldynamics.epfl.ch/online/Pt2.html
• Glackin, B., Harkin, J., McGinnity, T. M., Maguire, L. P., & Wu, Q. (2009). Emulating Spiking Neural Networks for edge detection on FPGA hardware. 2009 International Conference on Field Programmable Logic and Applications. https://doi.org/10.1109/fpl.2009.5272339
• Grassia, F., Kohno, T., & Levi, T. (2016). Digital hardware implementation of a stochastic two-dimensional neuron model. Journal of Physiology-Paris, 110(4), 409–416. https://doi.org/10.1016/j.jphysparis.2017.02.002
• Han, J., Li, Z., Zheng, W., & Zhang, Y. (2020). Hardware implementation of spiking neural networks on FPGA. Tsinghua Science and Technology, 25(4), 479–486. https://doi.org/10.26599/tst.2019.9010019
• Heidarpur, M., Ahmadi, A., Ahmadi, M., & Rahimi Azghadi, M. (2019). CORDIC-SNN: On-FPGA STDP Learning With Izhikevich Neurons. IEEE Transactions on Circuits and Systems I: Regular Papers, 66(7), 2651–2661. https://doi.org/10.1109/tcsi.2019.2899356
• Humaidi, A. J., Kadhim, T. M., Hasan, S., Kasim Ibraheem, I., & Taher Azar, A. (2020). A Generic Izhikevich-Modelled FPGA-Realized Architecture: A Case Study of Printed English Letter Recognition. 2020 24th International Conference on System Theory, Control and Computing (ICSTCC). https://doi.org/10.1109/icstcc50638.2020.9259707
• Iakymchuk, T., Rosado-Muñoz, A., Guerrero-Martínez, J. F., Bataller-Mompeán, M., & Francés-Víllora, J. V. (2015). Simplified spiking neural network architecture and STDP learning algorithm applied to image classification. EURASIP Journal on Image and Video Processing, 2015(1). https://doi.org/10.1186/s13640-015-0059-4
• Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(6), 1569–1572. https://doi.org/10.1109/tnn.2003.820440
• Ju, X., Fang, B., Yan, R., Xu, X., & Tang, H. (2020). An FPGA Implementation of Deep Spiking Neural Networks for Low-Power and Fast Classification. Neural Computation, 32(1), 182–204. https://doi.org/10.1162/neco_a_01245
• Lammie, C., Hamilton, T., & Azghadi, M. R. (2018). Unsupervised Character Recognition with a Simplified FPGA Neuromorphic System. 2018 IEEE International Symposium on Circuits and Systems (ISCAS). https://doi.org/10.1109/iscas.2018.8351532
• LeCun, Y., Cortes, C., & Burges, C. (2012). MNIST handwritten digit database. Retrieved April 20, 2022, from Lecun.com website: http://yann.lecun.com/exdb/mnist/
• Lemaire, E., Moretti, M., Daniel, L., Miramond, B., Millet, P., Feresin, F., & Bilavarn, S. (2020). An FPGA-Based Hybrid Neural Network Accelerator for Embedded Satellite Image Classification. 2020 IEEE International Symposium on Circuits and Systems (ISCAS). https://doi.org/10.1109/iscas45731.2020.9180625
• Li, S., Zhang, Z., Mao, R., Xiao, J., Chang, L., & Zhou, J. (2021). A Fast and Energy-Efficient SNN Processor With Adaptive Clock/Event-Driven Computation Scheme and Online Learning. IEEE Transactions on Circuits and Systems I: Regular Papers, 68(4), 1543–1552. https://doi.org/10.1109/tcsi.2021.3052885
• Liu, K., Cui, X., Zhong, Y., Kuang, Y., Wang, Y., Tang, H., & Huang, R. (2019). A Hardware Implementation of SNN-Based Spatio-Temporal Memory Model. Frontiers in Neuroscience, 13. https://doi.org/10.3389/fnins.2019.00835
• Losh, M., & Llamocca, D. (2019). A Low-Power Spike-like Neural Network Design. Electronics, 8(12), 1479. https://doi.org/10.3390/electronics8121479
• Maass, W. (1997). Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10(9), 1659–1671. https://doi.org/10.1016/s0893-6080(97)00011-7
• Markram, H., Gerstner, W., & Sjöström, P. J. (2012). Spike-Timing-Dependent Plasticity: A Comprehensive Overview. Frontiers in Synaptic Neuroscience, 4. https://doi.org/10.3389/fnsyn.2012.00002
• Miró-Amarante, L., Gómez-Rodríguez, F., Jiménez-Fernández, A., & Jiménez-Moreno, G. (2017). A spiking neural network for real-time Spanish vowel phonemes recognition. Neurocomputing, 226, 249–261. https://doi.org/10.1016/j.neucom.2016.12.005
• Mostafa, H., Pedroni, B. U., Sheik, S., & Cauwenberghs, G. (2017). Fast classification using sparsely active spiking networks. 2017 IEEE International Symposium on Circuits and Systems (ISCAS). https://doi.org/10.1109/iscas.2017.8050527
• Murali, S., Kumar, J., Kumar, J., & Bhakthavatchalu, R. (2016). Design and implementation of Izhikevich spiking neuron model on FPGA. 2016 IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT). https://doi.org/10.1109/rteict.2016.7807968
• Neil, D., & Liu, S.-C. (2014). Minitaur, an Event-Driven FPGA-Based Spiking Network Accelerator. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 22(12), 2621–2628. https://doi.org/10.1109/tvlsi.2013.2294916
• Ou, Q.-F., Xiong, B.-S., Yu, L., Wen, J., Wang, L., & Tong, Y. (2020). In-Memory Logic Operations and Neuromorphic Computing in Non-Volatile Random Access Memory. Materials, 13(16), 3532. https://doi.org/10.3390/ma13163532
• Rice, K. L., Bhuiyan, M. A., Taha, T. M., Vutsinas, C. N., & Smith, M. C. (2009). FPGA Implementation of Izhikevich Spiking Neural Networks for Character Recognition. 2009 International Conference on Reconfigurable Computing and FPGAs. https://doi.org/10.1109/reconfig.2009.77
• Schuman, C. D., Potok, T. E., Patton, R. M., Douglas, B. J., Dean, M. E., Rose, G. S., & Plank, J. S. (2017). A Survey of Neuromorphic Computing and Neural Networks in Hardware. ArXiv.org. https://doi.org/10.48550/arXiv.1705.06963
• Si, J., & Harris, S. L. (2018). Handwritten digit recognition system on an FPGA. 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC). https://doi.org/10.1109/ccwc.2018.8301757
• Sun, B., Guo, T., Zhou, G., Ranjan, S., Jiao, Y., Wei, L., … Wu, Y. A. (2021). Synaptic devices based neuromorphic computing applications in artificial intelligence. Materials Today Physics, 18, 100393. https://doi.org/10.1016/j.mtphys.2021.100393
• Sun, Q. Y., Wu, Q. X., Wang, X., & Hou, L. (2017). A spiking neural network for extraction of features in colour opponent visual pathways and FPGA implementation. Neurocomputing, 228, 119–132. https://doi.org/10.1016/j.neucom.2016.09.093
• T, P., A, A. B. R., & J, A. (2014). FPGA IMPLEMENTATION OF ADAPTIVE INTEGRATED SPIKING NEURAL NETWORK FOR EFFICIENT IMAGE RECOGNITION SYSTEM. ICTACT Journal on Image and Video Processing, 4(4), 848–852. https://doi.org/10.21917/ijivp.2014.0122
• Tang, H., Cho, D., Lew, D., Kim, T., & Park, J. (2020). Rank order coding based spiking convolutional neural network architecture with energy-efficient membrane voltage updates. Neurocomputing, 407, 300–312. https://doi.org/10.1016/j.neucom.2020.05.031
• Von Neumann, J. (1993). First draft of a report on the EDVAC. IEEE Annals of the History of Computing, 15(4), 27–75. https://doi.org/10.1109/85.238389
• Wang, Q., Li, Y., Shao, B., Dey, S., & Li, P. (2017). Energy efficient parallel neuromorphic architectures with approximate arithmetic on FPGA. Neurocomputing, 221, 146–158. https://doi.org/10.1016/j.neucom.2016.09.071
• Wang, R., Cohen, G., Stiefel, K. M., Hamilton, T. J., Tapson, J., & van Schaik, A. (2013). An FPGA Implementation of a Polychronous Spiking Neural Network with Delay Adaptation. Frontiers in Neuroscience, 7. https://doi.org/10.3389/fnins.2013.00014
• Yan, Y., Kappel, D., Neumarker, F., Partzsch, J., Vogginger, B., Hoppner, S., … Mayr, C. (2019). Efficient Reward-Based Structural Plasticity on a SpiNNaker 2 Prototype. IEEE Transactions on Biomedical Circuits and Systems, 13(3), 579–591. https://doi.org/10.1109/tbcas.2019.2906401
• Zhang, G., Li, B., Wu, J., Wang, R., Lan, Y., Sun, L., … Chen, Y. (2020). A low-cost and high-speed hardware implementation of spiking neural network. Neurocomputing, 382, 106–115. https://doi.org/10.1016/j.neucom.2019.11.045
• Zhang, J., Wu, H., Wei, J., Wei, S., & Chen, H. (2019). An Asynchronous Reconfigurable SNN Accelerator With Event-Driven Time Step Update. 2019 IEEE Asian Solid-State Circuits Conference (A-SSCC). https://doi.org/10.1109/a-sscc47793.2019.9056903
• Zhao, F., Zeng, Y., & Xu, B. (2018). A Brain-Inspired Decision-Making Spiking Neural Network and Its Application in Unmanned Aerial Vehicle. Frontiers in Neurorobotics, 12. https://doi.org/10.3389/fnbot.2018.00056