TY  - JOUR
T1  - FPGA BASED RECONFIGURABLE IMPLEMENTATIONS OF SPIKING NEURAL NETWORKS:  A MINI REVIEW
TT  - İĞNECİKLİ SİNİR AĞLARININ FPGA TABANLI YENİDEN YAPILANDIRILABİLİR UYGULAMALARI: MİNİ DERLEME
AU  - Yıldırım, Oğuzhan
AU  - Niyaz, Özden
AU  - Erkmen, Burcu
PY  - 2022
DA  - June
JF  - Tasarım Mimarlık ve Mühendislik Dergisi
JO  - DAE
PB  - Fenerbahçe Üniversitesi
WT  - DergiPark
SN  - 2757-9093
SP  - 152
EP  - 161
VL  - 2
IS  - 2
LA  - en
AB  - Due to the increasing data capacity, low power consumption, and high-speed data processing expectations of systems in our daily lives today, the Von Neumann bottleneck has become a more important problem than in the past. For these reasons, conventional computer architectures can no longer fully meet today's requirements. Neuromorphic designs have been considered as an alternative solution to all, as they are able to mimic the human brain in terms of processing large amounts of data quickly with low power consumption. Although the success of traditional Artificial Neural Network methods is satisfactory, biological systems are still much more advantageous in terms of power consumption. Neuromorphic hardware architectures based on spiking neural networks, which are the most biologically plausible and are referred to as third-generation neural networks, overcome the Von Neumann bottleneck and provide a more suitable hardware structure for intelligent systems. The use of reconfigurable hardware for the implementation of neuromorphic architectures creates a faster and updatable research field than integrated circuits and computational approaches. Therefore, this study has reviewed FPGA-based reconfigurable implementations of Spiking Neural Networks in the literature and compared these studies in terms of power consumption, learning capability, resource consumption, and accuracy.
KW  - Spiking Neural Networks
KW  - Reconfigurable Implementations
KW  - FPGA
KW  - Neuromorphic
N2  - Günümüzde, günlük hayatımızda sistemlerin artan veri kapasitesi, düşük güç tüketimi ve yüksek hızlı veri işleme beklentileri nedeniyle Von Neumann darboğazı geçmişe göre daha önemli bir sorun haline gelmiştir. Bu nedenle, geleneksel bilgisayar mimarileri artık günümüzün gereksinimlerini tam olarak karşılayamamaktadır. Nöromorfik tasarımlar, düşük güç tüketimi ile büyük miktarda veriyi hızlı bir şekilde işleme açısından insan beynini taklit edebildiklerinden, alternatif bir çözüm olarak görülmektedir. Geleneksel Yapay Sinir Ağı yöntemlerinin başarısı tatmin edici olsa da, biyolojik sistemler güç tüketimi açısından hala çok daha avantajlıdır. Biyolojik olarak en gerçekçi ve üçüncü nesil sinir ağları olarak anılan İğnecikli Sinir Ağlarına dayalı nöromorfik donanım mimarileri, Von Neumann darboğazını aşarak akıllı sistemler için daha uygun bir donanım yapısı sağlamaktadır. Nöromorfik mimarilerin uygulanması için yeniden yapılandırılabilir donanımın kullanılması, entegre devreler ve hesaplama yaklaşımlarından daha hızlı ve güncellenebilir bir araştırma alanı yaratmaktadır. Bu sebeple, bu çalışma kapsamında, literatürdeki İğnecikli Sinir Ağlarının FPGA tabanlı yeniden yapılandırılabilir uygulamaları gözden geçirilmiş ve bu çalışmalar güç tüketimi, öğrenme yeteneği, kaynak tüketimi ve doğruluk oranları açısından karşılaştırılmıştır.
CR  - Akbarzadeh-Sherbaf, K., Safari, S., & Vahabie, A.-H. (2020). A digital hardware implementation of spiking neural networks with binary FORCE training. Neurocomputing, 412, 129–142. https://doi.org/10.1016/j.neucom.2020.05.044
CR  - Ambroise, M., Levi, T., Bornat, Y., & Saïghi, S. (2013). Biorealistic spiking neural network on FPGA. 2013 47th Annual Conference on Information Sciences and Systems (CISS). https://doi.org/10.1109/ciss.2013.6616689
CR  - Cerezuela-Escudero, E., Jimenez-Fernandez, A., Paz-Vicente, R., Dominguez-Morales, M., Linares-Barranco, A., & Jimenez-Moreno, G. (2015). Musical notes classification with neuromorphic auditory system using FPGA and a convolutional spiking network. 2015 International Joint Conference on Neural Networks (IJCNN). https://doi.org/10.1109/ijcnn.2015.7280619
CR  - Chen, G. K., Kumar, R., Sumbul, H. E., Knag, P. C., & Krishnamurthy, R. K. (2018). A 4096-Neuron 1M-Synapse 3.8PJ/SOP Spiking Neural Network with On-Chip STDP Learning and Sparse Weights in 10NM FinFET CMOS. 2018 IEEE Symposium on VLSI Circuits. https://doi.org/10.1109/vlsic.2018.8502423
CR  - Diehl, P. U., & Cook, M. (2015). Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in Computational Neuroscience, 9. https://doi.org/10.3389/fncom.2015.00099
CR  - Fang, H., Mei, Z., Shrestha, A., Zhao, Z., Li, Y., & Qiu, Q. (2020). Encoding, model, and architecture. Proceedings of the 39th International Conference on Computer-Aided Design. https://doi.org/10.1145/3400302.3415608
CR  - Gerstner, W., Kistler, W. M., Naud, R., & Paninski, L. (2014). Part II Generalized Integrate-and-Fire Neurons | Neuronal Dynamics online book. Retrieved April 20, 2022, from Epfl.ch website: https://neuronaldynamics.epfl.ch/online/Pt2.html
CR  - Glackin, B., Harkin, J., McGinnity, T. M., Maguire, L. P., & Wu, Q. (2009). Emulating Spiking Neural Networks for edge detection on FPGA hardware. 2009 International Conference on Field Programmable Logic and Applications. https://doi.org/10.1109/fpl.2009.5272339
CR  - Grassia, F., Kohno, T., & Levi, T. (2016). Digital hardware implementation of a stochastic two-dimensional neuron model. Journal of Physiology-Paris, 110(4), 409–416. https://doi.org/10.1016/j.jphysparis.2017.02.002
CR  - Han, J., Li, Z., Zheng, W., & Zhang, Y. (2020). Hardware implementation of spiking neural networks on FPGA. Tsinghua Science and Technology, 25(4), 479–486. https://doi.org/10.26599/tst.2019.9010019
CR  - Heidarpur, M., Ahmadi, A., Ahmadi, M., & Rahimi Azghadi, M. (2019). CORDIC-SNN: On-FPGA STDP Learning With Izhikevich Neurons. IEEE Transactions on Circuits and Systems I: Regular Papers, 66(7), 2651–2661. https://doi.org/10.1109/tcsi.2019.2899356
CR  - Humaidi, A. J., Kadhim, T. M., Hasan, S., Kasim Ibraheem, I., & Taher Azar, A. (2020). A Generic Izhikevich-Modelled FPGA-Realized Architecture: A Case Study of Printed English Letter Recognition. 2020 24th International Conference on System Theory, Control and Computing (ICSTCC). https://doi.org/10.1109/icstcc50638.2020.9259707
CR  - Iakymchuk, T., Rosado-Muñoz, A., Guerrero-Martínez, J. F., Bataller-Mompeán, M., & Francés-Víllora, J. V. (2015). Simplified spiking neural network architecture and STDP learning algorithm applied to image classification. EURASIP Journal on Image and Video Processing, 2015(1). https://doi.org/10.1186/s13640-015-0059-4
CR  - Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(6), 1569–1572. https://doi.org/10.1109/tnn.2003.820440
CR  - Ju, X., Fang, B., Yan, R., Xu, X., & Tang, H. (2020). An FPGA Implementation of Deep Spiking Neural Networks for Low-Power and Fast Classification. Neural Computation, 32(1), 182–204. https://doi.org/10.1162/neco_a_01245
CR  - Lammie, C., Hamilton, T., & Azghadi, M. R. (2018). Unsupervised Character Recognition with a Simplified FPGA Neuromorphic System. 2018 IEEE International Symposium on Circuits and Systems (ISCAS). https://doi.org/10.1109/iscas.2018.8351532
CR  - LeCun, Y., Cortes, C., & Burges, C. (2012). MNIST handwritten digit database. Retrieved April 20, 2022, from Lecun.com website: http://yann.lecun.com/exdb/mnist/
CR  - Lemaire, E., Moretti, M., Daniel, L., Miramond, B., Millet, P., Feresin, F., & Bilavarn, S. (2020). An FPGA-Based Hybrid Neural Network Accelerator for Embedded Satellite Image Classification. 2020 IEEE International Symposium on Circuits and Systems (ISCAS). https://doi.org/10.1109/iscas45731.2020.9180625
CR  - Li, S., Zhang, Z., Mao, R., Xiao, J., Chang, L., & Zhou, J. (2021). A Fast and Energy-Efficient SNN Processor With Adaptive Clock/Event-Driven Computation Scheme and Online Learning. IEEE Transactions on Circuits and Systems I: Regular Papers, 68(4), 1543–1552. https://doi.org/10.1109/tcsi.2021.3052885
CR  - Liu, K., Cui, X., Zhong, Y., Kuang, Y., Wang, Y., Tang, H., & Huang, R. (2019). A Hardware Implementation of SNN-Based Spatio-Temporal Memory Model. Frontiers in Neuroscience, 13. https://doi.org/10.3389/fnins.2019.00835
CR  - Losh, M., & Llamocca, D. (2019). A Low-Power Spike-like Neural Network Design. Electronics, 8(12), 1479. https://doi.org/10.3390/electronics8121479
CR  - Maass, W. (1997). Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10(9), 1659–1671. https://doi.org/10.1016/s0893-6080(97)00011-7
CR  - Markram, H., Gerstner, W., & Sjöström, P. J. (2012). Spike-Timing-Dependent Plasticity: A Comprehensive Overview. Frontiers in Synaptic Neuroscience, 4. https://doi.org/10.3389/fnsyn.2012.00002
CR  - Miró-Amarante, L., Gómez-Rodríguez, F., Jiménez-Fernández, A., & Jiménez-Moreno, G. (2017). A spiking neural network for real-time Spanish vowel phonemes recognition. Neurocomputing, 226, 249–261. https://doi.org/10.1016/j.neucom.2016.12.005
CR  - Mostafa, H., Pedroni, B. U., Sheik, S., & Cauwenberghs, G. (2017). Fast classification using sparsely active spiking networks. 2017 IEEE International Symposium on Circuits and Systems (ISCAS). https://doi.org/10.1109/iscas.2017.8050527
CR  - Murali, S., Kumar, J., Kumar, J., & Bhakthavatchalu, R. (2016). Design and implementation of Izhikevich spiking neuron model on FPGA. 2016 IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT). https://doi.org/10.1109/rteict.2016.7807968
CR  - Neil, D., & Liu, S.-C. (2014). Minitaur, an Event-Driven FPGA-Based Spiking Network Accelerator. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 22(12), 2621–2628. https://doi.org/10.1109/tvlsi.2013.2294916
CR  - Ou, Q.-F., Xiong, B.-S., Yu, L., Wen, J., Wang, L., & Tong, Y. (2020). In-Memory Logic Operations and Neuromorphic Computing in Non-Volatile Random Access Memory. Materials, 13(16), 3532. https://doi.org/10.3390/ma13163532
CR  - Rice, K. L., Bhuiyan, M. A., Taha, T. M., Vutsinas, C. N., & Smith, M. C. (2009). FPGA Implementation of Izhikevich Spiking Neural Networks for Character Recognition. 2009 International Conference on Reconfigurable Computing and FPGAs. https://doi.org/10.1109/reconfig.2009.77
CR  - Schuman, C. D., Potok, T. E., Patton, R. M., Douglas, B. J., Dean, M. E., Rose, G. S., & Plank, J. S. (2017). A Survey of Neuromorphic Computing and Neural Networks in Hardware. ArXiv.org. https://doi.org/10.48550/arXiv.1705.06963
CR  - Si, J., & Harris, S. L. (2018). Handwritten digit recognition system on an FPGA. 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC). https://doi.org/10.1109/ccwc.2018.8301757
CR  - Sun, B., Guo, T., Zhou, G., Ranjan, S., Jiao, Y., Wei, L., … Wu, Y. A. (2021). Synaptic devices based neuromorphic computing applications in artificial intelligence. Materials Today Physics, 18, 100393. https://doi.org/10.1016/j.mtphys.2021.100393
CR  - Sun, Q. Y., Wu, Q. X., Wang, X., & Hou, L. (2017). A spiking neural network for extraction of features in colour opponent visual pathways and FPGA implementation. Neurocomputing, 228, 119–132. https://doi.org/10.1016/j.neucom.2016.09.093
CR  - T, P., A, A. B. R., & J, A. (2014). FPGA IMPLEMENTATION OF ADAPTIVE INTEGRATED SPIKING NEURAL NETWORK FOR EFFICIENT IMAGE RECOGNITION SYSTEM. ICTACT Journal on Image and Video Processing, 4(4), 848–852. https://doi.org/10.21917/ijivp.2014.0122
CR  - Tang, H., Cho, D., Lew, D., Kim, T., & Park, J. (2020). Rank order coding based spiking convolutional neural network architecture with energy-efficient membrane voltage updates. Neurocomputing, 407, 300–312. https://doi.org/10.1016/j.neucom.2020.05.031
CR  - Von Neumann, J. (1993). First draft of a report on the EDVAC. IEEE Annals of the History of Computing, 15(4), 27–75. https://doi.org/10.1109/85.238389
CR  - Wang, Q., Li, Y., Shao, B., Dey, S., & Li, P. (2017). Energy efficient parallel neuromorphic architectures with approximate arithmetic on FPGA. Neurocomputing, 221, 146–158. https://doi.org/10.1016/j.neucom.2016.09.071
CR  - Wang, R., Cohen, G., Stiefel, K. M., Hamilton, T. J., Tapson, J., & van Schaik, A. (2013). An FPGA Implementation of a Polychronous Spiking Neural Network with Delay Adaptation. Frontiers in Neuroscience, 7. https://doi.org/10.3389/fnins.2013.00014
CR  - Yan, Y., Kappel, D., Neumarker, F., Partzsch, J., Vogginger, B., Hoppner, S., … Mayr, C. (2019). Efficient Reward-Based Structural Plasticity on a SpiNNaker 2 Prototype. IEEE Transactions on Biomedical Circuits and Systems, 13(3), 579–591. https://doi.org/10.1109/tbcas.2019.2906401
CR  - Zhang, G., Li, B., Wu, J., Wang, R., Lan, Y., Sun, L., … Chen, Y. (2020). A low-cost and high-speed hardware implementation of spiking neural network. Neurocomputing, 382, 106–115. https://doi.org/10.1016/j.neucom.2019.11.045
CR  - Zhang, J., Wu, H., Wei, J., Wei, S., & Chen, H. (2019). An Asynchronous Reconfigurable SNN Accelerator With Event-Driven Time Step Update. 2019 IEEE Asian Solid-State Circuits Conference (A-SSCC). https://doi.org/10.1109/a-sscc47793.2019.9056903
CR  - Zhao, F., Zeng, Y., & Xu, B. (2018). A Brain-Inspired Decision-Making Spiking Neural Network and Its Application in Unmanned Aerial Vehicle. Frontiers in Neurorobotics, 12. https://doi.org/10.3389/fnbot.2018.00056
UR  - http://dergipark.org.tr/tr/pub/dae/issue//1107967
L1  - http://dergipark.org.tr/tr/download/article-file/2392562
ER  -