Zynq FPGA-Based Acceleration of Kernelized Correlation Filters via High-Level Synthesis of a Custom DFT Block

Abstract

This study presents a hardware-software co-design implementation of an accelerator for the Kernelized Correlation Filter (KCF) tracking algorithm. Leveraging the High-level synthesis (HLS) and the Zynq heterogeneous platform, the KCF algorithm’s performance is enhanced by using a custom hardware implementation for the computationally intensive Discrete Fourier Transform (DFT) operation. Within this framework, a custom combined DFT and inverse DFT IP, named CDFT, is developed and optimized on the Programmable Logic (PL) side of the Xilinx ZCU102 FPGA, whereas the rest of the KCF algorithm is run with customized Petalinux build on the (Processing System) side. To assess real-world performance, a driver for the CDFT IP and a user application were created to measure metrics like Center Location Error (CLE), Intersection over Union (IoU), and Frame per Second (FPS). The designed DFT accelerator achieves a remarkable speedup of 21x compared to a software DFT implementation. At the algorithm level, the KCF accelerator obtains a 6x speed up with negligible precision loss. In comparison to prior studies employing exclusively hardware implementations, the proposed approach demonstrates a high accuracy at a moderate speed, while there exists potential for further optimizations to enhance its performance even further.

Keywords

• Tracking
• FPGA
• KCF
• HLS
• DFT

References

1. [1] J.F. Henriques, R. Caseiro, P. Martins, et al., "High-speed tracking with kernelized correlation filters", IEEE Transactions on Pattern Analysis and Machine Intelligence, 37,(3), pp. 583-596, 2015.
2. [2] M. Mueller, N. Smith, B. Ghanem, "A benchmark and a simulator for UAV tracking" . European Conf. Computer Vision, pp. 445-461, Amsterdam, Netherlands, October 2016.
3. [3] H. Yang, J. Yu, S. Wang, X. Peng, et al. "Design of airborne target tracking accelerator based on KCF". J. Eng., 2019, Vol. 2019 Iss. 23, pp. 8966-8971. IET Journals doi: 10.1049/joe.2018.9159, 2019
4. [4] X. Liu, Z. Ma, M. Xie, J. Zhang, T. Feng, et al., "Design and implementation of Scale Adaptive Kernel Correlation Filtering Algorithm Based on HLS", IEEE ICSPCC doi:10.1109/ICSPCC52875.2021.9564815, 2021
5. [5] P. Cong, M. Xie, K. Yang, X. Zhang, H. Su and X. Fu, "Design and implementation of multi-feature fusion kernel correlation filtering algorithm based on HLS," IET International Radar Conference (IET IRC 2020), Online Conference, 2020, pp. 645-649, doi: 10.1049/icp.2021.0761.
6. [6] J. Faro, "C++ KCF Tracker" 2021. [Online]. Available: https://github.com/joaofaro/KCFcpp ,[Accessed: 30.08.2023]
7. [7] G.M. Amdahl, "Validity of the Single Processor Approach to Achieving Large-Scale Computing Capabilities" . AFIPS Conference Proceedings (30): pp. 483–485, doi:10.1145/1465482.1465560, 1967
8. [8] M. Kristan, J. Matas, A. Leonardis, and et al., "The visual object tracking vot2014 challenge results.", ECCV Workshop, 2014

9. [9] S.G. Raju, “Accessing BRAM in LINUX” 2021. [Online]. Available: https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842412/Accessing+BRAM+In+Linux , [Accessed: 01.04.2023]
10. [10] Hassan, J., Bilal, M., & Masud, S. (2020). A Hardware-Software Co-Design Framework for Real-Time Video Stabilization. J. Circuits Syst. Comput., 29, 2050027:1-2050027:18. https://doi.org/10.1142/S0218126620500279.
11. [11] Chien, C., Chien, C., & Hsu, C. (2019). Hardware-Software Co-Design of an Image Feature Extraction and Matching Algorithm. 2019 2nd International Conference on Intelligent Autonomous Systems (ICoIAS), 37-41. https://doi.org/10.1109/ICoIAS.2019.00013.
12. [12] Dang, T., & Hoshino, Y. (2019). Hardware/Software Co-design for a Neural Network Trained by Particle Swarm Optimization Algorithm. Neural Processing Letters, 49, 481-505. https://doi.org/10.1007/s11063-018-9826-4.
13. [13] Gudis, E., Lu, P., Berends, D., Kaighn, K., Wal, G., Buchanan, G., Chai, S., & Piacentino, M. (2013). An Embedded Vision Services Framework for Heterogeneous Accelerators. 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 598-603. https://doi.org/10.1109/CVPRW.2013.90.
14. [14] Ruta, A., Brzoza-Woch, R., & Zielinski, K. (2012). On fast development of FPGA-based SOA services—machine vision case study. Design Automation for Embedded Systems, 16, 45-69. https://doi.org/10.1007/s10617-012-9084-z.
15. [15] Pan, Y., Zhu, M., Luo, J., & Qiu, Y. (2018). A Hardware/Software Co-design Approach for Real-Time Binocular Stereo Vision Based on ZYNQ (Short Paper)., 719-733. https://doi.org/10.1007/978-3-030-12981-1_50.