A Transformer-Assisted Cooperative Caching Framework with Q-Learning Optimization for Internet of Vehicles

Abstract

The Internet of Vehicles (IoV) is witnessing escalating data traffic that demands sophisticated caching to minimize retrieval latency and sustain Quality of Service. Recent cooperative approaches, notably the Cooperative Caching Strategy with Content Request Prediction (CCCRP) framework grounded in LSTM-based request prediction, struggle to capture the complex, long-range temporal dependencies inherent to dynamic vehicular environments. This paper proposes Transformer-based CCCRP (T-CCCRP), which replaces the LSTM predictor with a Transformer model leveraging self-attention to model long-range dependencies and enable efficient parallelism. The prediction module is integrated with a reinforcement learning controller to optimize cache placement jointly across vehicles and roadside units, thereby aligning predicted popularity with resource constraints. To ensure a realistic and comprehensive evaluation, the proposed framework is assessed under both small-scale and newly introduced large-scale IoV scenarios, involving up to 200 vehicles, multiple caching nodes, and expanded content libraries. The experimental setup further adapts the input sequence length of the prediction model to reflect increased temporal complexity in large-scale environments. Simulation results demonstrate that T-CCCRP consistently outperforms the original CCCRP and conventional caching strategies (LFU and LRU) across all evaluated scales.

Keywords

• Edge caching
• Cooperative caching
• CCCRP
• Transformer model
• Reinforcement learning
• Temporal dependency

Supporting Institution

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Ethical Statement

This study does not involve human participants or animals. Therefore, ethical committee approval was not required.

References

1. H. Peng, L. Liang, X. Shen, and G. Y. Li, “Vehicular communications: A network layer perspective,” *IEEE Trans. Veh. Technol.*, vol. 68, no. 2, pp. 1064–1078, Feb. 2019.
2. S. Wang, X. Zhang, Y. Zhang, L. Wang, J. Yang, and W. Wang, “A survey on mobile edge networks: Convergence of computing, caching and communications,” *IEEE Access*, vol. 5, pp. 6757–6779, 2017.
3. Z. Su, Y. Hui, Q. Xu, T. Yang, J. Liu, and Y. Jia, “An edge caching scheme to distribute content in vehicular networks,” *IEEE Trans. Veh. Technol.*, vol. 67, no. 6, pp. 5346–5356, 2018.
4. L. Yao, A. Chen, J. Deng, J. Wang, and G. Wu, “A cooperative caching scheme based on mobility prediction in vehicular content centric networks,” *IEEE Trans. Veh. Technol.*, vol. 67, no. 6, pp. 5435–5444, Jun. 2018.
5. L. Li, Y. Xu, J. Yin, W. Liang, X. Li, W. Chen, and Z. Han, “Deep reinforcement learning approaches for content caching in cache-enabled D2D networks,” *IEEE Internet Things J.*, vol. 7, no. 1, pp. 544–557, Jan. 2020.
6. M. Chen, W. Saad, C. Yin, and M. Debbah, “Echo state networks for proactive caching in cloud-based radio access networks with mobile users,” *IEEE Trans. Wireless Commun.*, vol. 16, no. 6, pp. 3520–3535, Jun. 2017.
7. A. Vaswani et al., “Attention Is All You Need,” in *Proc. Advances in Neural Information Processing Systems (NeurIPS)*, vol. 30, pp. 5998–6008, Dec. 2017.
8. Z. Zhou, D. Xu, Q. Zhang, and X. Zhang, “State-of-the-art deep learning for multi-source traffic prediction,” *IEEE Trans. Intell. Transp. Syst.*, vol. 22, no. 8, pp. 5197–5212, Aug. 2021.

9. T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Silverman, and A. Y. Wu, “An efficient k-means clustering algorithm: Analysis and implementation,” *IEEE Trans. Pattern Anal. Mach. Intell.*, vol. 24, no. 7, pp. 881–892, Jul. 2002.
10. G. Ma, Z. Wang, M. Zhang, J. Ye, M. Chen, and W. Zhu, “Understanding performance of edge content caching for mobile video streaming,” *IEEE J. Sel. Areas Commun.*, vol. 35, no. 5, pp. 1076–1089, May 2017.
11. L. Yao, X. Xu, J. Deng, G. Wu, and Z. Li, “A cooperative caching scheme for VCCN with mobility prediction and consistent hashing,” *IEEE Trans. Intell. Transp. Syst.*, vol. 23, no. 11, pp. 20230–20242, 2022.
12. Z. Ding, H. Li, H. Bai, Y. Chen, X. Wang, and P. Lu, “A cooperative caching scheme in MIN-V2X,” in *2024 3rd International Joint Conference on Information and Communication Engineering (JCICE)*, 2024, pp. 13–19.
13. R. Wu, G. Tang, T. Chen, D. Guo, L. Luo, and W. Kang, “A profit-aware coalition game for cooperative content caching at the network edge,” *IEEE Internet Things J.*, vol. 9, no. 2, pp. 1361–1373, 2022.
14. S. Fang and P. Fan, “A cooperative caching algorithm for cluster-based vehicular content networks with vehicular caches,” in *2017 IEEE Globecom Workshops (GC Wkshps)*, 2017.
15. Z. Jin, T. Song, W. Jiang, and J. Hu, “A centralized edge cooperative caching strategy for VANETs,” in *2024 IEEE Wireless Communications and Networking Conference (WCNC)*, 2024.
16. L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker, “Web caching and Zipf-like distributions: Evidence and implications,” in *Proc. IEEE INFOCOM*, vol. 1, Mar. 1999, pp. 126–134.
17. C. Ling, W. Zhang, Q. Fan, Z. Feng, J. Wang, R. Yadav, and D. Wang, “Cooperative service caching in vehicular edge computing networks based on transportation correlation analysis,” *IEEE Internet Things J.*, vol. 11, no. 12, pp. 22754–22767, 2024.
18. C.-C. Lin, Y. Chiang, and H.-Y. Wei, “Collaborative edge caching with multiple virtual reality service providers using coalition games,” in *2023 IEEE WCNC*, 2023, pp. 1–6.
19. J. Liang, M. Ma, and G. Yang, “Efficient-lightweight CRL distribution in VANETs: A multilayer coded caching methodology,” in *2023 IEEE WCNC*, 2023, pp. 1–6.
20. S. K. uz Zaman et al., “Cooperative content caching framework using cuckoo search optimization in vehicular edge networks,” *Appl. Sci. (Basel)*, vol. 13, no. 2, p. 780, 2023.
21. T. Z. H. Ernest and A. S. Madhukumar, “Ensemble learning-based edge caching strategies for internet of vehicles: Outage and finite SNR analysis,” *IEEE Open J. Commun. Soc.*, vol. 4, pp. 239–252, 2023.
22. L. Yao, Y. Wang, X. Wang, and G. Wu, “Cooperative caching in vehicular content centric network based on social attributes and mobility,” *IEEE Trans. Mob. Comput.*, vol. 20, no. 2, pp. 391–402, 2021.
23. C. Kan, H. Wu, L. Xing, and H. Ma, “Cooperative caching strategy based mobile vehicle social-aware in internet of vehicles,” *Trans. Emerg. Telecommun. Technol.*, vol. 34, no. 7, 2023.
24. S. Tian, S. Zou, Y. Tan, D. Shen, and Y. Li, “Dynamic content cache strategy based on content prediction in the internet of vehicles,” in *2023 19th International Conference on Mobility, Sensing and Networking (MSN)*, 2023.
25. S. A. Elsayed, S. Abdelhamid, and H. S. Hassanein, “Predictive proactive caching in VANETs for social networking,” *IEEE Trans. Veh. Technol.*, vol. 71, no. 5, pp. 5298–5313, 2022.
26. L. Hou, L. Lei, K. Zheng, and X. Wang, “A Q-learning-based proactive caching strategy for non-safety related services in vehicular networks,” *IEEE Internet Things J.*, vol. 6, no. 3, pp. 4512–4520, 2019.
27. R. Wang, Z. Kan, Y. Cui, D. Wu, and Y. Zhen, “Cooperative caching strategy with content request prediction in internet of vehicles,” *IEEE Internet Things J.*, vol. 8, no. 11, pp. 8964–8975, 2021.
28. S. Bai, J. Z. Kolter, and V. Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” *IEEE Trans. Neural Netw. Learn. Syst.*, vol. 30, no. 9, pp. 3122–3131, Sep. 2019.
29. J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” *arXiv preprint arXiv:1412.3555*, 2014.
30. G. Barlacchi et al., “A multi-source dataset of urban life in the city of Milan and the Province of Trentino,” *Sci. Data*, vol. 2, Oct. 2015, Art. no. 150055.