VR-Assisted Comparative Analysis of HEVC Intra Prediction Modes Using Deep Learning and Rule-Based Systems

Yıl 2025, Cilt: 8 Sayı: 6, 1957 - 1966, 15.11.2025

Mücahit Kaplan

https://doi.org/10.34248/bsengineering.1706643

Öz

This paper proposes an artificial intelligence-based solution to classify the three basic intrinsic prediction modes (Planar, DC and Angular) defined in the High Efficiency Video Coding (HEVC) standard. A convolutional neural network (CNN) based deep learning model trained with 32×32 blocks obtained from 30+ classical gray level test images is developed. As a result of the training, the model demonstrated a successful classification performance with an overall accuracy of over 89% and a macro F1 score of approximately 88%. The model was converted into ONNX format and integrated into a Unity-based virtual reality (VR) environment, thus creating an interactive analysis platform where users can observe the predictions of both artificial intelligence and rule-based systems at the block level comparatively. In this environment, users can also examine the reasoning of the predictions. The proposed system provides a holistic solution in terms of classification performance, interpretability and user experience, and makes innovative contributions to the analysis and visualization of video coding processes for educational purposes.

Anahtar Kelimeler

HEVC , Intra prediction , Convolutional neural network , Rule-based expert system , VR visulation , Video coding

Etik Beyan

Ethics committee approval was not required for this study because there was no study on animals or humans.

Kaynakça

• Adeniji I, Casarona M, Bielory L, Bancairen L, Menzel M, Perigo N, Blackmon C, Niepielko MG, Insley J, Joiner D. 2024. Using Unity for Scientific Visualization as a Coursebased Undergraduate Research Experience. J Comput Sci Educ, 15(1): 35-40. https://doi.org/10.22369/issn.2153-4136/15/1/7
• Askin MB, Celikcan U. 2022. Learning based versus heuristic based: A comparative analysis of visual saliency prediction in immersive virtual reality. Comput Animat Virtual Worlds, 33(6): e2106. https://doi.org/10.1002/cav.2106
• Cucchiara R, Piccardi M, Mello P. 2000. Image Analysis and rule-based reasoning for a traffic monitoring system. IEEE Transact Intel Transport Syst, 1(2): 119-130.
• Cui W, Zhang T, Zhang S, Jiang F, Zuo W, Zhao D. 2018. Convolutional neural networks based intra prediction for HEVC. http://arxiv.org/abs/1808.05734
• Feng A, Gao C, Li L, Liu D, Wu F. 2021. Cnn-based depth map prediction for fast block partitioning in hevc intra coding. IEEE International Conference on Multimedia and Expo (ICME), Jult 05-09, Shenzhen, China, pp: 1-6. https://doi.org/10.1109/ICME51207.2021.9428069
• Galpin F, Racape F, Jaiswal S, Bordes P, Leannec F, Francois E. 2019. CNN-based driving of block partitioning for intra slices encoding. Data Compression Conference, March 26-29, Snowbird, UT, US, pp: 162-171. https://doi.org/10.1109/DCC.2019.00024
• Imen W, Amna M, Fatma B, Ezahra SF, Masmoudi N. 2022. Fast HEVC intra-CU decision partition algorithm with modified LeNet-5 and AlexNet. Signal Image Video Proces, 16(7): 1811-1819. https://doi.org/10.1007/s11760-022-02139-w
• Ishikawa S, Todo M, Taki M, Uchiyama Y, Matsunaga K, Lin P, Ogihara T, Yasui M. 2023. Example-based explainable AI and its application for remote sensing image classification. Int J Appl Earth Obser Geoinfo, 118: 103215. https://doi.org/10.1016/j.jag.2023.103215
• Islam RU, Hossain MS, Andersson K. 2020. A deep learning inspired belief rule-based expert system. IEEE Access, 8: 190637-190651. https://doi.org/10.1109/ACCESS.2020.3031438
• Kaplan M, Akman A. 2024. Parallel implementation of discrete cosine transform (DCT) methods on GPU for HEVC, 2nd International Congress of Electrical and Computer Engineering. pp. 281–293. https://doi.org/10.1007/978-3-031-52760-9_20
• Kuanar S, Rao KR, Bilas M, Bredow J. 2019. Adaptive CU mode selection in HEVC intra prediction: A deep learning approach. Circuits Syst Signal Proces, 38(11): 5081-5102. https://doi.org/10.1007/s00034-019-01110-4
• Laude T, Ostermann J. 2016. Deep learning-based intra prediction mode decision for HEVC. 2016 Picture Coding Symposium (PCS), December 04-07, Nuremberg, Germany, pp: 1-5. https://doi.org/10.1109/PCS.2016.7906399
• Li N, Wang Z, Zhang Q, He L, Zhang W. 2025. Fast intra-prediction mode decision algorithm for versatile video coding based on gradient and convolutional neural network. Electronics, 14(10): 2031. https://doi.org/10.3390/electronics14102031
• Liu X, Li Y, Liu D, Wang P, Yang LT. 2019. An adaptive CU size decision algorithm for HEVC intra prediction based on complexity classification using machine learning. IEEE Transact Circuits Syst Video Technol, 29(1): 144-155. https://doi.org/10.1109/TCSVT.2017.2777903
• Moran A, Gadepally V, Hubbell M, Kepner J. 2015. Improving big data visual analytics with interactive virtual reality. 2015 IEEE High Performance Extreme Computing Conference (HPEC), September 15-17, Waltham, MA, US, pp: 1-6. https://doi.org/10.1109/HPEC.2015.7322473
• Pattimi H, Srinivasarao BKN. 2024. High-speed coding unit depth identifications using CU-VGG deep learning architectures. Arabian J Sci Eng, 49: 16287-16298. https://doi.org/10.1007/s13369-024-08928-4
• Ren W, Su J, Sun C, Shi Z. 2019. An IBP-CNN Based fast block partition for intra prediction. 2019 Picture Coding Symposium (PCS), November 12-15, Ningbo, China, pp: 1-5. https://doi.org/10.1109/PCS48520.2019.8954522
• Schorr C, Goodarzi P, Chen F, Dahmen T. 2021. Neuroscope: An explainable AI toolbox for semantic segmentation and image classification of convolutional neural nets. Appl Sci, 11(5): 2199. https://doi.org/10.3390/app11052199
• Sullivan GJ, Ohm JR, Han WJ, Wiegand T. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Transact Circuits Syst Video Technol, 22(12): 1649-1668. https://doi.org/10.1109/TCSVT.2012.2221191
• Swamy TN, Ramesha K. 2022. Improved intra prediction algorithm for HEVC with conventional and convolutional neural network approach. Math Stat Eng Appl, 71(2): 272-280.
• Zaki F, Mohamed AE, Sayed SG. 2021. CtuNet: A deep learning-based Framework for Fast CTU Partitioning of H265/HEVC Intra- coding. Ain Shams Eng J, 12(2): 1859-1866. https://doi.org/10.1016/j.asej.2021.01.001
• Zhao L, Zhang L, Ma S, Zhao D. 2011. Fast mode decision algorithm for intra prediction in HEVC. 2011 Visual Communications and Image Processing (VCIP), November 06-09, Tainan, Taiwan, pp: 1-4. https://doi.org/10.1109/VCIP.2011.6115979
• Zisad SN, Chowdhury E, Hossain MS, Islam RU, Andersson K. 2021. An integrated deep learning and belief rule-based expert system for visual sentiment analysis under uncertainty. Algorithms, 14(7): 213. https://doi.org/10.3390/a14070213