TY  - JOUR
T1  - VR-Assisted Comparative Analysis of HEVC Intra Prediction Modes Using Deep Learning and Rule-Based Systems
TT  - VR-Assisted Comparative Analysis of HEVC Intra Prediction Modes Using Deep Learning and Rule-Based Systems
AU  - Kaplan, Mücahit
PY  - 2025
DA  - November
Y2  - 2025
DO  - 10.34248/bsengineering.1706643
JF  - Black Sea Journal of Engineering and Science
JO  - BSJ Eng. Sci.
PB  - Karyay Karadeniz Yayımcılık Ve Organizasyon Ticaret Limited Şirketi
WT  - DergiPark
SN  - 2619-8991
SP  - 1957
EP  - 1966
VL  - 8
IS  - 6
LA  - en
AB  - This paper proposes an artificial intelligence-based solution to classify the three basic intrinsic prediction modes (Planar, DC and Angular) defined in the High Efficiency Video Coding (HEVC) standard. A convolutional neural network (CNN) based deep learning model trained with 32×32 blocks obtained from 30+ classical gray level test images is developed. As a result of the training, the model demonstrated a successful classification performance with an overall accuracy of over 89% and a macro F1 score of approximately 88%. The model was converted into ONNX format and integrated into a Unity-based virtual reality (VR) environment, thus creating an interactive analysis platform where users can observe the predictions of both artificial intelligence and rule-based systems at the block level comparatively. In this environment, users can also examine the reasoning of the predictions. The proposed system provides a holistic solution in terms of classification performance, interpretability and user experience, and makes innovative contributions to the analysis and visualization of video coding processes for educational purposes.
KW  - HEVC
KW  - Intra prediction
KW  - Convolutional neural network
KW  - Rule-based expert system
KW  - VR visulation
KW  - Video coding
N2  - This paper proposes an artificial intelligence-based solution to classify the three basic intrinsic prediction modes (Planar, DC and Angular) defined in the High Efficiency Video Coding (HEVC) standard. A convolutional neural network (CNN) based deep learning model trained with 32×32 blocks obtained from 30+ classical gray level test images is developed. As a result of the training, the model demonstrated a successful classification performance with an overall accuracy of over 89% and a macro F1 score of approximately 88%. The model was converted into ONNX format and integrated into a Unity-based virtual reality (VR) environment, thus creating an interactive analysis platform where users can observe the predictions of both artificial intelligence and rule-based systems at the block level comparatively. In this environment, users can also examine the reasoning of the predictions. The proposed system provides a holistic solution in terms of classification performance, interpretability and user experience, and makes innovative contributions to the analysis and visualization of video coding processes for educational purposes.
CR  - Adeniji I, Casarona M, Bielory L, Bancairen L, Menzel M, Perigo N, Blackmon C, Niepielko MG, Insley J, Joiner D. 2024. Using Unity for Scientific Visualization as a Coursebased Undergraduate Research Experience. J Comput Sci Educ, 15(1): 35-40. https://doi.org/10.22369/issn.2153-4136/15/1/7
CR  - Askin MB, Celikcan U.  2022. Learning based versus heuristic based: A comparative analysis of visual saliency prediction in immersive virtual reality. Comput Animat Virtual Worlds, 33(6): e2106. https://doi.org/10.1002/cav.2106
CR  - Cucchiara R, Piccardi M, Mello P.  2000. Image Analysis and rule-based reasoning for a traffic monitoring system. IEEE Transact Intel Transport Syst, 1(2): 119-130.
CR  - Cui W, Zhang T, Zhang S, Jiang F, Zuo W, Zhao D. 2018. Convolutional neural networks based intra prediction for HEVC. http://arxiv.org/abs/1808.05734
CR  - Feng A, Gao C, Li L, Liu D, Wu F.  2021. Cnn-based depth map prediction for fast block partitioning in hevc intra coding. IEEE International Conference on Multimedia and Expo (ICME), Jult 05-09, Shenzhen, China, pp: 1-6. https://doi.org/10.1109/ICME51207.2021.9428069
CR  - Galpin F, Racape F, Jaiswal S, Bordes P, Leannec F, Francois E.  2019. CNN-based driving of block partitioning for intra slices encoding. Data Compression Conference, March 26-29, Snowbird, UT, US, pp: 162-171. https://doi.org/10.1109/DCC.2019.00024
CR  - Imen W, Amna M, Fatma B, Ezahra SF, Masmoudi N.  2022. Fast HEVC intra-CU decision partition algorithm with modified LeNet-5 and AlexNet. Signal Image Video Proces, 16(7): 1811-1819. https://doi.org/10.1007/s11760-022-02139-w
CR  - Ishikawa S, Todo M, Taki M, Uchiyama Y, Matsunaga K, Lin P, Ogihara T, Yasui M.  2023. Example-based explainable AI and its application for remote sensing image classification. Int J Appl Earth Obser Geoinfo, 118: 103215. https://doi.org/10.1016/j.jag.2023.103215
CR  - Islam RU, Hossain MS, Andersson K.  2020. A deep learning inspired belief rule-based expert system. IEEE Access, 8: 190637-190651. https://doi.org/10.1109/ACCESS.2020.3031438
CR  - Kaplan M, Akman A.  2024. Parallel implementation of discrete cosine transform (DCT) methods on GPU for HEVC, 2nd International Congress of Electrical and Computer Engineering. pp. 281–293. https://doi.org/10.1007/978-3-031-52760-9_20
CR  - Kuanar S, Rao KR, Bilas M, Bredow J.  2019. Adaptive CU mode selection in HEVC intra prediction: A deep learning approach. Circuits Syst Signal Proces, 38(11): 5081-5102. https://doi.org/10.1007/s00034-019-01110-4
CR  - Laude T, Ostermann J.  2016. Deep learning-based intra prediction mode decision for HEVC. 2016 Picture Coding Symposium (PCS), December 04-07, Nuremberg, Germany, pp: 1-5. https://doi.org/10.1109/PCS.2016.7906399
CR  - Li N, Wang Z, Zhang Q, He L, Zhang W. 2025. Fast intra-prediction mode decision algorithm for versatile video coding based on gradient and convolutional neural network. Electronics, 14(10): 2031. https://doi.org/10.3390/electronics14102031
CR  - Liu X, Li Y, Liu D, Wang P, Yang LT.  2019. An adaptive CU size decision algorithm for HEVC intra prediction based on complexity classification using machine learning. IEEE Transact Circuits Syst Video Technol, 29(1): 144-155. https://doi.org/10.1109/TCSVT.2017.2777903
CR  - Moran A, Gadepally V, Hubbell M, Kepner J.  2015. Improving big data visual analytics with interactive virtual reality. 2015 IEEE High Performance Extreme Computing Conference (HPEC), September 15-17, Waltham, MA, US, pp: 1-6. https://doi.org/10.1109/HPEC.2015.7322473
CR  - Pattimi H, Srinivasarao BKN.  2024. High-speed coding unit depth identifications using CU-VGG deep learning architectures. Arabian J Sci Eng, 49: 16287-16298. https://doi.org/10.1007/s13369-024-08928-4
CR  - Ren W, Su J, Sun C, Shi Z.  2019. An IBP-CNN Based fast block partition for intra prediction. 2019 Picture Coding Symposium (PCS), November 12-15, Ningbo, China, pp: 1-5. https://doi.org/10.1109/PCS48520.2019.8954522
CR  - Schorr C, Goodarzi P, Chen F, Dahmen T.  2021. Neuroscope: An explainable AI toolbox for semantic segmentation and image classification of convolutional neural nets. Appl Sci, 11(5): 2199. https://doi.org/10.3390/app11052199
CR  - Sullivan GJ, Ohm JR, Han WJ, Wiegand T. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Transact Circuits Syst Video Technol, 22(12): 1649-1668. https://doi.org/10.1109/TCSVT.2012.2221191
CR  - Swamy TN, Ramesha K. 2022. Improved intra prediction algorithm for HEVC with conventional and convolutional neural network approach. Math Stat Eng Appl, 71(2): 272-280.
CR  - Zaki F, Mohamed AE, Sayed SG.  2021. CtuNet: A deep learning-based Framework for Fast CTU Partitioning of H265/HEVC Intra- coding. Ain Shams Eng J, 12(2): 1859-1866. https://doi.org/10.1016/j.asej.2021.01.001
CR  - Zhao L, Zhang L, Ma S, Zhao D.  2011. Fast mode decision algorithm for intra prediction in HEVC. 2011 Visual Communications and Image Processing (VCIP), November 06-09, Tainan, Taiwan, pp: 1-4. https://doi.org/10.1109/VCIP.2011.6115979
CR  - Zisad SN, Chowdhury E, Hossain MS, Islam RU, Andersson K.  2021. An integrated deep learning and belief rule-based expert system for visual sentiment analysis under uncertainty. Algorithms, 14(7): 213. https://doi.org/10.3390/a14070213
UR  - https://doi.org/10.34248/bsengineering.1706643
L1  - https://dergipark.org.tr/en/download/article-file/4903272
ER  -