Full reference video quality evaluation using foveated vision and multiple fixation points

Abstract

In video applications it is necessary to continuously measure the video quality perceived by the end-user. Thus it is desirable to know which parts of video frame, i.e. which contents, attract viewers’ attention. If this information is known, then it is possible to estimate perceived video quality in a meaningful way. However, automatic detection of viewers’ fixation points is time-consuming process and often is omitted in objective video quality assessment (VQA) metrics. Based on our previous work, in which we proposed Foveation-based content Adaptive Root Mean Squared Error (FARMSE) VQA metric, in this work we propose two new full-reference (FR) VQA metrics called Multi-Point FARMSE (MP-FARMSE), and Simple-FARMSE (S-FARMSE). Both new-proposed metrics are based on foveated-vision features of human visual system and spatio-temporal features of video signal. In MP-FARMSE, by using an engineering approach, we implemented the fact that viewer’s attention can be directed out of the center of the frame, thus covering use-cases when objects of interest are not located in the center of the frame. The main idea when creating the S-FARMSE metric was to reduce the computational complexity of the final algorithm and to make S-FARMSE metric capable of processing high-resolution video signals in real-time. Performances of the new-proposed metrics are compared to performances of seven existing VQA metrics on two different video quality databases. The results show that performances achieved by MP-FARMSE and S-FARMSE are quite close to those of state-of-the-art VQA metrics, whereas at the same time their computational complexity level is significantly lower.

Keywords

• foveated vision,video quality,full reference

References

1. Glavota, Ivan; Kaprocki, Zvonimir; Vranješ, Mario; Herceg, Marijan., No-Reference Real-Time Video Transmission Artifacts Detection for Video Signals. // Journal of Real-Time Image Processing. 1 (2018) , 1; 1-22
2. Križanović, Višnja; Žagar, Drago; Grgić, Krešimir; Vranješ, Mario., Enhanced predictive modelling process of broadband services adoption based on time series data. // Advanced engineering informatics. 38 (2018) , October 2018; 142-167
3. Pul, Matija; Peković, Vukota; Vranješ, Mario; Grbić, Ratko., Automatic Functionality Verification of Hybrid Set-Top Boxes with Dynamic User Interface. // Ieee transactions on consumer electronics. 64 (2018) , 4; 1-9
4. Vranješ, Mario; Rimac-Drlje, Snježana; Vranješ, Denis., Foveation-based Content Adaptive Root Mean Squared Error for Video Quality Assessment. // Multimedia tools and applications. 77 (2018) , 16; 21053-21082