Image quality assessment based on manifold distortion

Abstract

An image quality metric is proposed by introducing a new framework for full reference image quality assessment from the perspective of image patch manifolds. Assuming that most natural scenes are sampled from low dimensional manifolds or submanifolds, perceived image degradations in structural variations can be quantitatively evaluated on the surfaces of highly nonlinear image manifolds. Manifold distortion image quality index first characterizes intrinsic geometric properties of the locally linear manifold structures of spatially local patch spaces, and then measures the deviation from the original smooth manifold structure to calculate the distortion index. Experimental results demonstrate a strong promise with a comparison to both subjective evaluation and state-of-the-art objective quality assessment methods

Keywords

• Image quality assessment
• Image quality index
• Manifold learning
• Neighbor embedding
• Neighbor embedding

Manifold bozulması ile imge kalitesi değerlendirme

Öz

Görüntü parçacık manifoldları perspektifinden, yeni bir tam referans görüntü kalitesi değerlendirmesi çerçevesi oluşturularak bir görüntü kalitesi metriği önerilmektedir. Çoğu doğal sahnenin düşük boyutlu manifoldlardan veya alt-manifoldlardan örneklendiği varsayılarak, yapısal varyasyonlarda algılanan görüntü bozulmaları yüksek derecede doğrusal olmayan görüntü manifoldlarının yüzeylerinde nicel olarak değerlendirilebilir. Manifold bozulması görüntü kalite endeksi önce uzamsal olarak yerel parçacık uzaylarının yerel doğrusal manifold yapılarının içsel geometrik özelliklerini karakterize etmekte ve daha sonra bozulma endeksini hesaplamak için orijinal pürüzsüz manifold yapısından sapmayı ölçmektedir. Deneysel sonuçlar hem öznel değerlendirme hem de gelişmiş objektif kalite değerlendirme yöntemleriyle kıyaslandığında güçlü bir taahhüt göstermektedir

Anahtar Kelimeler

• Görüntü kalite değerlendirmesi
• Görüntü kalite endeksi
• Manifold öğrenmesi
• Komşuluk gömülmesi

Kaynakça

1. [1] Wang Z, Bovik AC. “Mean squared error: Love it or leave it? A new look at signal fidelity measures”. IEEE Signal Processing Magazine, 26(1), 98-117, 2009.
2. [2] Mannos J, Sakrison D. “The effects of a visual fidelity criterion of the encoding of images”. IEEE Transactions on Information Theory, 20(4), 525-536, 1974.
3. [3] Mitsa T, Varkur KL. “Evaluation of contrast sensitivity functions for the formulation of quality measures incorporated in halftoning algorithms”. IEEE 1993 International Conference on Acoustics, Speech, and Signal Processing, Minneapolis, MN, USA, 27-30 April 1993.
4. [4] Damera-Venkata N, Kite TD, Geisler WS, Evans BL, Bovik AC. “Image quality assessment based on a degradation model”. IEEE Transactions on Image Processing, 9(4), 636-650, 2000.
5. [5] Wang Z, Bovik AC. “A universal image quality index”. IEEE Signal Processing Letters, 9(3), 81-84, 2002.
6. [6] Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. “Image quality assessment: From error visibility to structural similarity”. IEEE Transactions on Image Processing, 13(4), 600-612, 2004.
7. [7] Wang Z, Simoncelli EP, Bovik AC. “Multiscale structural similarity for image quality assessment”. 2003 Asilomar Conference on Signals, Systems & Computers, Pacific Grove, CA, USA, 9-12 November 2003.
8. [8] Wang Z, Li Q. “Information content weighting for perceptual image quality assessment”. IEEE Transactions on Image Processing, 20(5), 1185-1198, 2011.

9. [9] Larson EC, Chandler DM. “Most apparent distortion: Fullreference image quality assessment and the role of strategy”. Journal of Electronic Imaging, 2010. https://doi.org/10.1117/1.3267105.
10. [10] Liu A, Lin W, Narwaria M. “Image quality assessment based on gradient similarity”. IEEE Transactions on Image Processing, 21(4), 1500-1512, 2012.
11. [11] Sheikh HR, Bovik AC, de Veciana G. “An information fidelity criterion for image quality assessment using natural scene statistics”. IEEE Transactions on Image Processing, 14(12), 2117-2128, 2005.
12. [12] Sheikh HR, Bovik AC. “Image information and visual quality”. IEEE Transactions on Image Processing, 15(2), 430-444, 2006.
13. [13] Chandler DM, Hemami SS. “VSNR: A wavelet-based visual signal-to-noise ratio for natural images”. IEEE Transactions on Image Processing, 16(9), 2284-2298, 2007.
14. [14] Zhang L, Zhang L, Mou X. “RFSIM: A feature based image quality assessment metric using Riesz transforms”. IEEE 2010 International Conference on Image Processing, Hong Kong, China, 26-29 September 2010.
15. [15] Zhang L, Zhang L, Mou X, Zhang D. “FSIM: A feature similarity index for image quality assessment”. IEEE Transactions on Image Processing, 20(8), 2378-2386, 2011.
16. [16] Wu J, Lin W, Shi G. “Image quality assessment with degradation on spatial structure”. IEEE Signal Processing Letters, 21(4), 437-440, 2014.
17. [17] Wang F, Sun X, Guo Z, Huang Y, Fu K. “An object-distortion based image quality similarity”. IEEE Signal Processing Letters, 22(10), 1534-1537, 2015.
18. [18] Criminisi A, Perez P, Toyama K. “Region filling and object removal by exemplar-based image inpainting”. IEEE Transactions on Image Processing, 13(9), 1200-1212, 2004.
19. [19] Zhang Y, Xiao J, Shah M. “Region completion in a single image”. 2004 Eurographics, Grenoble, France, 30 August-3 September 2004.
20. [20] Sun J, Yuan L, Jia J, Shum HY. “Image completion with structure propagation”. ACM Transactions on Graphics, 24(3), 861-868, 2005.
21. [21] C. Barnes, Shechtman E, Finkelstein A, Goldman DB. “PatchMatch: A randomized correspondence algorithm for structural image editing”. ACM Transactions on Graphics, 2009. https://doi.org/10.1145/1531326.1531330.
22. [22] Buades A, Coll B, Morel J. “A non-local algorithm for image denoising”. IEEE 2005 Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20-25 June 2005.
23. [23] Mahmoudi M, Sapiro G. “Fast image and video denoising via nonlocal means of similar neighborhoods”. IEEE Signal Processing Letters, 12(12), 839-842, 2005.
24. [24] Dabov K, Foi A, Katkovnik V, Egiazarian K. “Image denoising with block-matching and 3D filtering”. SPIE 2006 Electronic Imaging, San Jose, CA, USA, 17 February 2006.
25. [25] Freeman WT, Jones TR, Pasztor EC. “Example-based super-resolution”. IEEE Computer Graphics and Applications, 22(2), 56-65, 2002.
26. [26] Glasner D, Bagon S, Irani M. “Super-resolution from a single image”. IEEE 2009 International Conference on Computer Vision, Kyoto, Japan, 29 September-2 October 2009.
27. [27] Freedman G, Fattal R. “Image and video upscaling from local self-examples”. ACM Transactions on Graphics, 30(2), 2011. https://doi.org/10.1145/1944846.1944852.
28. [28] Michaeli T, Irani M. “Nonparametric blind superresolution”. IEEE 2013 International Conference on Computer Vision, Sydney, NSW, Australia, 1-8 December 2013.
29. [29] Sugimoto K, Kobayashi M, Suzuki Y, Kato S, Boon CS. “Inter frame coding with template matching spatio-temporal prediction”. IEEE 2004 International Conference on Image Processing, Singapore, Republic of Singapore, 24-27 October 2004.
30. [30] Yang J, Yin B, Sun Y, Zhang N. “A block-matching based intra frame prediction for H.264/AVC”. IEEE 2006 International Conference on Multimedia and Expo, Toronto, Ontario, Canada, 9-12 July 2006.
31. [31] Tan TK, Boon CS, Suzuki Y. “Intra prediction by template matching”. IEEE 2006 International Conference on Image Processing, Atlanta, GA, USA, 8-11 October 2006.
32. [32] Tan TK, Boon CS, Suzuki Y. “Intra prediction by averaged template matching predictors”. IEEE 2007 Consumer Communications and Networking Conference, Las Vegas, NV, USA, 11-13 January 2007.
33. [33] Turkan M, Guillemot C. “Sparse approximation with adaptive dictionary for image prediction”. IEEE 2009 International Conference on Image Processing, Cairo, Egypt, 7-10 November 2009.
34. [34] Turkan M, Guillemot C. “Image prediction: Template matching vs. sparse approximation”. IEEE 2010 International Conference on Image Processing, Hong Kong, China, 26-29 September 2010.
35. [35] Turkan M, Guillemot C. “Image prediction based on neighbor-embedding methods”. IEEE Transactions on Image Processing, 21(4), 1885-1898, 2012.
36. [36] Efros AA, Leung TK. “Texture synthesis by non-parametric sampling”. IEEE 1999 International Conference on Computer Vision, Kerkyra, Greece, 20-27 September 1999.
37. [37] Wei LY, Levoy M. “Fast texture synthesis using treestructured vector quantization”. 2000 Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA, 23-28 July 2000.
38. [38] Ashikhmin M. “Synthesizing natural textures”. 2001 Symposium on Interactive 3D Graphics, Chapel Hill, NC, USA, 26-29 March 2001.
39. [39] Besag J. “Spatial interaction and the statistical analysis of lattice systems”. Journal of the Royal Statistical Society Series B, 36(2), 192-236, 1974.
40. [40] Cross GR, Jain AK. “Markov random field texture models”. IEEE Transactions on Pattern Analysis and Machine Intelligence, 5(1), 25-39, 1983.
41. [41] Chang H, Yeung DY, Xiong Y. “Super-resolution through neighbor embedding”. IEEE 2004 Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 27 June-2 July 2004.
42. [42] Turkan M, Thoreau D, Guillotel P. “Self-content superresolution for ultra-HD up-sampling”. 2012 European Conference on Visual Media Production, London, UK, 5-6 December 2012.
43. [43] Turkan M, Thoreau D, Guillotel P. “Optimized neighbor embeddings for single-image super-resolution”. IEEE 2013 International Conference on Image Processing, Melbourne, VIC, Australia, 15-18 September 2013.
44. [44] Turkan M, Thoreau D, Guillotel P. “Iterated neighborembeddings for image super-resolution”. IEEE 2014 International Conference on Image Processing, Paris, France, 27-30 October 2014.
45. [45] Tenenbaum JB, de Silva V, Langford JC. “A global geometric framework for nonlinear dimensionality reduction”. Science, 290(5500), 2319-2323, 2000.
46. [46] Roweis ST, Saul LK. “Nonlinear dimensionality reduction by locally linear embedding”. Science, 290(5500) 2323-2326, 2000.
47. [47] Donoho DL, Grimes C. “Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data”. 2003 Proceedings of the National Academy of Sciences, 100(10), 5591-5596, 2003.
48. [48] Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F, Kuo CCJ. “Image database TID2013: Peculiarities, results and perspectives”. Signal Processing: Image Communication, 30, 57-77, 2015.
49. [49] Sheikh HR, Sabir MF, Bovik AC. “A statistical evaluation of recent full reference image quality assessment algorithms”. IEEE Transactions on Image Processing, 15(11), 3440-3451, 2006.
50. [50] Sheikh HR, Wang Z, Cormack L, Bovik AC. “LIVE Image Quality Assessment Database Release 2”. http://live.ece.utexas.edu/research/quality (03.05.2020).
51. [51] Spearman C. “The proof and measurement of association between two things”. The American Journal of Psychology, 15(1), 72-101, 1904.
52. [52] Kendall MG. “A new measure of rank correlation”. Biometrika, 30(1-2), 81-93, 1938.
53. [53] Yang J, Wright J, Huang TS, Ma Y. “Image super-resolution via sparse representation”. IEEE Transactions on Image Processing, 19(11), 2861-2873, 2010.