A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment

Amine Gonca Toprak; Öykü Berfin Mercan; Mehmet Özdem

doi:10.69560/cujast.1843247

EN TR

A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment

Abstract

Accurately assessing the perceptual similarity of visual data is a critical requirement in many multimedia applications, including video coding, streaming, and image restoration. Traditional pixel-based metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), however, are limited in their ability to capture high-level semantic and contextual distortions perceived by humans, as they primarily focus on low-level structural differences. This limitation often results in inaccurate quality estimates, particularly when evaluating subtle texture losses or compression-induced visual artifacts. To address these challenges, this study introduces a new, fully neural, training-free evaluation metric, Neural Siamese Perceptual Quality (NSPQ). NSPQ measures perceptual similarity by comparing the feature embeddings of reference and distorted images using a pre-trained AlexNet-based Siamese Neural Network and translating the distance between embeddings into a perceptual similarity score. This approach shifts the focus of visual quality assessment from pixel-level differences to high-level feature representations, thereby providing estimates that better align with human visual perception. The proposed method was extensively evaluated on compressed natural video sequences obtained from the Xiph.org dataset as well as on a variety of distortions in the BAPPS dataset. Experimental results demonstrate that NSPQ achieves higher perceptual accuracy than traditional metrics. In the Xiph.org scenario, NSPQ reached an average performance of 99%, substantially surpassing SSIM (96%), normalized PSNR (39%), and VMAF (56%). In the BAPPS dataset, NSPQ obtained an average score of 85% in the p0 scenario, outperforming SSIM (65%). In the p1 scenario, NSPQ more accurately reflected distortion severity with an average score of 38%, remaining below the constant SSIM value of 65%. Overall, the findings indicate that NSPQ can serve not only as a complementary quality measure but also as a primary evaluation metric in applications such as video encoding, image restoration, and quality control.

Keywords

Image Quality Assessment, PSNR, SSIM, VMAF, AlexNet, Siamese Neural Networks

Supporting Institution

The Scientific and Technological Research Council of Türkiye (TÜBİTAK)

Project Number

5249902

Thanks

This work is supported by The Scientific and Technological Research Council of Türkiye (TÜBİTAK) 1515 Frontier R&D Laboratories Support Program for Türk Telekom 6G R&D Lab under project number 5249902.

Görüntü ve Video Değerlendirmesi için Siyam Sinir Ağı Tabanlı Algısal Kalite Ölçütü

Abstract

Görsel verilerin algısal benzerliğinin doğru bir şekilde değerlendirilmesi; video kodlama, akış (streaming) ve görüntü iyileştirme gibi birçok multimedya uygulaması için kritik bir gereksinimdir. Ancak, Tepe Sinyal-Gürültü Oranı (PSNR) ve Yapısal Benzerlik İndeksi (SSIM) gibi geleneksel piksel tabanlı metrikler, insan görsel sistemi tarafından algılanan yüksek seviyeli anlamsal ve bağlamsal bozulmaları yakalamakta yetersiz kalmaktadır. Bu metrikler düşük seviyeli yapısal farklara odaklandığından, özellikle ince doku kayıplarının veya sıkıştırmaya bağlı görsel artefaktların değerlendirilmesinde hatalı kalite tahminlerine neden olabilmektedir. Bu sınırlamaları aşmak amacıyla bu çalışma, bütünüyle sinirsel, eğitim gerektirmeyen yeni bir değerlendirme metriği olan Neural Siamese Perceptual Quality (NSPQ) yöntemini önermektedir. NSPQ, referans ve bozulmuş görüntülerin özellik gömülerini (feature embeddings) önceden eğitilmiş AlexNet tabanlı bir Siamese Sinir Ağı ile karşılaştırmakta ve bu gömüler arasındaki mesafeyi algısal benzerlik skoruna dönüştürmektedir. Böylece, görsel kalite değerlendirme süreci piksel düzeyindeki farklardan yüksek seviyeli temsil alanlarına kaydırılmakta ve insan görsel algısıyla daha tutarlı sonuçlar elde edilmektedir. Önerilen yöntem, Xiph.org veri setindeki sıkıştırılmış doğal video dizileri ile BAPPS veri setindeki çeşitli bozulma türleri üzerinde kapsamlı bir şekilde değerlendirilmiştir. Deneysel sonuçlar, NSPQ’nun geleneksel metriklere kıyasla daha yüksek algısal doğruluk sağladığını göstermektedir. Xiph.org senaryosunda NSPQ ortalama %99 performansa ulaşmış; bu değer SSIM’in (%96), normalize PSNR’nin (%39) ve VMAF’ın (%56) belirgin şekilde üzerindedir. BAPPS veri setinde NSPQ, p0 senaryosunda ortalama %85 skor ile SSIM’i (%65) geride bırakmıştır. p1 senaryosunda ise NSPQ, bozulma şiddetini daha doğru yansıtarak ortalama %38 skor üretmiş ve sabit kalan SSIM değerinin (%65) altında kalarak gerçekçi bir ayırt edicilik ortaya koymuştur. Genel olarak bulgular, NSPQ’nun yalnızca tamamlayıcı bir kalite ölçütü olarak değil, video kodlama, görüntü restorasyonu ve kalite kontrolü gibi uygulamalarda birincil değerlendirme metriği olarak da etkili bir şekilde kullanılabileceğini göstermektedir.

Keywords

Görüntü Kalite Değerlendirme, PSNR, SSIM, VMAF, AlexNet, Siyam Sinir Ağları

Supporting Institution

Türkiye Bilim ve Teknolojik Araştırma Kurumu (TÜBİTAK)

Project Number

5249902

Thanks

Bu çalışma, Türkiye Bilim ve Teknolojik Araştırma Kurumu (TÜBİTAK) tarafından Türk Telekom 6G Ar-Ge Laboratuvarı için yürütülen 1515 Sınır Ar-Ge Laboratuvarları Destek Programı kapsamında, 5249902 proje numarasıyla desteklenmiştir.

References

Afnan, F. Ullah, Yaseen, J. Lee, S. Jamil, and O.-J. Kwon, ‘‘Subjective assessment of objective image quality metrics range guaranteeing visually lossless compression,’’ Sensors, vol. 23, no. 3, 2023. [Online]. Available: https://www.mdpi.com/1424-8220/23/3/1297
Arabboev, M., S. Begmatov, M. Rikhsivoev, K. Nosirov, and S. Saydiakbarov, ‘‘A comprehensive review of image super-resolution metrics: classical and ai-based approaches,’’ Acta IMEKO, vol. 13, no. 1, pp. 1–8, 2024.
Bampis, C. G., Z. Li, and A. C. Bovik, ‘‘Spatiotemporal feature integration and model fusion for full reference video quality assessment,’’ IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 8, pp. 2256–2270, 2018.
Cong, H., L. Fu, R. Zhang, Y. Zhang, H. Wang, J. He, and J. Gao, ‘‘Image quality assessment with gradient siamese network,’’ in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 1201–1210.
Daş, R., B. Polat, and G. Tuna, ‘‘Derin Öğrenme ile resim ve videolarda nesnelerin tanınması ve takibi,’’ Fırat Üniversitesi Mühendislik Bilimleri Dergisi, vol. 31, no. 2, p. 571–581, 2019.
Demirol, D., Das, R., & Hanbay, D. (2019, September). Büyük veri üzerine perspektif bir bakış. In 2019 International Artificial Intelligence and Data Processing Symposium (IDAP) (pp. 1-9). IEEE.
Deng, S., J. Han, and Y. Xu, ‘‘Vmaf based rate-distortion optimization for video coding,’’ in 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2020, pp. 1–6.
Deshpande, R. G., L. L. Ragha, and S. K. Sharma, ‘‘Video quality assessment through psnr estimation for different compression standards,’’ Indonesian Journal of Electrical Engineering and Computer Science, vol. 11, no. 3, pp. 918–924, 2018.
Ding, K., K. Ma, S. Wang, and E. P. Simoncelli, ‘‘Image quality assessment: Unifying structure and texture similarity,’’ IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 5, pp. 2567–2581, 2020.
Dosovitskiy, A. and T. Brox, ‘‘Generating images with perceptual similarity metrics based on deep networks,’’ in Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, Eds., vol. 29. Curran Associates, Inc., 2016. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2016/file/371bce7dc83817b7893bcdeed13799b5-Paper.pdf

Ghildyal, A. and F. Liu, ‘‘Shift-tolerant perceptual similarity metric,’’ in European Conference on Computer Vision. Springer, 2022, pp. 91–107.
Gu, J., H. Cai, H. Chen, X. Ye, J. Ren, and C. Dong, ‘‘Image quality assessment for perceptual image restoration: A new dataset, benchmark and metric,’’ 2020. [Online]. Available: https://arxiv.org/abs/2011.15002
Guo, L., J. De Cock, and A. Aaron, ‘‘Compression performance comparison of x264, x265, libvpx and aomenc for on-demand adaptive streaming applications,’’ in 2018 Picture Coding Symposium (PCS). IEEE, 2018, pp. 26–30.
Heusel, M., H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, ‘‘Gans trained by a two time-scale update rule converge to a local nash equilibrium,’’ Advances in neural information processing systems, vol. 30, 2017.
Isik, B., O. G. Guleryuz, D. Tang, J. Taylor, and P. A. Chou, ‘‘Sandwiched video compression: Efficiently extending the reach of standard codecs with neural wrappers,’’ in 2023 IEEE International Conference on Image Processing (ICIP). IEEE, 2023, pp. 2055–2059.
Kah-Oberländer, A., M.-D. Boerzel, W. Ruppel, and M. Narroschke, ‘‘Vmaf score for subjectively lossless video quality on tvs, tablets and smartphones,’’ in 2024 4th International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME), 2024, pp. 1–4.
Klink, J. and T. Uhl, ‘‘Video quality assessment: Some remarks on selected objective metrics,’’ in 2020 International Conference on Software, Telecommunications and Computer Networks (SoftCOM). IEEE, 2020, pp. 1–6.
Li, B., Y. Liu, X. Niu, B. Bai, L. Deng, and D. Gündüz, ‘‘Extreme video compression with pre-trained diffusion models,’’ arXiv preprint arXiv:2402.08934, 2024.
Li, Z., A. Aaron, I. Katsavounidis, A. Moorthy, and M. Manohara, ‘‘Toward a practical perceptual video quality metric. netflix tech blog (2016),’’ 2016.
Liao, X., X. Wei, M. Zhou, Z. Li, and S. Kwong, ‘‘Image quality assessment: Measuring perceptual degradation via distribution measures in deep feature spaces,’’ IEEE Transactions on Image Processing, vol. 33, pp. 4044–4059, 2024.
Lin, X., X. Liu, H. Yang, X. He, and H. Chen, ‘‘Perception- and fidelity-aware reduced-reference super-resolution image quality assessment,’’ IEEE Transactions on Broadcasting, vol. 71, no. 1, pp. 323–333, 2025.
Liu, X., J. Van De Weijer, and A. D. Bagdanov, ‘‘Rankiqa: Learning from rankings for no-reference image quality assessment,’’ in Proceedings of the IEEE international conference on computer vision, 2017, pp. 1040–1049.
Martínez-Rach, M. O., P. Piñol, O. M. López, M. Perez Malumbres, J. Oliver, and C. T. Calafate, ‘‘On the performance of video quality assessment metrics under different compression and packet loss scenarios,’’ The Scientific World Journal, vol. 2014, no. 1, p. 743604, 2014.
Martini, M. (2023, September). A simple relationship between SSIM and PSNR for DCT-based compressed images and video: SSIM as content-aware PSNR. In 2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP) (pp. 1-5). IEEE.
Martini, M. G., ‘‘Measuring objective image and video quality: On the relationship between ssim and psnr for dct-based compressed images,’’ IEEE Transactions on Instrumentation and Measurement, 2025.
Min, X., H. Duan, W. Sun, Y. Zhu, and G. Zhai, ‘‘Perceptual video quality assessment: A survey,’’ Science China Information Sciences, vol. 67, no. 11, p. 211301, 2024.
Pham, T. T., X. Van Hoang, N. T. Nguyen, D. T. Dinh, and L. T. Ha, ‘‘End-to-end image patch quality assessment for image/video with compression artifacts,’’ IEEE Access, vol. 8, pp. 215 157–215 172, 2020.
Prashnani, E., H. Cai, Y. Mostofi, and P. Sen, ‘‘Pieapp: Perceptual image-error assessment through pairwise preference,’’ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1808–1817.
Rassool, R., ‘‘Vmaf reproducibility: Validating a perceptual practical video quality metric,’’ in 2017 IEEE international symposium on broadband multimedia systems and broadcasting (BMSB). IEEE, 2017, pp. 1–2.
Rehman, M. U., I. F. Nizami, F. Ullah, and I. Hussain, ‘‘Iqa vision transformed: A survey of transformer architectures in perceptual image quality assessment,’’ IEEE Access, vol. 12, pp. 183 369–183 393, 2024.
Sharma, D. (2024, November). Image Quality Assessment Metrics for Hyperspectral Image Compression Algorithms. In 2024 Second International Conference Computational and Characterization Techniques in Engineering & Sciences (IC3TES) (pp. 1-5). IEEE.
Shen, G., Z. Yan, X. Jin, L. Wu, J. Chen, I. Cho, and C.-H. Hahm, ‘‘Exploring simple siamese network for high-resolution video quality assessment,’’ in ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, pp. 1–5.
Snell, J., K. Ridgeway, R. Liao, B. D. Roads, M. C. Mozer, and R. S. Zemel, ‘‘Learning to generate images with perceptual similarity metrics,’’ in 2017 IEEE International Conference on Image Processing (ICIP), 2017, pp. 4277–4281.
Tang, Z., Z. Chen, Z. Li, B. Zhong, X. Zhang, and X. Zhang, ‘‘Unifying dual-attention and siamese transformer network for full-reference image quality assessment,’’ ACM Transactions on Multimedia Computing, Communications and Applications, vol. 19, no. 6, pp. 1–24, 2023.
Uddin, S., M. Leszczuk, and M. Grega, ‘‘Preliminary study on video codec optimization using vmaf,’’ in Asian Conference on Intelligent Information and Database Systems. Springer, 2022, pp. 469–480.
Unterthiner, T., S. Van Steenkiste, K. Kurach, R. Marinier, M. Michalski, and S. Gelly, ‘‘Fvd: A new metric for video generation,’’ 2019.
Valenzise, G., A. Purica, V. Hulusic, and M. Cagnazzo, ‘‘Quality assessment of deep-learning-based image compression,’’ in 2018 ieee 20th international workshop on multimedia signal processing (mmsp). IEEE, 2018, pp. 1–6.
de Vos, I. M. A., G. L. van den Boogerd, M. D. Fennema, and A. D. Correia, ‘‘Comparing in context: Improving cosine similarity measures with a metric tensor,’’ 2022. [Online]. Available: https://arxiv.org/abs/2203.14996
Vu, T. H., H. P. Cong, T. Sisouvong, X. HoangVan, S. NguyenQuang, and M. DoNgoc, ‘‘Vmaf based quantization parameter prediction model for low resolution video coding,’’ in 2022 International Conference on Advanced Technologies for Communications (ATC). IEEE, 2022, pp. 364–368.
Wang, L., ‘‘A survey on iqa,’’ 2022. [Online]. Available: https://arxiv.org/abs/2109.00347
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612.
Xiph.org video test media: Derf’s collection, https://media.xiph.org/video/derf/, access date: 2025-09-24.
Zhang, R., P. Isola, A. A. Efros, E. Shechtman, and O. Wang, ‘‘The unreasonable effectiveness of deep features as a perceptual metric,’’ in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595.
Zhang, R., P. Isola, A. A. Efros, E. Shechtman, and O. Wang, ‘‘The unreasonable effectiveness of deep features as a perceptual metric,’’ in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
Zhang, Z., W. Sun, X. Min, T. Wang, W. Lu, and G. Zhai, ‘‘A full-reference quality assessment metric for fine-grained compressed images,’’ in 2021 International Conference on Visual Communications and Image Processing (VCIP), 2021, pp. 1–4.
Zujovic, J., T. N. Pappas, and D. L. Neuhoff, ‘‘Perceptual similarity metrics for retrieval of natural textures,’’ in 2009 IEEE International Workshop on Multimedia Signal Processing, 2009, pp. 1–5.

Details

Primary Language

English

Subjects

Quantum Engineering Systems (Incl. Computing and Communications)

Journal Section

Research Article

Authors

Amine Gonca Toprak ^*
0000-0003-2425-5342
Türkiye

Öykü Berfin Mercan
0000-0001-7727-0197
Türkiye

Mehmet Özdem
0000-0002-2901-2342
Türkiye

Publication Date

June 29, 2026

Submission Date

December 16, 2025

Acceptance Date

February 26, 2026

Published in Issue

Year 2026 Volume: 5 Number: 1

DOI

https://doi.org/10.69560/cujast.1843247

IZ

https://izlik.org/JA66JY36MM

APA

Toprak, A. G., Mercan, Ö. B., & Özdem, M. (2026). A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment. Sivas Cumhuriyet Üniversitesi Bilim Ve Teknoloji Dergisi, 5(1), 40-52. https://doi.org/10.69560/cujast.1843247

AMA

1.Toprak AG, Mercan ÖB, Özdem M. A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment. CUJAST. 2026;5(1):40-52. doi:10.69560/cujast.1843247

Chicago

Toprak, Amine Gonca, Öykü Berfin Mercan, and Mehmet Özdem. 2026. “A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment”. Sivas Cumhuriyet Üniversitesi Bilim Ve Teknoloji Dergisi 5 (1): 40-52. https://doi.org/10.69560/cujast.1843247.

EndNote

Toprak AG, Mercan ÖB, Özdem M (June 1, 2026) A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment. Sivas Cumhuriyet Üniversitesi Bilim ve Teknoloji Dergisi 5 1 40–52.

IEEE

[1]A. G. Toprak, Ö. B. Mercan, and M. Özdem, “A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment”, CUJAST, vol. 5, no. 1, pp. 40–52, June 2026, doi: 10.69560/cujast.1843247.

ISNAD

Toprak, Amine Gonca - Mercan, Öykü Berfin - Özdem, Mehmet. “A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment”. Sivas Cumhuriyet Üniversitesi Bilim ve Teknoloji Dergisi 5/1 (June 1, 2026): 40-52. https://doi.org/10.69560/cujast.1843247.

JAMA

1.Toprak AG, Mercan ÖB, Özdem M. A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment. CUJAST. 2026;5:40–52.

MLA

Toprak, Amine Gonca, et al. “A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment”. Sivas Cumhuriyet Üniversitesi Bilim Ve Teknoloji Dergisi, vol. 5, no. 1, June 2026, pp. 40-52, doi:10.69560/cujast.1843247.

Vancouver

1.Amine Gonca Toprak, Öykü Berfin Mercan, Mehmet Özdem. A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment. CUJAST. 2026 Jun. 1;5(1):40-52. doi:10.69560/cujast.1843247

A Siamese Neural Network-Based Perceptual Quality Metric for Image and Video Assessment

Abstract

Keywords

Supporting Institution

Project Number

Thanks

Görüntü ve Video Değerlendirmesi için Siyam Sinir Ağı Tabanlı Algısal Kalite Ölçütü

Abstract

Keywords

Supporting Institution

Project Number

Thanks

References

Details

Primary Language

Subjects

Journal Section

Authors

Publication Date

Submission Date

Acceptance Date

Published in Issue

DOI

IZ

Cite