Audio forgery detection method based on visual word

Abstract

Changing the content of these recordings constitutes a crime if speech recordings are used as evidence in judicial cases. Audio copy-move forgery is the most common forgery made to change the content of the conversation. This forgery is carried out by copying a word or group of words in the speech and pasting it to any position within the same speech. In this study, a robust new method based on visual words is proposed to detect audio copy-move forgery. The proposed method uses Mel-Spectogram images of words extracted from the audio to detect forgery clues in the suspicious audio file. For this purpose, the audio file is first separated into words using the pitch-based VAD method. Each word is then converted into a Mel Spectogram image. DSSIM is used to calculate the similarity between spectrogram images. Forgery segments are marked according to the DSSIM values between word images. Experimental results show that the proposed method has significantly higher robustness to post-processing operations and yields higher accuracy compared to other works.

Keywords

• Audio forgery
• Copy-move
• Spectogram
• DSSIM

Görsel kelime tabanlı ses sahteciliği tespit yöntemi

Öz

Ses kayıtlarının adli olaylarda delil olarak kullanılması halinde, bu kayıtlarının içeriğinin değiştirilmesi suç teşkil etmektedir. Ses kopyala-yapıştır sahteciliği, konuşmanın içeriğini değiştirmek amacıyla yapılan sahteciliklerden en yaygın olanıdır. Bu sahtecilik, sesteki bir kelime ya da kelime grubunun kopyalanıp, aynı sesin içinde herhangi bir konuma yapıştırılmasıyla gerçekleştirilmektedir. Bu çalışmada ses kopyala-yapıştır sahteciliğini tespit etmek için görsel kelimelere dayalı sağlam ve yeni bir yöntem önerilmektedir. Önerilen yöntem, şüpheli ses dosyasındaki sahtecilik ipuçlarını tespit etmek için sesten elde edilen kelimelerin Mel-Spectogram görüntülerini kullanır. Bu amaçla ses dosyası öncelikle perde bazlı ses aktivite algılama (Voice Activity Detection-VAD) yöntemi kullanılarak kelimelere ayrılır. Daha sonra her kelime Mel Spectogram görüntüsüne dönüştürülür. Spectogram görüntüleri arasındaki benzerliği hesaplamak için yapısal farklılık (Structural Difference-DSSIM) kullanılır. Kelime görüntüleri arasındaki DSSIM değerlerine göre sahte kelimeler işaretlenir. Deneysel sonuçlar, önerilen yöntemin diğer çalışmalara kıyasla son işlem operasyonlarına karşı önemli ölçüde yüksek dayanıklılığa sahip olduğunu ve daha yüksek doğruluk değerini verdiğini göstermektedir.

Anahtar Kelimeler

• Ses sahteciliği
• Kopyala-Yapıştır
• Spectogram
• DSSIM

Kaynakça

1. S. Keser, Ö. N. Gerek, E. Seke, M. B. Gülmezoğlu, A subspace based progressive coding method for speech compression. Speech Communication, 94, 50-61, 2017. https://doi.org/10.1016/j.specom.2017.09.002
2. S. Keser, R. Edizkan, Phonem-based isolated Turkish word recognition with subspace classifier. In 2009 IEEE 17th Signal Processing and Communications Applications Conference, pp. 93-96, Antalya, Turkey 2009.
3. O. F. Çıplak, S. Kevser, Gerçek zamanlı ses tanıma ile robot kolu kontrolü. Avrupa Bilim ve Teknoloji Dergisi, 31, 34-39, 2021. https://doi.org/10.31590/ejosat.969608
4. J. N. Xiao, Y. Z Jia, E. D Fu., Z. Huang, Y. Li, & S. P. Shi, Audio authenticity: Duplicated audio segment detection in waveform audio file. Journal of Shanghai Jiaotong University (Science), 19, 392-397, 2014. https://doi.org/10.1007/s12204-014-1515-5
5. Z. Su, M. Li, G. Zhang, Q. Wu, & Y. Wang, Robust audio copy-move forgery detection on short forged slices using sliding window. Journal of Information Security and Applications, 75, 103507, 2023. https://doi.org/10.1016/j.jisa.2023.103507
6. Z. Su, M. Li, G. Zhang, Q. Wu, M. Li, W. Zhang & X. Yao, Robust Audio Copy-Move Forgery Detection Using Constant Q Spectral Sketches and GA-SVM. IEEE Transactions on Dependable and Secure Computing, 2022. https://doi.org/10.1109/TDSC.2022.3215280
7. F. Wang, C. Li, L. Tian, An algorithm of detecting audio copy-move forgery based on DCT and SVD. In: 2017 IEEE 17th International Conference on Communication Technology (ICCT). IEEE, pp. 1652–1657, Chengdu, China, 2017.
8. Q. Yan, R. Yang, J.Huang, Copy-move detection of audio recording with pitch similarity. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp. 1782–1786, Brisbane, Australia, 2015.

9. Z. Xie, W. Lu, X. Liu, Y. Xue, Y. Yeung, Copy-move detection of digital audio based on multifeature decision. Journal of Information Security and Applications, 43, 37-46, 2018. https://doi.org/10.1016/j.jisa.2018.10.003
10. M. Imran, Z. Ali, S.T. Bakhsh, S. Akram, Blind detection of copy-move forgery in digital audio forensics. IEEE Access 5, 12843–12855, 2017. https://doi.org/10.1109/ACCESS.2017.2717842
11. N.T. Anh, H.T.T. Hang, G. Chen, One approach in the time domain in detecting copy-move of speech recordings with the similar magnitude. International Journal of Engineering and Applied Sciences (IJEAS), 6(4), 9–11, 2019. https://dx.doi.org/10.31873/IJEAS/6.4.2019.05
12. K. Mannepalli, P. Krishna, K. Krishna, Copy and move detection in audio recordings using dynamic time warping algorithm. International Journal of Innovative Technology and Exploring Engineering (IJITEE), 9(2), 2244–2249, 2019. http://doi.org/10.35940/ijitee.B6678.129219
13. B. Ustubioglu, B. Kucukugurlu, G. Ulutas, Robust copy-move detection in digital audio forensics based on pitch and modifed discrete cosine transform. Multimedia Tools and Applications, 81, 27149–27185, 2022. https://doi.org/10.1007/s11042-022-13035-3
14. X. Huang, Zi Liu, W. Lu, H. Liu, S. Xiang, Fast and effective copy-move detection of digital audio based on auto segment. In: Digital forensics and forensic investigations: breakthroughs in research and practice. IGI Global, pp. 127–142, 2020.
15. Q. Yan, R. Yang, J. Huang, Robust copy–move detection of speech recording using similarities of pitch and formant. IEEE Transactions on Information Forensics and Security, 14(9), 2331–2341, 2019. https://doi.org/10.1109/TIFS.2019.2895965
16. A. Ustubioglu, B. Ustubioglu, G. Ulutas, Mel spectrogram-based audio forgery detection using CNN. Signal, Image and Video Processing, 17, 2211 - 2219, 2022. https://doi.org/10.1007/s11760-022-02436-4
17. B. Ustubioglu, G. Tahaoglu, G. Ulutas, Detection of audio copy-move-forgery with novel feature matching on Mel spectrogram. Expert Systems with Applications 213, 118963, 2023. https://doi.org/10.1016/j.eswa.2022.118963
18. B. Ustubioglu, G. Tahaoglu, G. Ulutas, A. Ustubioglu & M. Kilic, Audio forgery detection and localization with super-resolution spectrogram and keypoint-based clustering approach. The Journal of Supercomputing, 80, 486-518, 2023. https://doi.org/10.1007/s11227-023-05504-9
19. A. Stephen, H. Hu，A spectral/temporal method for robust fundamental frequency tracking，The Journal of the Acoustical Society of America, 123:6, pp:4559-4571，2008. https://doi.org/10.1121/1.2916590
20. Z. Wang, A.C, Bovik Sheikh, H.R., Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing. 13:4, 600–612, 2004. https://doi.org/10.1109/TIP.2003.819861
21. TIMIT Acoustic-Phonetic Continuous Speech Corpus, https://catalog.ldc.upenn.edu/LDC93s1