TY  - JOUR
T1  - Derin Sahte Ses Manipülasyonu Tespit Sistemleri Üzerine Bir Derleme
TT  - A Review of Deepfake Audio Manipulation Detection Systems
AU  - Tahaoglu, Gul
AU  - Kılıç, Muhammed
AU  - Üstübioğlu, Beste
AU  - Ulutaş, Güzin
PY  - 2024
DA  - April
DO  - 10.53433/yyufbed.1358880
JF  - Yüzüncü Yıl Üniversitesi Fen Bilimleri Enstitüsü Dergisi
JO  - YYU JINAS
PB  - Van Yüzüncü Yıl Üniversitesi
WT  - DergiPark
SN  - 1300-5413
SP  - 353
EP  - 402
VL  - 29
IS  - 1
LA  - tr
AB  - Gerçek kişilerin konuşmalarını içeren dijital ses dosyalarının kullanılması ile gerçekleştirilen derin sahte ses manipülasyonu, sesi taklit edilecek kişinin sesini klonlayarak kişinin söylemediği bir şeyi söylemiş gibi içerikte ses dosyalarını oluşturan bir sahtecilik türüdür. Konuşmacının kimliğini doğrulamak için güvenlik adımı olarak kabul edilen Otomatik Konuşmacı Doğrulama Sistemlerinin derin sahte ses sahtecilikleri saldırılarına karşı savunmasızlığı söz konusudur. Ayrıca mahkemelerde karar merciini etkileyecek delil olarak sunulan ses dosyalarının orijinal olup olmadığı kontrolü önemli bir ihtiyaç haline gelmiştir. Bu tür sahteciliklerin uzman sistemler tarafından tespit edilebilmesi günümüz çağı için oldukça önem arz etmektedir. Bu sahtecilik türündeki saldırıların tespit edilebilmesi için literatürde çeşitli yöntemler önerilmiştir. Literatürdeki çalışmalarda performans değerlendirmesinde kullanılan ücretsiz erişimli veri setleri de mevcut olup sonuç kıyaslamasında kullanabilmesi mümkündür. Bu çalışmada literatürdeki yöntemler ve verisetleri incelenmiş, yöntemlerin bu verisetleri üzerindeki performans değerlendirmeleri, avantaj ve dezavantajları vurgulanmıştır.
KW  - Derin sahte ses manipülasyonu
KW  - Derin sahte ses tespiti
KW  - Ses doğrulama
N2  - Besides facilitating access to audio content on the Internet, developments in deep learning methods have made it possible to produce deep fake audio. Automatic Speaker Verification systems considered a security step to authenticate the speaker, are vulnerable to deep spoofing attacks. It is crucial for today's age that expert systems can detect such frauds. Deep fake audio spoofing is carried out to produce audio files in the content by cloning the speaker's voice that is planned to be changed as if he said something he did not say. Various methods are proposed in the literature to detect this type of forgery. There are free-access datasets used in performance evaluation in studies in the literature, and it is possible to use them in result comparison. The planned research aims to reduce or eliminate the noise that may exist in the audio file of the system by passing the preprocessing stage of the audio signal received as input. This paper examines the methods and datasets in the literature, and the advantages and disadvantages of the methods on these datasets are emphasized.
CR  - Abdzadeh, P., & Veisi, H. (2023). A comparison of CQT spectrogram with STFT-based acoustic features in Deep Learning-based synthetic speech detection. Journal of AI and Data Mining, 11(1), 119-129. doi:10.22044/jadm.2022.12373.2382
CR  - Alluri, K. N. R. K., & Vuppala, A. K. (2019, September). IIIT-H spoofing countermeasures for automatic speaker verification spoofing and countermeasures challeng. Interspeech 2019, Graz, Austria. doi:10.21437/Interspeech.2019-1623
CR  - Alzantot, M., Wang, Z., & Srivastava, M. B. (2019, September). Deep residual neural networks for audio spoofing detection. Interspeech 2019, Graz, Austria. doi:10.21437/Interspeech.2019-3174
CR  - Balamurali, B. T., Lin, K. W. E., Lui, S., Chen, J. M., & Herremans, D. (2019). Toward robust audio spoofing detection: a detailed comparison of traditional and learned features. IEEE Access, 7, 84229-84241. doi:10.1109/ACCESS.2019.2923806
CR  - Borrelli, C., Bestagini, P., Antonacci, F., Sarti, A., & Tubaro, S. (2021). Synthetic speech detection through short-term and long-term prediction traces. EURASIP Journal on Information Security, 2021, 2. doi:10.1186/s13635-021-00116-3
CR  - Cai, W., Wu, H., Cai, D., & Li, M. (2019, September). The DKU replay detection system for the ASVspoof 2019 challenge: on data augmentation, feature representation, classification, and fusion. Interspeech 2019, Graz, Austria. doi:10.21437/Interspeech.2019-1230
CR  - Chen, T., Kumar, A., Nagarsheth, P., Sivaraman, G., & Khoury, E. (2020, November). Generalization of audio deepfake detection.  The Speaker and Language Recognition Workshop (Odyssey 2020), Tokyo, Japan. doi:10.21437/Odyssey.2020-19
CR  - Chen, Z., Xie, Z., Zhang, W., & Xu, X. (2017, August). ResNet and model fusion for automatic spoofing detection. Interspeech 2017, Stockholm, Sweeden. doi:10.21437/Interspeech.2017-1085
CR  - Cheng, X., Xu, M., & Zheng, T. F. (2019, March). Replay detection using CQT-based modified group delay feature and ResNeWt network in ASVspoof 2019. 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Lanzhou, China. doi:10.1109/APSIPAASC47483.2019.9023158
CR  - Chettri, B., Stoller, D., Morfi, V., Ramírez, M. A. M., Benetos, E., Sturm, B. L. (2019, September). Ensemble models for spoofing detection in automatic speaker verification. Interspeech 2019, Graz, Austria. doi:10.21437/Interspeech.2019-2505
CR  - Chintha, A., Thai, B., Sohrawardi, S. J., Bhatt, K., Hickerson, A., Wright, M., & Ptucha, R. (2020). Recurrent convolutional structures for audio spoof and video deepfake detection. Journal of Selected Topics in Signal Processing, 14(5), 1024-1037. doi:10.1109/JSTSP.2020.2999185
CR  - Dua, M., Jain, C., & Kumar, S. (2022). LSTM and CNN based ensemble approach for spoof detection task in automatic speaker verification systems. Journal of Ambient Intelligence and Humanized Computing, 13, 1985-2000. doi:10.1007/s12652-021-02960-0
CR  - Font, R., Espín, J. M., & Cano, M. J. (2017, August). Experimental analysis of features for replay attack detection — results on the ASVspoof 2017 challenge. Interspeech 2017, Stockholm, Sweeden. doi:10.21437/Interspeech.2017-450
CR  - Gunendradasan, T., Wickramasinghe, B., Le, N. P., Ambikairajah, E., & Epps, J. (2018, September). Detection of replay-spoofing attacks using frequency modulation features.  Interspeech 2018, Hyderabad, India. doi:10.21437/Interspeech.2018-1473
CR  - Hua, G., Teoh, A. B. J., & Zhang, H. (2021). Towards end-to-end synthetic speech detection. IEEE Signal Processing Letters, 28, 1265-1269. doi:10.1109/LSP.2021.3089437
CR  - Jiang, Z., Zhu, H., Peng, L., Ding, W., & Ren, Y. (2020, October). Self-supervised spoofing audio detection scheme. Interspeech 2020, Shangai, China. doi:10.21437/Interspeech.2020-1760
CR  - Kinnunen, T., Delgado, H., Evans, N., Lee, K.A., Vestman, V., Nautsch, A., …, & Reynolds, D. A. (2020). t-DCF: a detection cost function for the tandem assessment of spoofing countermeasures and automatic speaker verification. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28, 2195-2210 2020.
CR  - Korshunov, P., Marcel, S., Muckenhirn, H., Gonçalves, A. R., Souza Mello, A. G., Velloso, V. R. P., …, & Sahidullah, M. (2016, September). Overview of BTAS 2016 speaker anti-spoofing competition. 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS), Niagara Falls, NY, USA. doi:10.1109/BTAS.2016.7791200
CR  - Kwak, Y., Kwag, S., Lee, J., Jeon, Y., Hwang, J., Choi, H.J., …, & Yoon, J. W. (2023). Voice spoofing detection through residual network, max feature map, and depthwise separable convolution. IEEE Access, 11, 49140-49152. doi:10.1109/ACCESS.2023.3275790
CR  - Lai, CI, Chen, N., Villalba, J., & Dehak, N. (2019, September). ASSERT: Anti-spoofing with squeeze-excitation and residual networks. Interspeech 2019, Graz, Austria. doi:10.21437/Interspeech.2019-1794
CR  - Lavrentyeva, G., Novoselov, S., Malykh, E., Kozlov, A., Kudashev, O., & Shchemelinin, V. (2017, August). Audio replay attack detection with deep learning frameworks. Interspeech 2017, Stockholm, Sweden. doi:10.21437/Interspeech.2017-360
CR  - Mewada, H., Al-Asad, J. F., Almalki, F. A., Khan, A. H., Almujally, N. A., El-Nakla, S., & Naith, Q. (2023). Gaussian-filtered high-frequency-feature trained optimized BiLSTM network for spoofed-speech classification. Sensors, 23, 6637. doi:10.3390/s23146637
CR  - Nagarsheth, P., Khoury, E., Patil, K., & Garland, M. (2017, August). Replay attack detection using DNN for channel discrimination. Interspeech 2017, Stockholm, Sweden. doi:10.21437/Interspeech.2017-1377
CR  - Nautsch, A., Wang, X., Evans, N., Kinnunen, T. H., Vestman, V., Todisco, M., …, & Lee, K. A. (2021).  ASVspoof 2019: Spoofing countermeasures for the detection of synthesized, converted and replayed speech. IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(2), 252-265. doi:10.1109/TBIOM.2021.3059479
CR  - Patel, T. B., & Patil, H. A. (2015, September). Combining evidences from mel cepstral, cochlear filter cepstral and instantaneous frequency features for detection of natural vs. spoofed speech. Interspeech 2015, Dresden, Germany. doi:10.21437/Interspeech.2015-467
CR  - Paul, D., Sahidullah, M., & Saha, G. (2017, March). Generalization of spoofing countermeasures: A case study with ASVspoof 2015 and BTAS 2016 corpora. 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA. doi:10.1109/ICASSP.2017.7952516
CR  - Qian, Y., Chen, N., & Yu, K. (2016). Deep features for automatic spoofing detection. Speech Communication, 85, 43-52. doi:10.1016/j.specom.2016.10.007
CR  - Rahul, T. P., Aravind, P. R., Ranjith, C., Usamath, N., & Paramparambath, N. (2020). Audio spoofing verification using deep convolutional neural networks by transfer learning. ArXiv, abs/2008.03464,2020. doi:10.48550/arXiv.2008.03464
CR  - Reimao, R., & Tzerpos, V. (2019, October). FoR: A dataset for synthetic speech detection. 2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Timisoara, Romania. doi:10.1109/SPED.2019.8906599
CR  - Suthokumar, G., Sethu, V., Wijenayake, C., & Ambikairajah, E. (2018, September). Modulation dynamic features for the detection of replay attacks. Interspeech 2018, Hyderabad, India. doi:10.21437/Interspeech.2018-1846
CR  - Sriskandaraja, K., Sethu, V., Ambikairajah, E., & Li, H. (2017). Front-end for antispoofing countermeasures in speaker verification: Scattering spectral decomposition. IEEE Journal of Selected Topics in Signal Processing, 11(4), 632-643. doi:10.1109/JSTSP.2016.2647202
CR  - Sriskandaraja, K., Sethu, V., & Ambikairajah, E. (2018, September). Deep siamese architecture based replay detection for secure voice biometric. Interspeech 2018, Hyderabad, India. doi:10.21437/Interspeech.2018-1819
CR  - Tak, H., Patino, J., Todisco, M., Nautsch, A., Evans, N., & Larcher, A. (2021, June). End-to-End anti-spoofing with RawNet2. ICASSP 2021- 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada. doi:10.1109/ICASSP39728.2021.9414234
CR  - Tan, C. B., Hijazi, M. H. A., & Nohuddin, P. N. E. (2023, September). A hybrid classification approach for artificial speech detection. 2023 IEEE International Conference on Artificial Intelligence in Engineering and Technology (IICAIET), Kota Kinabalu, Malaysia. doi:10.1109/IICAIET59451.2023.10291764
CR  - Xiao, X., Tian, X., Du, S., Xu, H., Siong, C. E., & Li, H. (2015, September). Spoofing speech detection using high dimensional magnitude and phase features: the NTU approach for ASVspoof 2015 challenge. Interspeech 2015, Dresden, Germany. doi:10.21437/Interspeech.2015-465
CR  - Wu, Z., Kinnunen, T., Evans, N., Yamagishi, J., Hanilçi, C., Sahidullah, M., & Sizov, A. (2015, September). ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge. Interspeech 2015, Dresden, Germany. doi:10.21437/Interspeech.2015-462
CR  - Wu, Z., Yamagishi, J., Kinnunen, T., Hanilçi, C., Sahidullah, M., Sizov, ..., & Delgado, H. (2017). ASVspoof: the automatic speaker verification spoofing and countermeasures challenge. IEEE Journal of Selected Topics in Signal Processing, 11(4), 588-604. doi:10.1109/JSTSP.2017.2671435
CR  - Witkowski, M., Kacprzak, S., Żelasko, P., Kowalczyk, K., & Gałka, J. (2017, August). Audio replay attack detection using high-frequency features. Interspeech 2017, Stockholm, Sweden. doi:10.21437/Interspeech.2017-776
CR  - Wang, L., Yoshida, Y., Kawakami, Y., & Nakagawa, S. (2015, September). Relative phase information for detecting human speech and spoofed speech. Interspeech 2015, Dresden, Germany. doi:10.21437/Interspeech.2015-473
CR  - Yamagishi, J., Wang, X., Todisco, M., Sahidullah, M., Patino, J., Nautsch, A., …, & Delgado H. (2021, September). ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection. 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge, France. doi:10.21437/ASVSPOOF.2021-8
CR  - Yang, J., You, C., & He, Q. (2018, September). Feature with complementarity of statistics and principal information for spoofing detection. Interspeech 2018, Hyderabad, India. doi:10.21437/Interspeech.2018-1693
CR  - Yu, H., Tan, Z. H., Zhang, Y., Ma, Z., Guo, J. (2017). DNN filter bank cepstral coefficients for spoofing detection. IEEE Access, 5, 4779-4787. doi:10.1109/ACCESS.2017.2687041
CR  - Zarish, A., Javed, A., & Khalid, M. (2022). AEXANet: An end-to-end deep learning based voice anti-spoofing system. Workshop on Artificial Intelligence for Multimedia Forensics and Disinformation Detection (AI4MFDD).
CR  - Zhang, Y., Jiang, F., & Duan, Z. (2021). One-class learning towards synthetic voice spoofing detection. IEEE Signal Processing Letters, 28, 937-941. https://doi.org/10.1109/LSP.2021.3076358
CR  - Zhang, Z., Yi, X., & Zhao, X. (2021, June). Fake speech detection using residual network with transformer encoder. Proceedings of the 2021 ACM workshop on information hiding and multimedia security, Belgium. doi:10.1145/3437880.3460408
CR  - Zhang, J., Tu, G., Liu, S., & Cai, Z. (2023). Audio anti-spoofing based on audio feature fusion. Algorithms, 16, 317. doi:10.3390/a16070317
UR  - https://doi.org/10.53433/yyufbed.1358880
L1  - https://dergipark.org.tr/tr/download/article-file/3401680
ER  -