Evrişimsel Sinir Ağı Tabanlı Hibrit Yaklaşım Kullanılarak Deepfake Video Algılama

Year 2024, EARLY VIEW, 1 - 1

Aynur Koçak , Mustafa Alkan , Süleyman Muhammed Arıkan

https://doi.org/10.2339/politeknik.1523983

Abstract

Son derece gerçekçi sahte içeriklerin oluşturulmasına olanak tanıyan deepfake teknolojisinin hızla ilerlemesi göz önüne alındığında, bu teknolojiyle ilişkili güvenlik risklerini ele almak için etkili bir çözüme acil ihtiyaç duyulmaktadır. Deepfake videoları, kimlik hırsızlığı potansiyeli, yanlış bilginin yayılması ve ulusal güvenliğin tehlikeye atılması gibi önemli etkileri nedeniyle yaygın olarak bilinmektedir. Bu nedenle, deepfake tespit algoritmalarının geliştirilmesi ve güvenilirliğinin artırılması hayati önem taşımaktadır. Bu çalışmada, DFDC veri setini kullanarak bir video veri setindeki deepfake'leri tespit etmek için Xception ve ResNet50 gibi derin öğrenme algoritmalarını kullanmak üzere özellik çıkarma teknikleri gerçekleştirildi. Ek olarak, SVM, KNN, MLP ve RF gibi çeşitli sınıflandırma algoritmaları kullanılarak toplam sekiz hibrit model geliştirildi. ResNet50 ve RF hibrit modelleri, %99,65'lik bir AUC değeriyle %98'lik en yüksek doğruluk oranına ulaştı. Bu çalışma, deepfake tespiti alanındaki farklı teknik zorlukları ele almak ve deepfake'leri etkili bir şekilde tespit etmek için geliştirilen bir makine öğrenimi yöntemini sunmaktadır. Önerilen yöntem, videolardaki sahte içeriği doğru bir şekilde tespit etmede etkinliğini kanıtlayarak, mevcut modellerle karşılaştırıldığında başarılı bir performans göstermiştir.

Keywords

sahte video tespiti, derin öğrenme, makine öğrenimi, Xception, ResNet50

Deepfake Video Detection Using Convolutional Neural Network Based Hybrid Approach

Abstract

Given the rapid advancement of deepfake technology, which allows for the creation of highly realistic fake content, there is a pressing need for an efficient solution to address the security risks associated with this technology. Deepfake videos are widely recognized for their significant implications, including the potential for identity theft, the dissemination of false information, and the endangerment of national security. Therefore, it is crucial to develop and enhance the reliability of deepfake detection algorithms. In this study, feature extraction techniques were performed to utilize deep learning algorithms such as Xception and ResNet50 to detect deepfakes in a video dataset using the DFDC dataset. Additionally, a total of eight hybrid models were developed using various classification algorithms such as SVM, KNN, MLP, and RF. The ResNet50 and RF hybrid models achieved the highest accuracy rate of 98%, with an AUC value of 99.65%. This study presents a machine learning method that has been developed to address different technical challenges in the field of deepfake detection and effectively identify deepfakes. The proposed method has demonstrated successful performance compared to state-of-the-art models, proving its effectiveness in accurately detecting fake content within videos.

Keywords

Deepfake video detection, deep learning, machine learning, Xception, ResNet50

References

• [1] M. Nawaz, Z. Mehmood, M. Bilal, A. M. Munshi, M. Rashid, R. M. Yousaf, et al., "Single and multiple regions duplication detections in digital images with applications in image forensic", Journal of Intelligent & Fuzzy Systems, vol. 40, pp. 10351-10371, (2021).
• [2] T. Nazir, A. Irtaza, A. Javed, H. Malik, A. Mehmood, and M. Nawaz, "Digital image forensic analysis using hybrid features", in 2021 International Conference on Artificial Intelligence (ICAI), pp. 33-36, (2021).
• [3] B. Chesney and D. Citron, "Deep fakes: A looming challenge for privacy, democracy, and national security", California Law Review, vol. 107, p. 1753, (2019).
• [4] FaceApp. Available: https://www. faceapp.com/ (12.06.2024).
• [5] FaceSwap. Available: https://www. faceswap.dev/ (12.06.2024).
• [6] B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, et al., "The deepfake detection challenge (dfdc) dataset", arXiv preprint arXiv:2006.07397, (2020).
• [7] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., "Generative adversarial net", Advances in Neural İnformation Processing Systems, vol. 27, (2014).
• [8] P. Yu, Z. Xia, J. Fei, and Y. Lu, "A survey on deepfake video detection", Iet Biometrics, vol. 10, pp. 607-624, (2021).
• [9] X. Chang, J. Wu, T. Yang, and G. Feng, "Deepfake face image detection based on improved VGG convolutional neural network", 39th Chinese Control Conference, pp. 7252-7256, (2020).
• [10] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", arXiv preprint arXiv:1409.1556, (2014).
• [11] Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, "Celeb-df: A large-scale challenging dataset for deepfake forensics", in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3207-3216, (2020).
• [12] B. Zi, M. Chang, J. Chen, X. Ma, and Y.-G. Jiang, "Wilddeepfake: A challenging real-world dataset for deepfake detection", in Proceedings of the 28th ACM International Conference on Multimedia, pp. 2382-2390, (2020).
• [13] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, "Joint face detection and alignment using multitask cascaded convolutional networks", IEEE Signal Processing Letters, vol. 23, pp. 1499-1503, (2016).
• [14] F. Chollet, "Xception: Deep learning with depthwise separable convolutions", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251-1258, (2017).
• [15] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, et al., "Imagenet large scale visual recognition challenge", International Journal of Computer Vision, vol. 115, pp. 211-252, (2015).
• [16] P. Korshunov and S. Marcel, "Deepfakes: a new threat to face recognition? assessment and detection", arXiv preprint arXiv:1812.08685, (2018).
• [17] A. G. Nicholas Dufour, Per Karlsson, Alexey Victor Vorbyov, Thomas Leung, Jeremiah Childs, and Christoph Bregler, "Deepfakes detection dataset by google & jigsaw", (2019).
• [18] D. Wodajo and S. Atnafu, "Deepfake video detection using convolutional vision transformer", arXiv preprint arXiv:2102.11126, (2021).
• [19] S. Fung, X. Lu, C. Zhang, and C.-T. Li, "Deepfakeucl: Deepfake detection via unsupervised contrastive learning", in 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, (2021).
• [20] A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, "Faceforensics++: Learning to detect manipulated facial images", in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1-11, (2019).
• [21] X. Yang, Y. Li, and S. Lyu, "Exposing deep fakes using inconsistent head poses", in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8261-8265, (2019).
• [22] L. Chen, Y. Zhang, Y. Song, L. Liu, and J. Wang, "Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection", in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18710-18719, (2022).
• [23] J. Wang, Z. Wu, W. Ouyang, X. Han, J. Chen, Y.-G. Jiang, et al., "M2tr: Multi-modal multi-scale transformers for deepfake detection", in Proceedings of the 2022 International Conference on Multimedia Retrieval, pp. 615-623, (2022).
• [24] V. V. V. N. S. Vamsi, S. S. Shet, S. S. M. Reddy, S. S. Rose, S. R. Shetty, S. Sathvika, et al., "Deepfake detection in digital media forensics", Global Transitions Proceedings, vol. 3, pp. 74-79, (2022).
• [25] S. Kingra, N. Aggarwal, and N. Kaur, "SiamNet: exploiting source camera noise discrepancies using Siamese network for Deepfake detection", Information Sciences, vol. 645, p. 119341, (2023).
• [26] L. Jiang, R. Li, W. Wu, C. Qian, and C. C. Loy, "Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection". in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2889-2898, (2022).
• [27] Y.-J. Heo, W.-H. Yeo, and B.-G. Kim, "Deepfake detection algorithm based on improved vision transformer", Applied Intelligence, vol. 53, pp. 7512-7527, (2023).
• [28] Z. Yang, J. Liang, Y. Xu, X.-Y. Zhang, and R. He, "Masked relation learning for deepfake detection", IEEE Transactions on Information Forensics and Security, vol. 18, pp. 1696-1708, (2023).
• [29] F. Khalid, A. Javed, H. Ilyas, and A. Irtaza, "DFGNN: An interpretable and generalized graph neural network for deepfakes detection", Expert Systems with Applications, vol. 222, p. 119843, (2023).
• [30] L. Zhang, T. Qiao, M. Xu, N. Zheng, and S. Xie, "Unsupervised learning-based framework for deepfake video detection", IEEE Transactions on Multimedia, 25, pp. 4785-4799, (2022).
• [31] A. Mitra, S. P. Mohanty, P. Corcoran, and E. Kougianos, "A machine learning based approach for deepfake detection in social media through key video frame extraction", SN Computer Science, vol. 2, pp. 98, (2021).
• [32] S. Mohiuddin, K. H. Sheikh, S. Malakar, J. D. Velásquez, and R. Sarkar, "A hierarchical feature selection strategy for deepfake video detection", Neural Computing and Applications, vol. 35, pp. 9363-9380, (2023).
• [33] S. Mirjalili, S. M. Mirjalili, and A. Lewis, "Grey wolf optimizer", Advances in Engineering Software, vol. 69, pp. 46-61, (2014).
• [34] B. Doğan and T. Ölmez, "A new metaheuristic for numerical function optimization: Vortex Search algorithm", Information Sciences, vol. 293, pp. 125-145, (2015).
• [35] Ş. Korkmaz and M. Alkan, "Derin öğrenme algoritmalarını kullanarak deepfake video tespiti", Politeknik Dergisi, vol. 26, pp. 855-862, (2023).
• [36] H. R. Hasan and K. Salah, "Combating deepfake videos using blockchain and smart contracts", Ieee Access, vol. 7, pp. 41596-41606, (2019).
• [37] Y. Nirkin, Y. Keller, and T. Hassner, "Fsgan: Subject agnostic face swapping and reenactment", in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7184-7193, (2019).
• [38] D. Huang and F. De La Torre, "Facial action transfer with personalized bilinear regression", in Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, pp. 144-158, (2012).
• [39] E. Zakharov, A. Shysheya, E. Burkov, and V. Lempitsky, "Few-shot adversarial learning of realistic neural talking head models", in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9459-9468, (2019).
• [40] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition", in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 770-778, (2016).
• [41] J. Pu, N. Mangaokar, L. Kelly, P. Bhattacharya, K. Sundaram, M. Javed, et al., "Deepfake videos in the wild: Analysis and detection", in Proceedings of the Web Conference 2021, pp. 981-992, (2021).
• [42] F. F. Kharbat, T. Elamsy, A. Mahmoud, and R. Abdullah, "Image feature detectors for deepfake video detection", in 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), pp. 1-4, (2019).
• [43] M. Masood, M. Nawaz, A. Javed, T. Nazir, A. Mehmood, and R. Mahum, "Classification of Deepfake videos using pre-trained convolutional neural networks", in 2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2), pp. 1-6, (2021).
• [44] N. Chakravarty and M. Dua, "A lightweight feature extraction technique for deepfake audio detection" Multimedia Tools and Applications, pp. 1-25, (2024).
• [45] A. Hamza, A. R. R. Javed, F. Iqbal, N. Kryvinska, A. S. Almadhor, Z. Jalil, et al., "Deepfake audio detection via MFCC features using machine learning", IEEE Access, vol. 10, pp. 134018-134028, (2022).