ResNet Modellerini Kullanarak Deepfake Video Tespiti için Topluluk Tabanlı Derin Öğrenme Çerçevesi

Öz

Deepfake teknolojisi, manipüle edilmiş dijital içerik yoluyla yanlış bilgilendirme, kimlik sahtekarlığı ve güvenlik tehditlerine olanak sağlama potansiyeli nedeniyle giderek artan endişelere yol açmıştır. Bu riskler artmaya devam ettikçe, güvenilir deepfake tespit yöntemlerinin geliştirilmesi hayati önem kazanmıştır. Bu çalışmada, tespit doğruluğunu ve kararlılığını iyileştirmek için topluluk stratejisine dayalı hibrit bir derin öğrenme yaklaşımı önerilmiştir. Analizi ilgili alanlara odaklamak için öncelikle MTCNN algoritması kullanılarak video karelerinden yüz bölgeleri çıkarılmıştır. Daha sonra, farklı soyutlama seviyelerinde tamamlayıcı temsilleri yakalamak için değişen derinliklerinden yararlanılarak derin ve ayırt edici özellikler çıkarmak için ResNet50, ResNet101 ve ResNet152 kullanılmıştır. Çıkarılan özellik vektörleri daha sonra çok katmanlı algılayıcı (MLP) sınıflandırıcılar kullanılarak sınıflandırılmış ve tahminleri birleştirmek ve sınıflandırma hatalarını azaltmak için çoğunluk oylama şeması uygulanmıştır. Önerilen yöntem, Celeb-DF veri kümesinin dengeli bir alt kümesi üzerinde değerlendirilmiştir. Deneysel sonuçlar, topluluğun bireysel modellere kıyasla daha istikrarlı ve sağlam bir performans sağladığını ve %98,82 AUC, %96,92 doğruluk ve %97,67 geri çağırma oranına ulaştığını göstermektedir. Bağımsız modeller arasında ResNet50 en yüksek performansı göstermiştir. Bu bulgular, farklı derinliklerdeki ağlar tarafından öğrenilen özelliklerin entegre edilmesinin güvenilirliği artırdığını ve daha tutarlı deepfake video tespitini desteklediğini göstermektedir.

Anahtar Kelimeler

• Deepfake tespiti
• Topluluk öğrenimi
• ResNet
• MTCNN
• MLP
• Derin Öğrenme

An Ensemble Deep Learning Framework for Deepfake Video Detection Using ResNet Models

Öz

Deepfake technology has raised growing concerns due to its potential to enable misinformation, identity fraud, and security threats through manipulated digital content. As these risks continue to increase, the development of reliable deepfake detection methods has become essential. In this study, a spatial feature-based ensemble deep learning framework is proposed to improve detection accuracy and classification stability. Facial regions were first extracted from video frames using the MTCNN algorithm to focus the analysis on relevant facial areas. The proposed framework focuses on spatial facial representations extracted from individual video frames rather than explicit temporal modeling. Subsequently, ResNet50 and ResNet101 were employed as the main spatial feature extractors to obtain deep facial representations at different abstraction levels. ResNet152 was also evaluated experimentally to examine the effect of a deeper feature extraction branch; however, it was not included in the final framework because it increased runtime without improving video-level classification performance. The extracted feature vectors were classified using independent multilayer perceptron (MLP) classifiers, and the final prediction was obtained using a two-model weighted soft voting ensemble. The proposed method was evaluated on a balanced subset of the Celeb-DF dataset using a video-level train-test split to reduce data leakage between training and test sets. Experimental results showed that the final ResNet50–ResNet101 weighted soft voting ensemble achieved 94.17% video-level accuracy, 0.986 AUC, 0.9440 F1-score, and 0.8864 MCC. The two-model ensemble achieved performance comparable to the three-model configuration while providing a more favorable runtime profile. These findings suggest that combining complementary spatial facial representations through weighted soft voting provides a reliable and computationally efficient approach for deepfake video detection.

Anahtar Kelimeler

• Deepfake detection
• Ensemble learning
• ResNet
• MTCNN
• MLP
• Deep Learning

Kaynakça

1. [1] Demircan B. “Gördüğün Her Şeye İnanma”: Derin Sahte Çalışmaları Üzerine Bir Doküman Analizi. TRT Akademi. 2021;6(13):728-747.
2. [2] Albahar M, Almalki J. Deepfakes: Threats and countermeasures systematic review. Journal of Theoretical and Applied Information Technology. 2019;97(22):3242-3250.
3. [3] Belen N, Yalçın N. Deepfake Teknolojisi ve Ceza Hukuku: Ses ve Görüntü Manipülasyonlarından Doğan Cezai Sorumluluk. The Journal of International Scientific Researches. 2025;10(2):202-228.
4. [4] FaceApp. FaceApp: Perfect Face Editor. https://www.faceapp.com/. Accessed 2026 Nov 6.
5. [5] Faceswap. Deepfakes software for all. https://github.com/deepfakes/faceswap. Accessed 2026 May 21.
6. [6] Papastratis I. Deepfakes: Face synthesis with GANs and Autoencoders. AI Summer. https://theaisummer.com/deepfakes/. Published 2020.
7. [7] Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In: Advances in Neural Information Processing Systems 27 (NIPS 2014); 2014 Dec 8-13; Montréal, Canada. Curran Associates, Inc.; 2014. p. 2672-2680.
8. [8] Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models. In: Advances in Neural Information Processing Systems 33 (NeurIPS 2020); 2020 Dec 6-12; Virtual conference (Vancouver, Canada). Curran Associates, Inc.; 2020. p. 6840-6851. Available from: https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html

9. [9] Yu P, Xia Z, Fei J, Lu Y. A survey on deepfake video detection. IET Biometrics. 2021;10(6):607-624.
10. [10] Koçak A, Alkan M, Arıkan SM. Deepfake video detection using convolutional neural network based hybrid approach. Politeknik Dergisi. 2025;28(3):957-968.
11. [11] Li Y, Yang X, Sun P, Qi H, Lyu S. Celeb-DF: A large-scale challenging dataset for DeepFake forensics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020 Jun 13-19; Virtual conference. IEEE; 2020. p. 3207-3216. Available from: https://doi.org/10.1109/CVPR42600.2020.00327
12. [12] Chang X, Wu J, Yang T, Feng G. DeepFake face image detection based on improved VGG convolutional neural network. In: Proceedings of the 39th Chinese Control Conference (CCC); 2020 Jul 27-29; Shenyang, China. IEEE; 2020. p. 7252-7256. Available from: https://doi.org/10.23919/CCC50068.2020.9189596
13. [13] Rossler A, Cozzolino D, Verdoliva L, Riess C, Thies J, Niessner M. FaceForensics++: Learning to detect manipulated facial images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); 2019 Oct 27-Nov 2; Seoul, South Korea. IEEE; 2019. p. 1-11. Available from: https://doi.org/10.1109/ICCV.2019.00009
14. [14] Li Y, Lyu S. Exposing DeepFake videos by detecting face warping artifacts [Preprint]. arXiv; 2018. Available from: https://doi.org/10.48550/arXiv.1811.00656
15. [15] Afchar D, Nozick V, Yamagishi J, Echizen I. MesoNet: A compact facial video forgery detection network. In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS); 2018. p. 1-7. Available from: https://doi.org/10.1109/WIFS.2018.8630761
16. [16] Nguyen HH, Yamagishi J, Echizen I. Use of a capsule network to detect fake images and videos [Preprint]. arXiv; 2019. Available from: https://doi.org/10.48550/arXiv.1910.12467
17. [17] Zhang K, Zhang Z, Li Z, Qiao Y. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters. 2016;23(10):1499-1503.
18. [18] Pu J, Mangaokar N, Kelly L, Bhattacharya P, Sundaram K, Javed M, Wang B, Viswanath B. Deepfake videos in the wild: Analysis and detection. In: Proceedings of the Web Conference 2021 (WWW '21); 2021 Apr 19-23; Ljubljana, Slovenia. ACM; 2021. p. 981-992. Available from: https://doi.org/10.1145/3442381.3449978
19. [19] Prakash Kumar PS, MoheshKumar G, Shanmuga Priya K, Dharun Kumar V, Nanda Krishnan P, Barathraj SA. Enhanced deepfake detection using ResNet50 and facial landmark analysis. In: Proceedings of the 1st International Conference on Research and Development in Information, Communication, and Computing Technologies (ICRDICCT); 2025 Apr 4-5; Nagapattinam, India. SciTePress; 2025. p. 848-855. Available from: https://doi.org/10.5220/0013948000004919
20. [20] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 Jun 27-30; Las Vegas, USA. IEEE; 2016. p. 770-778. Available from: https://doi.org/10.1109/CVPR.2016.90.
21. [21] Öksüz C, Urhan O, Güllü MK. An integrated convolutional neural network with attention guidance for improved performance of medical image classification. Neural Computing and Applications. 2024;36(4):2067-2099.
22. [22] Öksüz C, Narter A, Ece B, Koyun M, Taşkent İ, Güllü MK. I2IReg–ClfNet: A cascaded multi-task deep learning framework for ROI-aware kidney stone detection in abdominal CT images. Biomedical Signal Processing and Control. 2026;119:109857.
23. [23] Bıçakcı Yeşilkaya HS, Guest R. Activity-aware electrocardiogram biometric verification utilising deep learning on wearable devices. EURASIP Journal on Information Security. 2025;2025(1):7.
24. [24] Zhang D, Li C, Lin F, Zeng D, Ge S. Detecting deepfake videos with temporal dropout 3DCNN. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21); 2021 Aug 19-26; Montréal, Canada (virtual conference). International Joint Conferences on Artificial Intelligence Organization; 2021. p. 1288-1294. Available from: https://doi.org/10.24963/ijcai.2021/178
25. [25] Cheng Z, Wang Y, Wan Y, Jiang C. DeepFake detection method based on multi-scale interactive dual-stream network. Journal of Visual Communication and Image Representation. 2024; 104:104263.
26. [26] Xia Z, Qiao T, Xu M, Wu X, Han L, Chen Y. Deepfake video detection based on MesoNet with preprocessing module. Symmetry. 2022;14(5):939.
27. [27] Almestekawy A, Zayed HH, Taha A. Deepfake detection: Enhancing performance with spatiotemporal texture and deep learning feature fusion. Egyptian Informatics Journal. 2024; 27:100535.