A Quantitative Study on Forensic Analysis of Images Produced with Deepfake Tools and Deepfake Detection

Öz

Today's technology pushes the limits of imagination and takes its place in a large part of our lives with fast and accessible devices. Technological growth provides great convenience to people in many areas. However, considering the speed and quantity of social media and technology reaching individuals, the impact of this technological acceleration on individuals and societies is increasing day by day. In addition to the material and moral benefits provided by social media and technology, adverse situations such as manipulated images, videos, sounds, fake news and other cyber-crimes may also be encountered. Therefore, it is important to be aware that artifacts left in the virtual world can be used by malicious individuals. This study is a methodologically quantitative study implemented in the 2022-2023 academic year. The study group of the research consists of a total of 120 participants who work in the field of computer forensics (60 participants) and those who are not computer forensic experts (60 participants). The data collection tools of the research are the socio-demographic form and the "True False Test" developed by the researcher, which consists of 30 items to measure deepfake detection skills. According to some results of the research, the correct detection rate is lower in photographs taken through the Swapface deep editing program. It has been observed that deep editing photographs made through the Swapface program are more successful than deep editing photographs made with the Face Swapper program. It has been determined that it is not easy to detect photographs created with deep editing technology with the naked human eye and that some tools must be used.

Anahtar Kelimeler

• Forensics
• deepfake
• machine learning
• manipulation
• cyber security
• artificial intelligence

Derin Kurgu (Deepfake) Araçları ile Üretilen Resimlerin Adli Analizi ve Derin Kurgu Tespiti Üzerine Nicel Bir Çalışma

Öz

Günümüz teknolojisi hayal gücünün sınırlarını zorlayarak hızlı ve ulaşılabilir cihazlarla yaşantımızın büyük bir bölümünde yerini almaktadır. Teknolojik büyüme birçok alanda insanlara büyük kolaylıklar sağlamaktadır. Ancak, sosyal medyanın ve teknolojinin bireylere ulaşma hızı ve niceliği göz önüne alındığında, bu teknolojik ivmenin bireyler ve toplumlar üzerindeki etkisi her geçen gün artmaktadır. Sosyal medya ve teknolojinin sağladığı maddi ve manevi faydaların yanı sıra, manipüle edilmiş resimler, videolar, sesler, sahte haberler ve diğer siber suçlar gibi aksi durumlarla da karşılaşılabilmektedir. Bu nedenle, sanal dünyada bırakılan kalıntıların kötü niyetli kişiler tarafından kullanılabileceği konusunda bilinçli olmak önemlidir. Bu çalışma, 2022-2023 eğitim-öğretim yılında uygulanmış, metodolojik açıdan nicel bir çalışmadır. Araştırmanın çalışma grubu, adli bilişim alanında çalışan (60 katılımcı) ve adli bilişimci olmayan (60 katılımcı) toplam 120 katılımcıdan oluşmaktadır. Araştırmanın veri toplama araçları, sosyo-demografik form ve araştırmacı tarafından geliştirilen ve derin kurgu (deepfake) tespit becerisini ölçmek için 30 maddeden oluşan "Doğru Yanlış Testi"dir. Araştırmanın bazı sonuçlarına göre, Swapface derin kurgu yapma programı vasıtasıyla yapılan fotoğraflarda doğru tespit oranı daha düşüktür. Swapface programı vasıtasıyla yapılan derin kurgu fotoğraflarının, Face Swapper programıyla yapılan derin kurgu fotoğraflarına göre daha başarılı olduğu görülmüştür. Derin kurgu teknolojisiyle oluşturulan fotoğrafların tespit edilmesinde çıplak insan gözüyle tespitin kolay olmadığı, birtakım araçların kullanılması gerektiği belirlenmiştir.

Anahtar Kelimeler

• Adli bilişim
• deepfake
• makine öğrenmesi
• manipülasyon
• siber güvenlik
• yapay zekâ.

Kaynakça

1. A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, ve M. Niessner, “FaceForensics++: Learning to detect manipulated facial images”, Proceedings of the IEEE International Conference on Computer Vision, c. 2019-Octob, ss. 1–11, 2019, doi: 10.1109/ICCV.2019.00009.
2. I. Goodfellow vd., “Generative adversarial networks”, Communications of the ACM, c. 63, sayı 11, ss. 139–144, 2020, doi: 10.1145/3422622.
3. Y. Li, X. Yang, P. Sun, H. Qi, ve S. Lyu, “Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, ss. 3204–3213, 2020, doi: 10.1109/CVPR42600.2020.00327.
4. L. Jiang, R. Li, W. Wu, C. Qian, ve C. C. Loy, “Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, ss. 2886–2895, 2020, doi: 10.1109/CVPR42600.2020.00296.
5. M. Taeb ve H. Chi, “Comparison of Deepfake Detection Techniques through Deep Learning”, Journal of Cybersecurity and Privacy, c. 2, sayı 1, ss. 89–106, 2022, doi: 10.3390/jcp2010007.
6. E. Şafak ve N. Barışçı, “Hafif Evrişimsel Sinir Ağları Kullanılarak Sahte Yüz Görüntülerinin Tespiti”, El-Cezeri Fen ve Mühendislik Dergisi, c. 2022, sayı 4, ss. 1282–1289, 2022, doi: 10.31202/ecjse.1133527.
7. M. Westerlund, “The emergence of deepfake technology: A review”, Technology Innovation Management Review, c. 9, sayı 11, ss. 39–52, 2019, doi: 10.22215/TIMREVIEW/1282.
8. P. Korshunov ve S. Marcel, “DeepFakes: a New Threat to Face Recognition? Assessment and Detection”, ss. 1–5, 2018, . Available at: http://arxiv.org/abs/1812.08685

9. D. Dagar ve D. K. Vishwakarma, “A literature review and perspectives in deepfakes: generation, detection, and applications”, International Journal of Multimedia Information Retrieval, c. 11, sayı 3, ss. 219–289, 2022, doi: 10.1007/s13735-022-00241-w.
10. A. A. Maksutov, V. O. Morozov, A. A. Lavrenov, ve A. S. Smirnov, “Methods of Deepfake Detection Based on Machine Learning”, Proceedings of the 2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering, EIConRus 2020, ss. 408–411, 2020, doi: 10.1109/EIConRus49466.2020.9039057.
11. S. H. Silva, M. Bethany, A. M. Votto, I. H. Scarff, N. Beebe, ve P. Najafirad, “Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models”, Forensic Science International: Synergy, c. 4, sayı January, s. 100217, 2022, doi: 10.1016/j.fsisyn.2022.100217.
12. S. Tariq, S. Lee, H. Kim, Y. Shin, ve S. S. Woo, “GaN is a friend or foe? A framework to detect various fake face images”, Proceedings of the ACM Symposium on Applied Computing, c. Part F1477, sayı April, ss. 1296–1303, 2019, doi: 10.1145/3297280.3297410.
13. D. Cozzolino, J. Thies, A. Rössler, C. Riess, M. Nießner, ve L. Verdoliva, “ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection”, 2018, . Available at: http://arxiv.org/abs/1812.02510
14. S. Y. Wang, O. Wang, R. Zhang, A. Owens, ve A. A. Efros, “CNN-Generated Images Are Surprisingly Easy to Spot.. For Now”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, ss. 8692–8701, 2020, doi: 10.1109/CVPR42600.2020.00872.
15. Y. Li ve S. Lyu, “Exposing DeepFake Videos By Detecting Face Warping Artifacts”, sayı November 2018, 2018, . Available at: http://arxiv.org/abs/1811.00656
16. K. Simonyan ve A. Zisserman, “Very deep convolutional networks for large-scale image recognition”, 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, ss. 1–14, 2015.
17. X. Chang, J. Wu, T. Yang, ve G. Feng, “DeepFake Face Image Detection based on Improved VGG Convolutional Neural Network”, Chinese Control Conference, CCC, c. 2020-July, ss. 7252–7256, 2020, doi: 10.23919/CCC50068.2020.9189596.
18. J. Kim, S. Han, ve S. S. Woo, “Classifying Genuine Face images from Disguised Face Images”, Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019, ss. 6248–6250, 2019, doi: 10.1109/BigData47090.2019.9005683.
19. G. Huang, Z. Liu, L. Van Der Maaten, ve K. Q. Weinberger, “Densely connected convolutional networks”, Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, c. 2017-Janua, ss. 2261–2269, 2017, doi: 10.1109/CVPR.2017.243.
20. C. C. Hsu, Y. X. Zhuang, ve C. Y. Lee, “Deep fake image detection based on pairwise learning”, Applied Sciences (Switzerland), c. 10, sayı 1, 2020, doi: 10.3390/app10010370.
21. F. Matern, C. Riess, ve M. Stamminger, “Exploiting visual artifacts to expose deepfakes and face manipulations”, Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019, ss. 83–92, 2019, doi: 10.1109/WACVW.2019.00020.
22. P. Korshunov ve S. Marcel, “Speaker inconsistency detection in tampered video”, European Signal Processing Conference, c. 2018-Septe, ss. 2375–2379, 2018, doi: 10.23919/EUSIPCO.2018.8553270.
23. S. Agarwal, H. Farid, Y. Gu, M. He, K. Nagano, ve H. Li, “Protecting world leaders against deep fakes”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, c. 2019-June, ss. 38–45, 2019.
24. F. Chollet, “Xception: Deep learning with depthwise separable convolutions”, Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, c. 2017-Janua, ss. 1800–1807, 2017, doi: 10.1109/CVPR.2017.195.
25. Q. V. Le Mingxing Tan, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing”, Canadian Journal of Emergency Medicine, c. 15, sayı 3, s. 190, 2013.
26. A. Kumar, A. Bhavsar, ve R. Verma, “Detecting Deepfakes with Metric Learning”, 2020 8th International Workshop on Biometrics and Forensics, IWBF 2020 - Proceedings, sayı March, 2020, doi: 10.1109/IWBF49977.2020.9107962.
27. A. Ismail, M. Elpeltagy, M. S. Zaki, ve K. Eldahshan, “An integrated spatiotemporal-based methodology for deepfake detection”, Neural Computing and Applications, c. 34, sayı 24, ss. 21777–21791, 2022, doi: 10.1007/s00521-022-07633-3.
28. A. A. Pokroy ve A. D. Egorov, “EfficientNets for DeepFake Detection: Comparison of Pretrained Models”, Proceedings of the 2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering, ElConRus 2021, ss. 598–600, 2021, doi: 10.1109/ElConRus51938.2021.9396092.
29. D. A. Coccomini, N. Messina, C. Gennaro, ve F. Falchi, “Combining EfficientNet and Vision Transformers for Video Deepfake Detection”, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), c. 13233 LNCS, ss. 219–229, 2022, doi: 10.1007/978-3-031-06433-3_19.
30. A. Mitra, S. P. Mohanty, P. Corcoran, ve E. Kougianos, “A Novel Machine Learning based Method for Deepfake Video Detection in Social Media”, Proceedings - 2020 6th IEEE International Symposium on Smart Electronic Systems, iSES 2020, ss. 91–96, 2020, doi: 10.1109/iSES50453.2020.00031.
31. M. Dolecki, P. Karczmarek, A. Kiersztyn, ve W. Pedrycz, “Utility functions as aggregation functions in face recognition”, 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016, sayı 2014, 2017, doi: 10.1109/SSCI.2016.7850120.
32. H. Zhao, W. Zhou, D. Chen, T. Wei, W. Zhang, ve N. Yu, “Multi-attentional Deepfake Detection”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, ss. 2185–2194, 2021, doi: 10.1109/CVPR46437.2021.00222.
33. N. Bonettini, L. Bondi, E. D. Cannas, P. Bestagini, S. Mandelli, ve S. Tubaro, “Video face manipulation detection through ensemble of CNNs”, Proceedings - International Conference on Pattern Recognition, ss. 5012–5019, 2020, doi: 10.1109/ICPR48806.2021.9412711.
34. C. X. T. Du, L. H. Duong, H. T. Trung, P. M. Tam, N. Q. V. Hung, ve J. Jo, “Efficient-Frequency: A hybrid visual forensic framework for facial forgery detection”, 2020 IEEE Symposium Series on Computational Intelligence, SSCI 2020, ss. 707–712, 2020, doi: 10.1109/SSCI47803.2020.9308305.
35. B. Tunguz, “1 Million Fake Faces - 1 | Kaggle”. https://www.kaggle.com/datasets/tunguz/1-million-fake-faces (erişim 21 Mayıs 2023).
36. R. F. Burton, “Multiple-choice and true/false tests: Myths and misapprehensions”, Assessment and Evaluation in Higher Education, c. 30, sayı 1, ss. 65–72, 2005, doi: 10.1080/0260293042003243904.
37. B. M. Byrne, Structural Equation Modeling with Amos: Basic Concepts, Applications, and Programming, Third edition. 2016. doi: 10.4324/9781315757421.
38. Darren George and Paul Mallery, “Front 1 Front 2 Open Data”, ss. 1–404, 2019.
39. J. Hair, R. Anderson, B. Babin, ve W. Black, “Multivariate Data Analysis.pdf”, Australia : Cengage, c. 7 edition. s. 758, 2010.
40. J. Cohen, “F Tests on Means in the Analysis of Variance and Covariance”, Statistical Power Analysis for the Behavioral Sciences, ss. 273–406, Oca. 1977, doi: 10.1016/B978-0-12-179060-8.50013-X.
41. J. T. E. Richardson, “Eta squared and partial eta squared as measures of effect size in educational research”, Educational Research Review, c. 6, sayı 2, ss. 135–147, 2011, doi: 10.1016/j.edurev.2010.12.001.
42. Ş. Özdemir, “Yeni Nesil Tehdit: Derin Kurgu (DeepFake)”, TRT Akademi, c. 6, sayı 13, ss. 904–917, 2021, doi: 10.37679/trta.1002526.
43. Tekin, H. (1996). Eğitimde Ölçme ve Değerlendirme, Yargı Yayınları, 9. Baskı, Ankara, 312s.
44. Tezci, E. (2016). Eğitimde ölçümler ve değerlendirme. Detay Yayıncılık, Ankara.