Query by Image Examination: Classification of Digital Image-Based Forensics Using Deep Learning Methods

Year 2023, Volume: 7 Issue: 2, 348 - 359, 29.12.2023

İlker Kara

https://doi.org/10.26650/acin.1282567

Abstract

The continuous increase in the use of information systems and online services has also spurred the forensic examination of digital and image data, which serves as the primary platform for information transfer. In particular, according to the latest reports, the examination of the images obtained from all kinds of recording devices that have the quality of evidence as a result of the forensic case and that can provide the clarification of the incident and the detection of the criminal elements are becoming a critical problem due to the huge amount of data. Our contribution in this study is two-folded. First, we present a new approach that classifies digital images into eight different crime categories using six different models. Second, we have created a new dataset for the classification of crimes and opened it to the public. Throughout the study, we have used our new dataset which has a total of 15,065 image samples from 8 different crime categories including Bet, ChildAbuse, Credit Card and Banking, Drugs, Frightening, Knives, Pornographic and Weapons. In this study, six different models were used to classify crime images. The CNN model was developed by us and five other models used for transfer learning. Pre-trained network model parameters VGG16, VGG19, Xception Model, InceptionResNetV2 and NASNetLarge were used for crime image classification tasks. In addition, the performance of these models is compared using test accuracy and time metrics. Resultly, we achieved prediction accuracy of up to 89.74% using the NASNetLarge model.

Keywords

image processing, deep learning method, image classification, data mining, forensic investigation

Görüntü İncelemesine Göre Sorgulama: Dijital Görüntü Tabanlı Adli Görüntülerin Derin Öğrenme Yöntemleri Kullanılarak Sınıflandırılması

Abstract

Bilgi sistemlerinin ve çevrimiçi hizmetlerin kullanımındaki sonsuz artış, bilgi aktarımı için temel platformlardan biri olan dijital ve görüntü içeren verilerin adli incelemelerini de tetiklemiştir. Adli görüntü inceleme temel olarak bilimsel yöntemlerin ve adli inceleme yazılımlar kullanılarak ilgili görüntüler hakkında delil oluşturulmasını sağlayan bilimsel bir disiplindir. Özellikle, son raporlara göre, adli vaka sonucunda delil niteliği taşıyan ve olayın aydınlanmasını sağlayabilecek her türlü kayıt cihazından elde edilmiş görüntülerin incelenmesi ve suç unsuru olanlarının tespiti artan veri miktarı nedeniyle giderek büyük bir problem haline gelmektedir. Bu çalışmada katkımız iki katkı sunmaktadır. İlk olarak dijital görüntülerin altı farklı model kullanarak sekiz farklı suç kategorisi olarak sınıflandıran yeni bir yaklaşım sunuyor. İkincisi, suçların sınıflandırılması için yeni bir veri kümesinin oluşturarak paylaşıma sunuyor. Çalışma boyunca, Bet, ChildAbuse, kredi kartı ve bankacılık, uyuşturucu, korkutucu, bıçak, pornografik ve silah dâhil olmak üzere 8 farklı suç Kategorisine ait toplam 15.065 görüntü örneğini kapsayan yeni veri setimizi kullanıldı. Suç görüntülerini sınıflandırmak için bu çalışmada 6 farklı model kullanılmıştır. CNN modeli kendimiz ve öğrenmeyi ince ayarlara aktarmak için kullanılan diğer beş model tarafından yaratılmıştır. Görüntü sınıflandırma görevleri için VGG16, VGG19, Xception modeli, InceptionResNetV2 ve NASNetLarge önceden eğitilmiş ağ modeli parametreleri kullanıldı. Ayrıca, bu modellerin performansı test doğruluğu ve zaman ölçümleri kullanılarak karşılaştırılır. Sonuçlar, NASNetLarge modeli kullanarak %89.74’e kadar tahmin doğruluğu elde edilmiştir.

Keywords

Görüntü işleme, derin öğrenme yöntemi, görüntü sınıflandırması, veri madenciliği, adli inceleme

References

• adfsolutio, nshttps://www.adfsolutions.com/, 2021. google scholar
• Belkasoft, https://belkasoft.com/, 2021. google scholar
• Birajdar, G.K., & Mankar, V.H. (2013). Digital image forgery detection using passive techniques: A survey. Digital investigation, 10(3), 226-245. google scholar
• Cao, H., & Kot, A.C. 2009. Accurate detection of demosaicing regularity for digital image forensics. IEEE Transactions on Information Forensics and Security, 4(4), 899-910. google scholar
• Chandra, M., Pandey, S., Chaudhary, R. (2010). Digital watermarking technique for protecting digital images. In 2010 3rd International Conference on Computer Science and Information Technology 7, 226-233. IEEE. google scholar
• Choodum, A., Boonsamran, P., NicDaeid, N., Wongniramaikul, W. (2015). On-site semi-quantitative analysis for ammonium nitrate detection using digital image colourimetry. Science & Justice, 55(6), 437-445. google scholar
• Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251-1258). google scholar
• Coulibaly, S., Kamsu-Foguem, B., Kamissoko, D., Traore, D. (2019). Deep neural networks with transfer learning in millet crop images. Computers in Industry, 108, 115-120. google scholar
• Europol, (2020). How Are Organised Crime Groups Involved in Sports Corruption?. https://www.europol.europa.eu/newsroom/news/how-areorganised-crime-groups-involved-in-sports-corruption. google scholar
• de Castro Polastro, M., da Silva Eleuterio, P.M. (2010). Nudetective: A forensic tool to help combat child pornography through automatic nudity detection. In 2010 Workshops on Database and Expert Systems Applications, 349-353. IEEE. google scholar
• Del Mar-Raave, J. R., Bahşi, H., Mrsic, L., Hausknecht, K. 2021. A machine learning-based forensic tool for image classification-A design science approach. Forensic Science International: Digital Investigation, 38, 301265. google scholar
• Dey, T., Mandal, S., Varcho, W. (2017). Improved image classification using topological persistence. In Proceedings of the conference on Vision, Modeling and Visualization, 161-168). google scholar
• Ferreira, W.D., Ferreira, C.B., da Cruz Junior, G., Soares, F. (2020). A review of digital image forensics. Computer & Electrical Engineering, 85, 106685. google scholar
• Hafiz, R., Haque, M. R., Rakshit, A., Uddin, M. S. (2020). Image-based soft drink type classification and dietary assessment system using deep convolutional neural network with transfer learning. Journal of King Saud University-Computer and Information Sciences. 34(5), 1775-1784. google scholar
• Garfinkel, S.L., Parker-Wood, A., Huynh, D., Migletz, J. (2010). An automated solution to the multiuser carved data ascription problem. IEEE Transactions on Information Forensics and Security, 5(4), 868-882. google scholar
• Grillo, A., Lentini, A., Me, G., Ottoni, M. (2009). Fast user classifying to establish forensic analysis priorities. In 2009 Fifth International Conference on IT Security Incident Management and IT Forensics, 69-77. IEEE. google scholar
• Gomez, L. S. M. (2012). Triage in-Lab: case backlog reduction with forensic digital profiling. In Proceedings of the Argentine Conference on Informatics and Argentine Symposium on Computing and Law, 217-225. google scholar
• Isnard, A., Council, T. C. (2001). Can surveillance cameras be successful in preventing crime and controlling anti-social behaviours. In Character, Impact and Prevention of Crime in Regional Australia Conference. google scholar
• Kara, I. (2017). A Review About Child Abuse Crimes Committed Through Internet In Turkey. Int J Forensic Sci Pathol, 5(3), 337-340. google scholar
• Karakuş, S., Kaya, Ö. Ü., Ertam, Ö. Ü. F., Talu, M. F. (2018). Derin Öğrenme Yöntemlerinin Kullanılarak Dijital Deliller Üzerinde Adli Bilişim İncelemesi. google scholar
• Keras, https://keras.io/api/applications/, 2021. google scholar
• Kuhle, L.F., Oezdemir, U., Beier, K.M. (2021). Child Sexual Abuse and the Use of Child Sexual Abuse Images. In Pedophilia, Hebephilia and Sexual Offending against Children (pp. 15-25). Springer, Cham. google scholar
• Lin, X., Li, J.H., Wang, S.L., Cheng, F., Huang, X.S. (2018). Recent advances in passive digital image security forensics: A brief review. Engineering, 4(1), 29-39. google scholar
• Mahalakshmi, S.D., Vijayalakshmi, K., Priyadharsini, S. (2012). Digital image forgery detection and estimation by exploring basic image manipulations. Digital Investigation, 8(3-4), 215-225. google scholar
• Marturana, F., Tacconi, S. (2013). A Machine Learning-based Triage methodology for automated categorization of digital media. Digital Investigation, 10(2), 193-204. google scholar
• Marturana, F., Me, G., Berte, R., Tacconi, S. (2011). A quantitative approach to triaging in mobile forensics. In 2011IEEE 10th International Conference on Trust, Sec urity and Privacy in Computing and Communications (pp. 582-588). IEEE. google scholar
• McClelland, D., Marturana, F. (2014). A Digital Forensics Triage methodology based on feature manipulation techniques. In 2014 IEEE International Conference on Communications Workshops (ICC) (pp. 676-681). IEEE. google scholar
• McDown, R.J., Varol, C., Carvajal, L., Chen, L. (2016). In-Depth Analysis of Computer Memory Acquisition Software for Forensic Purposes. Journal Of Forensic Sciences, 61, S110-S116. google scholar
• M Kirchner& R. Böhme (2007). Tamper hiding: Defeating image forensics In International Workshop on Information Hiding, Springer, Berlin, Heidelberg. (2007), pp.326-341. google scholar
• Olmos, R., Tabik, S., Herrera, F. (2018). Automatic handgun detection alarm in videos using deep learning. Neurocomputing, 275, 66-72. google scholar
• Paluszek, M., & Thomas, S. (2020). Practical Matlab deep learning. A Project-Based Approach, Michael Paluszek and Stephanie Thomas. google scholar
• Pearson, H. (2006). Forensic software traces tweaks to images. Nature, 439(7076), 520-522. google scholar
• Peng, F., Liu, J., Long, M. (2013). Identification of natural images and computer generated graphics based on hybrid features. In Emerging Digital Forensics Applications for Crime Detection, Prevention, and Security (pp. 18-34). IGI Global. google scholar
• Peng, C., Liu, Y., Yuan, X., & Chen, Q. (2022). Research of image recognition method based on enhanced inception-ResNet-V2. Multimedia Tools and Applications, 81(24), 34345-34365. google scholar
• Piva, A. (2013). An overview on image forensics. International Scholarly Research Notices, 2013. google scholar
• Rahimzadeh, M., Parvin, S., Safi, E., Mohammadi, M.R. (2021). Wise-SrNet: A Novel Architecture for Enhancing Image Classification by Learning Spatial Resolution of Feature Maps. arXiv preprint arXiv:2104.12294. google scholar
• Ren, S., He, K., Girshick, R., Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28, 91-99. google scholar
• Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Fei-Fei, L. (2015). Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211-252. google scholar
• Saber, A. H., Khan, M. A., & Mejbel, B. G. (2020). A survey on image forgery detection using different forensic approaches. Advances in Science, Technology and Engineering Systems Journal, 5(3), 361-370. google scholar
• Sharma, M., & Vig, L. (2018). Automatic classification of low-resolution chromosomal images. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops (pp. 0-0). google scholar
• Sanap, V.K., & Mane, V. (2015). Comparative study and simulation of digital forensic tools. Int J Comput Appl, 975, 8887. google scholar
• Seigfried, K.C., Lovely, R.W., Rogers, M.K. (2008). Self-Reported Online Child Pornography Behavior: A Psychological Analysis. International Journal of Cyber Criminology, 2(1). google scholar
• Sharma, A., Singh, A., Choudhury, T., Sarkar, T. (2021). Image Classification using ImageNet Classifiers in Environments with Limited Data. google scholar
• Simonyan, K., Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. google scholar
• Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A.A. (2017). Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence. google scholar
• Tammina, S. (2019). Transfer learning using vgg-16 with deep convolutional neural network for classifying images. International Journal of Scientific and Research Publications (USRP), 9(10), 143-150. google scholar
• Thakur, R., Rohilla, R. (2020). Recent advances in digital image manipulation detection techniques: A brief review. Forensic Science International, 312, 110311. google scholar
• Verma, G.K., Dhillon, A. (2017). A handheld gun detection using faster r-cnn deep learning. In Proceedings of the 7th International Conference on Computer and Communication Technology (pp. 84-88). google scholar
• x-ways, https://www.x-ways.net/, 2021. google scholar
• Wu, L.T., Parrott, A.C., Ringwalt, C L., Yang, C., Blazer, D.G. (2009). The variety of ecstasy/MDMA users: results from the National Epidemiologic Survey on alcohol and related conditions. The American Journal on Addictions, 18(6), 452-461. google scholar
• Zhang, X., Zou, J., He, K., Sun, J. (2015). Accelerating very deep convolutional networks for classification and detection. IEEE transactions on pattern analysis and machine intelligence, 38(10), 1943-1955. google scholar
• Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V. (2018). Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8697-8710). google scholar
• Wang, W., Dong, J., Tan, T. (2009). A survey of passive image tampering detection. In International Workshop on Digital Watermarking (pp. 308-322). Springer, Berlin, Heidelberg. google scholar
• Dateset, https://ilkerkara.karatekin.edu.tr/RequestDataset.htmlekle dataset, 2021. google scholar