Personalized advertisement using deep learning-based object detection algorithms

Abstract

Today, internet ads are personalized by accessing people's cookies and session information and achieving high success. This study aims to apply in an environment similar to real-life advertising on Internet ads. Personal advertisements were suggested by examining the age, gender, and dressing styles of customers who came to the locations with a camera and screen to be placed at the entry points or billboards of the stores. Thus, it is planned to increase sales by showing the products that the users may like and attracting the user's attention. Next, the image datasets obtained from the internet were examined with deep learning algorithms, and the age, gender, and clothing style of the person in the image were analyzed and determined. YOLOv3 object detection algorithm was used in the clothing, and a model that was previously trained in the age and gender section was retrained with the help of the TensorFlow library. A suggestion system was created according to the estimation results of the models found after the training was completed. For example, a young woman, who wears a shirt and a skirt, is shown exclusively in the advertising inventory of the store, with a skirt or shirt advertisement for young women. Then, the study made suggestions by taking images of the people with the help of a camera, and the results were determined as acceptable.

Keywords

• Object detection
• personalized advertisement
• deep learning

Derin öğrenme temelli nesne tespiti algoritmaları kullanılarak kişiye özgü reklam sunulması

Abstract

Günümüzde internet reklamları kişilerin çerez ve oturum bilgilerine erişerek kişiselleştirilmekte ve yüksek bir başarı elde etmektedir. Bu çalışmanın amacı internet reklamlarına benzer bir ortamın gerçek hayattaki reklamlar üzerinde uygulanmasıdır. Mağazaların giriş noktalarına veya ilan tahtalarına koyulacak bir kamera ve ekran ile gelen müşterilerin yaş, cinsiyet ve giyim tarzlarını inceleyerek kişiye özel reklamlar önerilmiştir. Böylelikle kullanıcıya beğenebileceği ürünleri gösterip kullanıcının ilgisini çekerek, satışların arttırılması planlanmaktadır. Bir sonraki aşamada internetten elde edilen görüntü verisetleri derin öğrenme algoritmaları ile incelenerek, görüntüdeki kişinin yaş, cinsiyet ve giyim tarzı analiz ve tespit edilmiştir. Giysi kısmında YOLOv3 algoritması kullanılmış olup, yaş ve cinsiyet kısmında önceden eğitilmiş olan bir model TensorFlow kütüphanesi yardımıyla tekrar eğitilerek kullanılmıştır. Eğitimler tamamlandıktan sonra elde edilen modellerin tahmin sonuçlarına göre bir öneri sistemi oluşturulmuştur. Örneğin gömlek ve etek giyen genç bir kadına, mağazanın reklam envanterinde, genç kadınlar için bulunan etek veya gömlek reklamı kişiye özgü olarak gösterilmektedir. Daha sonra çalışma bir kamera yardımıyla kişilerin görüntüsü alınarak önerilerde bulunmuş ve sonuçlar kabul edilebilir belirlenmiştir.

Keywords

• Nesne tespiti
• kişiye özgü reklam
• derin öğrenme

References

1. Deloitte, Reklamcılar Derneği, Türkiye’de Tahmini Medya ve Reklam Yatırımları: 2019 ilk 6 Ay Raporu, Technical Report, Istanbul, (2019).
2. Cloudian, Intel, Dentsu, Deep Learning Enables Intelligent Billboard for Dynamic, Targeted Advertising on Tokyo Expressway, Technical Report, USA, (2017).
3. Sternson, T., Wang, Q., Gulmammadova, W., Sun G. ve Kenzie J., AWS DeepAds Advertising, https://aws.amazon.com/tr/deeplens/community-projects/DeepAds_Advertising/, (06-Feb-2021).
4. Doğan, F. ve Türkoğlu, İ., Derin Öğrenme Algoritmalarının Yaprak Sınıflandırma Başarımlarının Karşılaştırılması,” Sakarya University Journal of Computer and Information Sciences, 1, 1, 10–21, (2018).
5. Fan, S., Li, J., Zhang, Y., Tian, X., Wang, Q., He, X., Zhang, C. and Huang, W., On line detection of defective apples using computer vision system combined with deep learning methods, Journal of Food Engineering, 286, 110102, (2020).
6. Kayaalp, K. ve Metlek, S., Classification of Robust and Rotten Apples by Deep Learning Algorithm, Sakarya University Journal of Computer and Information Sciences, 3, 2, 112–120, (2020).
7. Momeny M., Jahanbakhshi A., Jafarnezhad K. ve Zhang, Y.-D., Accurate classification of cherry fruit using deep CNN based on hybrid pooling approach, Postharvest Biology and Technology, 166, 111204, (2020).
8. Uğuz, S., Automatic Olive Peacock Spot Disease Recognition System Development by Using Single Shot Detector, Sakarya University Journal of Computer and Information Sciences, 3, 3, 158–168, (2020).

9. Adak, M. F., Identification of Plant Species by Deep Learning and Providing as A Mobile Application, Sakarya University Journal of Computer and Information Sciences, 3, 3, 231–238, (2020).
10. Uçar, M. ve Uçar, E., Computer-Aided Detection of Lung Nodules in Chest X-Rays using Deep Convolutional Neural Networks, Sakarya University Journal of Computer and Information Sciences, 2, 1, 1–8, (2018).
11. Anthimopoulos, M., Christodoulidis, S., Ebner, L., Christe, A., ve Mougiakakou, S., Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network, IEEE Transactions on Medical Imaging, 35, 5, 1207–1216, (2016).
12. Akyol, F. B. ve Altun, O., Detection of Cervix Cancer from Pap-smear Images, Sakarya University Journal of Computer and Information Sciences, 3, 2, 99–111, (2020).
13. Khamparia, A., Gupta, D., de Albuquerque, V. H. C., Sangaiah, A. K. ve Jhaveri, R. H., Internet of health things-driven deep learning system for detection and classification of cervical cells using transfer learning, The Journal of Supercomputing, 76, 11, 8590–8608, (2020).
14. Erdem, E. ve Aydın, T., Detection of Pneumonia with a Novel CNN-based Approach, Sakarya University Journal of Computer and Information Sciences,4, 1, 26–34, (2021).
15. Çelik, G. ve Talu, M. F., Çekişmeli üretken ağ modellerinin görüntü üretme performanslarının incelenmesi, Balıkesir Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 22, 1, 181–192, (2020).
16. Gündüz, G. ve Cedimoğlu, İ. H., Derin Öğrenme Algoritmalarını Kullanarak Görüntüden Cinsiyet Tahmini, Sakarya University Journal of Computer and Information Sciences, 2, 1, 9–17, (2019).
17. Bilgin, M. ve Şentürk, İ. F., Danışmanlı ve yarı danışmanlı öğrenme kullanarak doküman vektörleri tabanlı tweetlerin duygu analizi, Balıkesir Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 21, 2, 822–839, (2019).
18. Erdoğmuş, P., Deep Learning Performance on Medical Image, Data and Signals, Sakarya University Journal of Computer and Information Sciences, 2, 1, 28–40, (2019).
19. Bedeli, M., Geradts, Z. ve van Eijk, E., Clothing identification via deep learning: forensic applications, Forensic Sciences Research, 3, 3, 219–229, (2018).
20. Saxena, K. ve Shibata, T., Garment Recognition and Grasping Point Detection for Clothing Assistance Task using Deep Learning, IEEE/SICE International Symposium on System Integration (SII), 632–637, Paris, (2019).
21. Dong, Q., Gong, S. ve Zhu, X., Multi-task Curriculum Transfer Deep Learning of Clothing Attributes, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 520–529, California, (2017).
22. Lin, K., Yang, H.-F., Liu, K.-H., Hsiao, J.-H. ve Chen C.-S., Rapid Clothing Retrieval via Deep Learning of Binary Codes and Hierarchical Search, Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, 499–502, Shanghai, (2015).
23. Cychnerski, J., Brzeski, A., Boguszewski, A., Marmolowski, M. ve Trojanowicz, M., Clothes detection and classification using convolutional neural networks, 2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 1–8, Limassol, Cyprus, (2017).
24. Liu, Z., Luo, P., Qiu, S., Wang, X. ve Tang, X., DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1096–1104, Nevada, (2016).
25. Han X., Wu Z., Huang, P.X., Zhang, X., Zhu, M., Li, Y., Zhao, Y. ve Davis L.S., Automatic Spatially-Aware Fashion Concept Discovery, IEEE International Conference on Computer Vision (ICCV), 1472–1480, Venice, (2017).
26. Gong, K., Liang, X., Zhang, D., Shen, X. ve Lin, L., Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6757–6765, Hawaii, (2017).
27. Krizhevsky A., Sutskever I. ve Hinton G. E., ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems 25 (NIPS 2012), 1097–1105, Nevada, (2012).
28. Bouwmans, T., Javed, S., Sultana, M. ve Jung, S.K., Deep neural network concepts for background subtraction: A systematic review and comparative evaluation, Neural Networks, 117, 8–66, (2019).
29. Girshick, R., Donahue, J., Darrell, T. ve Malik, J., Rich feature hierarchies for accurate object detection and semantic segmentation, 2014 IEEE Conference on Computer Vision and Pattern Recognition, 580-587, Ohio, 2014.
30. Ipek, B. ve Akpinar, M. Application of Deep Learning Based Object Detection on Unmanned Aerial Vehicle, 2020 IEEE 5th International Conference on Computer Science and Engineering (UBMK), 1-5, Diyarbakır, (2020).
31. Girshick, R., Fast R-CNN, 2015 IEEE International Conference on Computer Vision (ICCV), 1440-1448, Santiago, (2015).
32. Ren, S., He, K., Girshick, R. ve Sun, J., Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 6, 1137-1149, (2017).
33. Redmon, J., Divvala, S., Girshick, R. ve Farhadi A., You Only Look Once: Unified, Real-Time Object Detection, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779-788, Nevada, (2016).
34. Chen, W., Huang, H., Peng, S., Zhou, C. ve Zhang, C., YOLO-face: a real-time face detector, The Visual Computer, (2020).
35. Du, J., Understanding of Object Detection Based on CNN Family and YOLO, Journal of Physics: 2nd International Conference on Machine Vision and Information Technology (CMVIT 2018), 012029, Hong Kong, (2018).
36. Tao, J., Wang, H., Zhang, X., Li, X. ve Yang, H., An object detection system based on YOLO in traffic scene, 6th International Conference on Computer Science and Network Technology (ICCSNT), 315–319, Dalian, (2017).
37. Buric, M., Pobar, M. ve Ivasic-Kos, M., Ball Detection Using Yolo and Mask R-CNN,” 2018 International Conference on Computational Science and Computational Intelligence (CSCI), 319–323, Nevada, (2018).
38. Redmon, J. ve Farhadi, A., YOLO9000: Better, Faster, Stronger, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6517–6525, Hawaii, (2017).
39. Laval, M., Tomato detection based on modified YOLOv3 framework, Scientific Reports, 11, 1, 1447, 1-11, (2021).
40. Viola, P. ve Jones, M. J., Robust Real-Time Face Detection, International Journal of Computer Vision, 57, 2, 137–154, (2004).
41. Levi, G. ve Hassncer, T., Age and gender classification using convolutional neural networks,” 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 34–42, Massachusetts, (2015).
42. Hassner, T., Harel, S., Paz, E. ve Enbar, R., Effective face frontalization in unconstrained images, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4295–4304, Massachusetts, (2015).
43. Eidinger, E., Enbar, R., ve Hassner, T., Age and Gender Estimation of Unfiltered Faces, IEEE Transactions on Information Forensics and Security, 9,12, 2170–2179, (2014).
44. Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S. ve Murphy, K., “Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3296–3297, Hawaii, (2017).
45. Tian, Y., Yang, G., Wang, Z., Wang, H., Li, E. ve Liang, Z., Apple detection during different growth stages in orchards using the improved YOLO-V3 model, Computers and Electronics in Agriculture, 157, 417–426, (2019).