Extracting book titles from book recommendation videos using a deep learning approach

Abstract

Extracting text from images and videos is an emerging field of research with a wide range of applications, including video search, video editing, and translation. Nowadays, book promotion videos in different languages are shared on social media and especially on YouTube. In this study; It is recommended to take book titles through book promotion videos. The developed system takes video as input and separates the names of the books. The viewer can select the desired book by clicking on the detected book titles and watch the relevant part of the video. This application result in time saving by the viewer. In order to achieve this application, a deep learning-based system was developed to retrieve the names of books from videos. YOLO-based method was used in the study. Different YOLO algorithms were used in the study, and YOLOv5 was found to be more successful. This study contributes to the field of text extraction and video analysis by developing a deep learning-based approach to extract book titles from book promotion videos.

Keywords

• Book promotion videos
• book titles
• deep learning
• YOLOv5
• time saving

References

1. [1]. He, J., Zhang, Z., Yang, W., Zhang, J., Zhang, Y., and Zhang, H. (2018). Scene Text Detection and Recognition: The Deep Learning Era. International Journal of Computer Vision, 129(7), 1641-1671.
2. [2]. Chen, D., Odobez, J.-M., and Bourlard, H. (2014). Text detection and recognition in images and video frames. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3), 697-711.
3. [3]. Busta, M., Neumann, L., and Matas, J. (2017). Deep textspotter: an end-to-end trainable scene text localization and recognition framework. In Proceedings of the IEEE international conference on computer vision (pp. 2204-2212).
4. [4]. Liu, X., Meng, G., and Pan, C. (2019). Scene text detection and recognition with advances in deep learning: a survey. International Journal on Document Analysis and Recognition, 22, 143-162.
5. [5]. Chu, W.-S., Lin, Y.-H., & Yang, Y.-H. (2012). Title extraction from book cover images using histogram of oriented gradients and color information. International Journal of Contents, 8(4), 95-102.
6. [6]. Na, I. S. (2016). A Novel Scene Text Detection Method Using Histogram-Based Thresholding. IEEE Transactions on Image Processing, 25(2), 675-689.
7. [7]. Chen, Y., Liu, J., and Yu, K. (2018). Book Cover Image Recommendation Using Deep Learning. In Proceedings of the 2018 ACM Multimedia Conference on Multimedia Information Retrieval (MIR) (pp. 530-537).
8. [8]. Al-Omari, A. A., Al-Duwairi, M., and Al-Omari, K. (2020). Book Title Extraction from Book Cover Images Using Deep Learning. IEEE Transactions on Image Processing, 29(5), 2540-2551.

9. [9]. Akar, H. A., Al-Omari, A. A., and Al-Omari, K. (2022). Book Title Extraction from Book Cover Images Using Image Matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 2726-2739.
10. [10]. https://www.youtube.com
11. [11]. Li, Y., Fu, X., and Wang, J. (2019). A Survey on Object Detection and Tracking in Videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(11), 2789-2812.
12. [12]. Li, Z., Zhang, K., Mei, T., and Rui, Y. (2021). A Survey on Video Summarization Techniques. IEEE Transactions on Multimedia, 23(1), 281-304.
13. [13]. Li, H., Wang, Y., and Wang, W. (2022). A Survey on Multimodal Machine Learning for Video Understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 2711-2725.
14. [14]. He, J., Zhang, Z., Yang, W., Zhang, J., Zhang, Y., and Zhang, H. (2022). Deep Learning for Video Text Extraction: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 2699-2710.
15. [15]. He, J., Zhang, Z., Yang, W., Zhang, J., Zhang, Y., and Zhang, H. (2022). Deep Learning for Text Extraction from Images: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 2684-2698.