Detecting subtitle regions in multimedia images using image processing techniques

Year 2023, , 1 - 15, 31.12.2023

Erdal Güvenoğlu

https://doi.org/10.55198/artibilimfen.1385122

Abstract

With the widespread use of mobile devices and multimedia technologies, the acquisition of images has become much easier. However, obtaining the subtitles in the images and using them for different purposes have emerged as a problem. In this study, a simple and effective process is proposed for detecting the regions where subtitles are present in multimedia images. The process consists of different successive steps. The coordinates of the subtitle region, which is the text, are determined with the help of image processing techniques on a 24-bit colour image given as an input image. Then, it is marked on the colour image. Experimental studies were carried out on images with different features and sizes. Harris corner detection algorithm was used to mark the corner points, Gaussian filtering and morphological image processing techniques were used to remove noise. A success rate of 94% was achieved in the studies performed. In time measurement tests, a good performance time of 1.56 seconds was achieved on average. The time measurements were compared with other studies in the literature. It has been observed that the proposed process has an excellent performance in terms of time.

Keywords

Subtitle detection, Image processing, Edge detection, Corner detection, Morphological image processing.

Görüntü işleme teknikleri kullanılarak multimedya görüntülerinde alt yazı bölgelerinin tespit edilmesi

Abstract

Mobil cihazların ve multimedya teknolojilerinin yaygın olarak kullanımı ile birlikte görüntülerin elde edilmesi çok daha kolay hale gelmiştir. Bununla birlikte görüntüler içerisinde yer alan alt yazıların elde edilmesi ve bunların farklı amaçlar için kullanımı bir problem olarak ortaya çıkmıştır. Bu çalışmada multimedya görüntülerinde yer alan altyazıların bulunduğu bölgelerin tespit edilmesi için kullanımı basit ve etkili bir yöntem önerilmiştir. Yöntem birbirini takip eden farklı adımlardan oluşmaktadır. Giriş görüntüsü olarak verilen 24 bit renkli bir görüntüler üzerinde görüntü işleme teknikleri yardımıyla metin olan alt yazı bölgesine ait koordinatlar belirlenmektedir. Ardından renkli görüntü üzerinde işaretlenmektedir. Birbirinden farklı özellik ve ölçülerde görüntüler üzerinde deneysel çalışmalar gerçekleştirilmiştir. Çalışmanın gerçekleştirilmesinde köşe noktaların işaretlenmesi amacıyla Harris köşe saptama algoritması, gürültülerin giderilmesi için gauss filtreleme ve morfolojik görüntü işleme teknikleri kullanılmıştır. Gerçekleştirilen çalışmalarda %94 oranında bir başarım elde edilmiştir. Süre ölçüm testlerinde ise ortalama olarak 1.56 sn gibi iyi bir başarım süresine ulaşılmıştır. Süre ölçümleri literatürdeki diğer çalışmalar ile karşılaştırılmıştır. Önerilen yöntemin, süre bakımından oldukça iyi bir performansa sahip olduğu görülmüştür.

Keywords

Altyazı tespiti, Görüntü işleme, Kenar belirleme, Köşe saptama, Morfolojik görüntü işleme.

References

• Elshahaby, H., Rashwan, M. (2022). An end to end system for subtitle text extraction from movie videos. Journal of Ambient Intelligence Humanized Computing, 13, 1853-1865.
• Wang, Y., Wu, M., Zheng, B., Zhu, S. (2022). HuZhouSpeech: A huzhou dialect speech recognition corpus. 5th International Conference on Information Communication and Signal Processing (ICICSP), 153-157, Shenzhen, China.
• Wang, D. (2018). The experimental implementation of grabcut for hardcode subtitle extraction. 17th International Conference on Computer and Information Science (ICIS), 1-4, Singapore.
• Ye, Q., Doermann, D. (2014). Text detection and recognition in imagery: A survey. IEEE Transactions on Pattern Analysis Machine Intelligence, 37 (7), 1480-1500.
• Liu, X. (2008). A camera phone based currency reader for the visually impaired. Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility, 305-306, Canada.
• Huang, M. et. al., (2022). Swintextspotter: Scene text spotting via better synergy between text detection and text recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4593-4603, USA.
• Naiemi, F., Ghods, V., Khalesi, H. (2022). Scene text detection and recognition: a survey. Multimedia Tools Applications, 81 (14), 20255-20290.
• Kim, G., Yokoo, S., Seo, S., Osanai, A., Okamoto, Y., Baek, Y. (2023). On text localization in end-to-end OCR-Free document understanding transformer without text localization supervision. International Conference on Document Analysis and Recognition, 215-232, USA.
• Chaitra, Y., Dinesh, R. (2022). An impact of radon transforms and filtering techniques for text localization in natural scene text images. ICT with Intelligent Applications: Proceedings of ICTIS 2021, 563-573, India.
• Goud, D. S., Vigneshwari, M., Aparna, P., Vijayasekaran, G., Yadav, A. S., Kumar, A. (2022). Text localization and recognition from natural scene images using AI. International Conference on Automation, Computing and Renewable Systems (ICACRS), 1153-1158, India.
• Jayanth, R. M., Kapanaiah, M. (2022). Dominating set based arbitrary oriented bilingual scene text localization. International Journal of Electrical Computer Engineering, 12 (4), 3730-3738.
• Champa, H. (2023). Text localization and recognition in video and images. Journal of Data Acquisition Processing, 38 (2), 3512.
• Wang, P., Da, C., Yao, C. (2022). Multi-granularity prediction for scene text recognition. European Conference on Computer Vision, 339-355, Israel.
• Favorskaya, M.N., Zotin, A.G., Damov, M.V. (2010). Intelligent inpainting system for texture reconstruction in videos with text removal. International Congress on Ultra Modern Telecommunications and Control Systems, 867-874, Moscow, Russia.
• Khodadadi, M., Behrad, A. (2012). Text localization, extraction and inpainting in color images. 20th Iranian Conference on Electrical Engineering (ICEE2012), 1035-1040, Tehran, Iran.
• Neumann, L., Matas, J. (2015). Real-time lexicon-free scene text localization and recognition. IEEE Transactions on Pattern Analysis Machine Intelligence, 38 (9), 1872-1885.
• Koo, H.I., Kim, D.H. (2013). Scene text detection via connected component clustering and nontext filtering. IEEE Transactions on Image Processing, 22 (6), 2296-2305.
• Neumann, L., Matas, J. (2011). A method for text localization and recognition in real-world images. Computer Vision–ACCV 2010: 10th Asian Conference on Computer Vision, 770-783, Queenstown, New Zealand.
• Neumann, L., Matas, J. (2012). Real-time scene text localization and recognition. IEEE Conference on Computer Vision and Pattern Recognition, 3538-3545, USA.
• Zulkeflee, A. N., Yussof, W.N.J.H.W., Umar, R., Ahmad, N., Mohamad, F. S., Man, M., Awalludin, E. A. (2022). Detection of a new crescent moon using the Maximally Stable Extremal Regions (MSER) technique. Astronomy Computing, 41, 100651.
• Tian, Z., Huang, W., He, T., He, P., Qiao, Y. (2016). Detecting text in natural image with connectionist text proposal network. Computer Vision–ECCV 2016: 14th European Conference, 56-72, Netherland.
• He, P., Huang, W., Qiao, Y., Loy, C., Tang, X. (2016). Reading scene text in deep convolutional sequences. Proceedings of the AAAI conference on artificial intelligence, USA.
• Shi, B., Bai, X., Belongie, S. (2017). Detecting oriented text in natural images by linking segments. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2550-2558, USA.
• Zhou, X., Yao, C., Wen, H., Wang, Y., Zhou, S., He, W., Liang, J. (2017). East: an efficient and accurate scene text detector. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5551-5560, USA.
• Kazmi, W., Nabney, I., Vogiatzis, G., Rose, P., Codd, A., (2020). An efficient industrial system for vehicle tyre (tire) detection and text recognition using deep learning. IEEE Transactions on Intelligent Transportation Systems, 22 (2), 1264-1275.
• Hassan, H., El-Mahdy, A., Hussein, M. E. (2021). Arabic scene text recognition in the deep learning era: Analysis on a novel dataset. IEEE Access, 9, 107046-107058.
• Wang, M., Niu, S., Gao, Z. (2019). A novel scene text recognition method based on deep learning. Computers, Materials Continua, 60 (2), 781-794.
• Long, S., He, X., Yao, C. (2021). Scene text detection and recognition: The deep learning era. International Journal of Computer Vision, 129, 161-184.
• Wang, X.-F., He, Z.-H., Wang, K., Wang, Y.-F., Zou, L., Wu, Z.-Z. (2023). A survey of text detection and recognition algorithms based on deep learning technology. Neurocomputing, 556, 126702.
• Wang, X., Jiang, Y., Luo, Z., Liu, C.-L., Choi, H., Kim, S. (2019). Arbitrary shape scene text detection with adaptive text region representation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6449-6458, USA.
• Zhang, C., Liang, B., Huang, Z., En, M., Han, J., Ding, E., Ding, X. (2019). Look more than once: An accurate detector for text of arbitrary shapes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10552-10561, USA.
• Liu, Y., He, T., Chen, H., Wang, X., Luo, C., Zhang, S., ... Jin, L. (2021). Exploring the capacity of an orderless box discretization network for multi-orientation scene text detection. International Journal of Computer Vision, 129, 1972-1992.
• Baek, Y., Lee, B., Han, D., Yun, S., Lee, H. (2019). Character region awareness for text detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9365-9374, USA.
• Wang, W., Xie, E., Song, X., Zang, Y., Wang, W., Lu, T., ... Shen, C. (2019). Efficient and accurate arbitrary-shaped text detection with pixel aggregation network. Proceedings of the IEEE/CVF International Conference on Computer Vision, 8440-8449, Korea.
• Wang, W., Xie, E., Li, X., Hou, W., Lu, T., Yu, G., Shao, S. (2019). Shape robust text detection with progressive scale expansion network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9336-9345, USA.
• Guiming, S. Jidong, S. (2018). Multi-scale Harris corner detection algorithm based on canny edge-detection. IEEE International Conference on Computer and Communication Engineering Technology (CCET), 305-309, China.
• Yao, C., Bai, X., Liu, W., Ma, Y., Tu, Z. (2012). Detecting texts of arbitrary orientations in natural images. 2012 IEEE Conference on Computer Vision and Pattern Recognition, 1083-1090, Providence, RI, USA.
• Shivakumara, P., Phan, T. Q., Tan, C. L. (2010). A laplacian approach to multi-oriented text detection in video. IEEE Transactions on Pattern Analysis Machine Intelligence, 33(2), 412-419.
• Koo, H. I., Kim, D. H. (2013). Scene text detection via connected component clustering and nontext filtering. IEEE Transactions on Image Processing, 22 (6), 2296-2305.
• Yin, X.-C., Yin, X., Huang, K., Hao, H.-W. (2013). Robust text detection in natural scene images. IEEE Transactions on Pattern Analysis Machine Intelligence, 36 (5), 970-983.
• Ye, Q., Doermann, D. (2014). Scene text detection via integrated discrimination of component appearance and consensus. Camera-Based Document Analysis and Recognition: 5th International Workshop, CBDAR 2013, 47-59, Washington, DC, USA.
• Karatzas, D., Shafait, F., Uchida, S., Iwamura, M., i Bigorda, L. G., Mestre, S. R., ... & De Las Heras, L. P. (2013). ICDAR 2013 robust reading competition. 12th International Conference on Document Analysis and Recognition, 1484-1493, Washington, DC, USA.