Derin öğrenme tabanlı nesne algılama işlemlerinin farklı uygulama alanları

Year 2021, Volume: 4 Issue: 2, 148 - 164, 29.11.2021

Sevcan Turan , Bahar Milani , Feyzullah Temurtaş

https://doi.org/10.51513/jitsa.957371

Cited By: 2

Abstract

Otomasyon, insan yaşamını ve çalışma koşullarını kolaylaştırmak için günlük yaşamda ve iş hayatında yaygınlaşmaktadır. Robotlar, sürücüsüz arabalar, insansız araçlar, robot kollar, otomatik fabrikalar vs. hayatımıza hızla girmektedir. Bu otomatikleştirilmiş aktörler için önemli görevlerden biri, çalışılacak ortamdaki nesneleri ve engelleri tanımaktır. Nesne algılama -nesnelerin cinsinin ve ortamdaki konumlarının belirlenmesi- bu görev için en önemli çözümlerden biridir. Evrişimli Sinir Ağı ve GPU işleme gibi derin öğrenme teknikleri ile nesne algılama işlemleri daha doğru ve hızlı sonuç üretmeye başlamış ve araştırmacıların dikkatini çekmiştir. Son yıllarda nesne algılama algoritmaları ve nesne algılamanın kullanımı ile ilgili birçok makale yayınlanmıştır. Nesne algılama algoritmaları hakkında inceleme makaleleri de bulunmaktadır, ancak genel itibariyle algoritmaları tanıtmış ve çok yaygın olarak bilinen uygulama alanlarına odaklanmışlardır. Diğer inceleme makalelerinden farklı olarak, bu çalışmada nesne algılama algoritmalarının çok geniş ve farklı uygulama alanına sahip olduğu gösterilmek istenmektedir. Çalışmada, derin öğrenmeye kısa bir giriş yapıldıktan sonra derin öğrenmeye dayalı standart nesne algılama algoritmaları ve bunların son yıllarda farklı araştırma alanlarındaki uygulamalarına yer verilerek gelecekteki çalışmalar için rehber olmak amaçlanmaktadır. Ayrıca makalelerde kullanılan veri setleri ve değerlendirme ölçütleri de listelenmiştir.

Keywords

Nesne algılama, Derin öğrenme, CNN, LSTM

DIFFERENT APPLICATION AREAS OF OBJECT DETECTION WITH DEEP LEARNING

Abstract

Automation is spread in all daily life and business activities to facilitate human life and working conditions. Robots, automated cars, unmanned vehicles, robot arms, automated factories etc. are getting place in our lives. For these automated actors, one important task is recognizing objects and obstacles in the target environment. Object detection, determining the objects and their location in the environment, is one of the most important solution for this task. With deep learning techniques like Convolutional Neural Network and GPU processing, object detection has become more accurate and faster, and getting attention of researchers. In recent years, many articles about object detection algorithms and usage of object detection have been published. There are surveys about the object detection algorithms, but they have introduced algorithms and focused on common application areas. With this survey, we aim to show that object detection algorithms have very large and different application area. In this study, we have given a brief introduction to deep learning. We have then focused on standard object detection algorithms based on deep learning and their applications in different research areas in recent years to give an idea for future works. Also, the datasets and evaluation metrics used in the research are listed.

Keywords

Object detection, deep learning, CNN, LSTM

References

• Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., & Bengio, Y. (2016). End-to-end attention-based large vocabulary speech recognition. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4945–4949. https://doi.org/10.1109/ICASSP.2016.7472618
• Caicedo, J. C., Goodman, A., Karhohs, K. W., Cimini, B. A., Ackerman, J., Haghighi, M., Heng, C. K., Becker, T., Doan, M., McQuin, C., Rohban, M., Singh, S., & Carpenter, A. E. (2019). Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nature Methods. https://doi.org/10.1038/s41592-019-0612-7
• Cevikalp, H., Benligiray, B., & Gerek, O. N. (2020). Semi-supervised robust deep neural networks for multi-label image classification. Pattern Recognition, 100, 107164. https://doi.org/https://doi.org/10.1016/j.patcog.2019.107164
• Chen, J.-W., Lin, W.-J., Cheng, H.-J., Hung, C.-L., Lin, C.-Y., & Chen, S.-P. (2021). A Smartphone-Based Application for Scale Pest Detection Using Multiple-Object Detection Methods. Electronics, 10(4), 372. https://doi.org/10.3390/electronics10040372
• Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., & Li, H. (2020). Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection. ArXiv:2012.15712.
• Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14(2), 179–211. https://doi.org/https://doi.org/10.1016/0364-0213(90)90002-E
• Face Masks. (2020). https://www.kaggle.com/andrewmvd/face-mask-detection
• Fan, D. P., Ji, G. P., Sun, G., Cheng, M. M., Shen, J., & Shao, L. (2020). Camouflaged object detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR42600.2020.00285
• Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. 2012 IEEE Conference on Computer Vision and Pattern Recognition, 3354–3361. https://doi.org/10.1109/CVPR.2012.6248074
• Gers, F. A., Schmidhuber, J., & Cummins, F. (1999). Learning to forget: continual prediction with LSTM. 1999 Ninth International Conference on Artificial Neural Networks ICANN 99. (Conf. Publ. No. 470), 2, 850–855 vol.2. https://doi.org/10.1049/cp:19991218
• Girshick, R, Donahue, J., Darrell, T., & Malik, J. (2016). Region-Based Convolutional Networks for Accurate Object Detection and Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1), 142–158. https://doi.org/10.1109/TPAMI.2015.2437384
• Girshick, Ross. (2015). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV).
• Hacıefendioğlu, K., Başağa, H. B., & Demir, G. (2021). Automatic detection of earthquake-induced ground failure effects through Faster R-CNN deep learning-based object detection using satellite images. Natural Hazards, 105(1), 383–403. https://doi.org/10.1007/s11069-020-04315-y
• Han, F., Yao, J., Zhu, H., & Wang, C. (2020). Underwater Image Processing and Object Detection Based on Deep CNN Method. Journal of Sensors, 2020, 1–20. https://doi.org/10.1155/2020/6707328
• Han, W., Zhang, Z., Caine, B., Yang, B., Sprunk, C., Alsharif, O., Ngiam, J., Vasudevan, V., Shlens, J., & Chen, Z. (2020). Streaming Object Detection for 3-D Point Clouds. http://arxiv.org/abs/2005.01864
• Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation. https://doi.org/10.1162/neco.1997.9.8.1735
• Houben, S., Stallkamp, J., Salmen, J., Schlipsing, M., & Igel, C. (2013). Detection of traffic signs in real-world images: The German traffic sign detection benchmark. The 2013 International Joint Conference on Neural Networks (IJCNN), 1–8. https://doi.org/10.1109/IJCNN.2013.6706807
• Hung, J., Goodman, A., Ravel, D., Lopes, S. C. P., Rangel, G. W., Nery, O. A., Malleret, B., Nosten, F., Lacerda, M. V. G., Ferreira, M. U., Rénia, L., Duraisingh, M. T., Costa, F. T. M., Marti, M., & Carpenter, A. E. (2020). Keras R-CNN: library for cell detection in biological images using deep neural networks. BMC Bioinformatics, 21(1), 300. https://doi.org/10.1186/s12859-020-03635-x
• Indolia, S., Goswami, A. K., Mishra, S. P., & Asopa, P. (2018). Conceptual Understanding of Convolutional Neural Network- A Deep Learning Approach. Procedia Computer Science. https://doi.org/10.1016/j.procs.2018.05.069
• Jelodar, H., Wang, Y., Orji, R., & Huang, H. (2020). Deep Sentiment Classification and Topic Discovery on Novel Coronavirus or COVID-19 Online Discussions: NLP Using LSTM Recurrent Neural Network Approach. ArXiv:2004.11695.
• Jing, R., Liu, S., Gong, Z., Wang, Z., Guan, H., Gautam, A., & Zhao, W. (2020). Object-based change detection for VHR remote sensing images based on a Trisiamese-LSTM. International Journal of Remote Sensing, 41(16), 6209–6231. https://doi.org/10.1080/01431161.2020.1734253
• Kechaou, A., Martinez, M., Haurilet, M., & Stiefelhagen, R. (2020). Detective: An Attentive Recurrent Model for Sparse Object Detection. http://arxiv.org/abs/2004.12197
• Kingma, D. P., & Ba, J. (2014). Adam: A Method for Stochastic Optimization. http://arxiv.org/abs/1412.6980
• Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L. J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L. (2017). Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations. International Journal of Computer Vision. https://doi.org/10.1007/s11263-016-0981-7
• Kristo, M., Ivasic-Kos, M., & Pobar, M. (2020). Thermal Object Detection in Difficult Weather Conditions Using YOLO. IEEE Access, 8, 125459–125476. https://doi.org/10.1109/ACCESS.2020.3007481
• Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM, 60(6), 84–90. https://doi.org/10.1145/3065386
• Law, H., & Deng, J. (2018). CornerNet: Detecting Objects as Paired Keypoints. Proceedings of the European Conference on Computer Vision (ECCV), 734–750.
• Lawrence, S., Giles, C. L., & Fong, S. (2000). Natural language grammatical inference with recurrent neural networks. IEEE Transactions on Knowledge and Data Engineering, 12(1), 126–140. https://doi.org/10.1109/69.842255
• Le Cun, Y., Jackel, L. D., Boser, B., Denker, J. S., Graf, H. P., Guyon, I., Henderson, D., Howard, R. E., & Hubbard, W. (1989). Handwritten digit recognition: applications of neural network chips and automatic learning. IEEE Communications Magazine, 27(11), 41–46. https://doi.org/10.1109/35.41400
• Le, T.-N., Nguyen, T. V, Nie, Z., Tran, M.-T., & Sugimoto, A. (2019). Anabranch network for camouflaged object segmentation. Computer Vision and Image Understanding, 184, 45–56. https://doi.org/https://doi.org/10.1016/j.cviu.2019.04.006
• LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539 Lee, D.-H. (2020). CNN-based single object detection and tracking in videos and its application to drone detection. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-020-09924-0
• Li, G., & Yu, Y. (2016). Visual Saliency Detection Based on Multiscale Deep CNN Features. IEEE Transactions on Image Processing, 25(11), 5012–5024. https://doi.org/10.1109/TIP.2016.2602079
• Li, J., Zhao, R., Sun, E., Wong, J. H. M., Das, A., Meng, Z., & Gong, Y. (2020). High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 7699–7703. https://doi.org/10.1109/ICASSP40776.2020.9054387
• Li, R., Liu, H., Wang, X., & Qian, Y. (2018). DSBI: Double-Sided Braille Image Dataset and Algorithm Evaluation for Braille Dots Detection. Proceedings of the 2018 the 2nd International Conference on Video and Image Processing.
• Li, Y., Hou, X., Koch, C., Rehg, J. M., & Yuille, A. L. (2014). The Secrets of Salient Object Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition, 280–287. https://doi.org/10.1109/CVPR.2014.43
• Liu, Z., Li, Q., & Li, W. (2020). Deep layer guided network for salient object detection. Neurocomputing, 372, 55–63. https://doi.org/https://doi.org/10.1016/j.neucom.2019.09.018
• Ljosa, V., Sokolnicki, K. L., & Carpenter, A. E. (2012). Annotated high-throughput microscopy image sets for validation. In Nature Methods. https://doi.org/10.1038/nmeth.2083
• Loey, M., Manogaran, G., Taha, M. H. N., & Khalifa, N. E. M. (2020). Fighting against COVID-19: A novel deep learning model based on YOLO-v2 with ResNet-50 for medical face mask detection. Sustainable Cities and Society. https://doi.org/10.1016/j.scs.2020.102600
• Medical Mask. (n.d.). Retrieved January 17, 2021, from https://humansintheloop.org/medical-mask-dataset/
• Møgelmose, A., Trivedi, M. M., & Moeslund, T. B. (2012). Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2012.2209421
• Ovodov, I. G. (2020). Optical Braille Recognition Using Object Detection CNN. ArXiv:2012.12412.
• Öztemel, E. (2012). Yapay Sinir Ağları (3rd ed.). Papatya Yayıncılık Eğitim A.Ş.
• Pascalvoc. (n.d.). Retrieved January 11, 2020, from http://host.robots.ox.ac.uk/pascal/VOC/
• Pavan, G. S., Kumar, N., Karthik N., K., & Manikandan, J. (2020). Design of a Real-Time Speech Recognition System using CNN for Consumer Electronics. 2020 Zooming Innovation in Consumer Technologies Conference (ZINC), 5–10. https://doi.org/10.1109/ZINC50678.2020.9161432
• Pi, Y., Nath, N. D., & Behzadan, A. H. (2020). Convolutional neural networks for object detection in aerial imagery for disaster response and recovery. Advanced Engineering Informatics, 43, 101009. https://doi.org/https://doi.org/10.1016/j.aei.2019.101009
• Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779–788.
• Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
• Sardoğan, M., Özen, Y., & Tuncer, A. (2020). Faster R-CNN Kullanarak Elma Yaprağı Hastalıklarının Tespiti. Düzce Üniversitesi Bilim ve Teknoloji Dergisi, 1110–1117. https://doi.org/10.29130/dubited.648387
• Skurowski, P., Abdulameer, H., Błaszczyk, J., Depta, T., Kornacki, A., & Kozieł, P. (2017). CHAMELEON. http://kgwisc.aei.polsl.pl/index.php/en/dataset/63-animal-camouflage-analysis
• Sorano, D., Carrara, F., Cintia, P., Falchi, F., & Pappalardo, L. (2020). Automatic Pass Annotation from Soccer VideoStreams Based on Object Detection and LSTM. http://arxiv.org/abs/2007.06475
• Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., Vasudevan, V., Han, W., Ngiam, J., Zhao, H., Timofeev, A., Ettinger, S., Krivokon, M., Gao, A., Joshi, A., … Anguelov, D. (2020). Scalability in Perception for Autonomous Driving: Waymo Open Dataset. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2443–2451. https://doi.org/10.1109/CVPR42600.2020.00252
• Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR.2016.308
• Tassinari, P., Bovo, M., Benni, S., Franzoni, S., Poggi, M., Mammi, L. M. E., Mattoccia, S., Di Stefano, L., Bonora, F., Barbaresi, A., Santolini, E., & Torreggiani, D. (2021). A computer vision approach based on deep learning for the detection of dairy cows in free stall barn. Computers and Electronics in Agriculture, 182, 106030. https://doi.org/10.1016/j.compag.2021.106030
• Tsai, W.-J., Huang, Z.-J., & Chung, C.-E. (2020). Joint Detection, Re-Identification, And Lstm In Multi-Object Tracking. 2020 IEEE International Conference on Multimedia and Expo (ICME), 1–6. https://doi.org/10.1109/ICME46284.2020.9102884
• Turan, S., & Bilgin, G. (2019). Semantic nuclei segmentation with deep learning on breast pathology images. 2019 Scientific Meeting on Electrical-Electronics Biomedical Engineering and Computer Science (EBBT), 1–4. https://doi.org/10.1109/EBBT.2019.8741715
• Ufldl Tutorial. (n.d.). UFLDL Tutorial. Retrieved December 24, 2020, from http://ufldl.stanford.edu/tutorial/
• Vinayakumar, R., Soman, K. P., & Poornachandran, P. (2017). Applying convolutional neural network for network intrusion detection. 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 1222–1228. https://doi.org/10.1109/ICACCI.2017.8126009
• Wang, L., Lu, H., Wang, Y., Feng, M., Wang, D., Yin, B., & Ruan, X. (2017). Learning to Detect Salient Objects with Image-Level Supervision. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3796–3805. https://doi.org/10.1109/CVPR.2017.404
• Wu, A., Zhang, Q., Fang, W., Deng, H., Jiang, S., & Liu, Q. (2020). Mask R-CNN Based Object Detection for Intelligent Wireless Power Transfer. http://arxiv.org/abs/2004.10021
• Xiao, H., Xu, J., & Shi, J. (2020). Exploring diverse and fine-grained caption for video by incorporating convolutional architecture into LSTM-based model. Pattern Recognition Letters, 129, 173–180. https://doi.org/https://doi.org/10.1016/j.patrec.2019.11.003
• Yan, Q., Xu, L., Shi, J., & Jia, J. (2013). Hierarchical Saliency Detection. CVPR 2013.
• Yang, C., Zhang, L., Lu, H., Ruan, X., & Yang, M.-H. (2013). Saliency detection via graph-based manifold ranking. Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference On, 3166–3173.
• Yuan, D., Li, X., He, Z., Liu, Q., & Lu, S. (2020). Visual object tracking with adaptive structural convolutional network.
• Knowledge-Based Systems, 194, 105554. https://doi.org/https://doi.org/10.1016/j.knosys.2020.105554 Zhang, J., Huang, M., Jin, X., & Li, X. (2017). A real-time Chinese traffic sign detection algorithm based on modified YOLOv2. Algorithms. https://doi.org/10.3390/a10040127
• Zhang, J., Xie, Z., Sun, J., Zou, X., & Wang, J. (2020). A Cascaded R-CNN with Multiscale Attention and Imbalanced Samples for Traffic Sign Detection. IEEE Access. https://doi.org/10.1109/ACCESS.2020.2972338
• Zhang, W., Xu, L., Li, Z., Lu, Q., & Liu, Y. (2016). A Deep-Intelligence Framework for Online Video Processing. IEEE Software, 33(2), 44–51. https://doi.org/10.1109/MS.2016.31
• Zhao, D., Chang, Z., & Guo, S. (2020). Cross-scale fusion detection with global attribute for dense captioning. Neurocomputing. https://doi.org/10.1016/j.neucom.2019.09.055
• Zhu, J., Guo, Y., Yue, F., Yuan, H., Yang, A., Wang, X., & Rong, M. (2020). A deep learning method to detect foreign objects for inspecting power transmission lines. IEEE Access. https://doi.org/10.1109/ACCESS.2020.2995608