A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning

Muhammed Kocaoğlu; Yunus Emre Işıkdemir; Muhammed Ali Uzun; Kaya Turgut; Muhammed Oguz Tas; Burak Kaleci

doi:10.5281/zenodo.4589489

EN TR

A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning

Abstract

This study aims to create semantic and metric maps of a post-disaster indoor environment similar to standard the National Institute of Standards and Technology (NIST) search and rescue test arenas that first-responders can easily read. We prefer to use point cloud data acquired with an RGB-D camera since it does not be affected by post-disaster environments’ dusty and dull nature. Besides, each point cloud data is processed separately so that the semantic and metric maps grow incrementally. The Dynamic Graph Convolutional Neural Network (DGCNN) is used to classify points as sematic categories such as walls, terrain, and inclined and straight ramps. RTAB-Map and the semantic map are utilized to generate the octree-based 3D metric map. The experiments are conducted in a simulated environment modelled with Gazebo similar to NIST test arenas to show the effectiveness of the proposed method.

Keywords

Search and Rescue , Mobile Robot , 3D Semantic Map , 3D Metric Map , Point Cloud , Point-Based Deep Learning

Arama Kurtarma Alanlarında Metrik ve Anlamsal Harita Üretmek için Nokta Tabanlı Derin Öğrenme ile Bir Gezgin Robot Uygulaması

Öz

Zehirli madde yayılımı, sel, yangın ve deprem gibi afetlerden sonra bina içi ortamlarda arama ve kurtarma görevleri için robotların kullanılmasına yönelik çalışmalar son yıllarda hız kazanmıştır. Bu çalışmanın ana motivasyonu, ilk yardım ekiplerinin kolaylıkla kullanabileceği afet sonrası bina içi ortamın metrik ve anlamsal haritalarını oluşturmaktır. Bu çalışmada, afet ortamında karşılaşılabilecek toz, duman ve yetersiz ışıklandırma gibi faktörlerden etkilenmeyen ve nesnelerin geometrik yapısını yüksek doğrulukta temsil edebilen nokta bulutu verilerinin kullanılmasına karar verilmiştir. Her bir adımda alınan nokta bulutu ayrı ayrı işlenerek önerilen yöntemin hesaplama karmaşıklığının düşürülmesi amaçlanmıştır. Anlamsal haritanın üretilmesi aşamasında geçmiş çalışmalardan farklı olarak nokta tabanlı derin öğrenme mimarisi DGCNN kullanılmıştır. Böylece nokta bulutunda yer alan her noktanın anlamsal sınıfı (duvar, zemin, eğimli ve düz rampa) belirlenmiştir. 3B metrik haritanın oluşturulması için RTAB-Map ve anlamsal harita birlikte kullanılarak 8-li ağaç yapısında bir gösterim elde edilmiştir. Bu haritada önceki çalışmalardan farklı olarak sadece dolu ve boş vokseller değil, aynı zamanda duvar, zemin ve rampa sınıflarına ait olan vokseller de yer almaktadır. Önerilen yöntemin test edilmesi için Gazebo benzetim ortamında NIST ortamlarına benzer bir test alanı modellenmiş ve bir Pionner 3-AT gezgin robot teleoperasyon yöntemi ile gezdirilmiştir. Test sonuçları önerilen yöntemin başarılı bir şekilde anlamsal ve metrik harita üretebildiğini göstermiştir.

Anahtar Kelimeler

Arama ve Kurtarma , Gezgin Robot , 3B Anlamsal Harita , 3B Metrik Harita , Nokta Bulutu , Nokta Tabanlı Derin Öğrenme

Kaynakça

H. Kitano and S. Tadokoro, “RoboCup Rescue: A Grand Challenge for Multiagent and Intelligent Systems,” AI Magazine, vol. 22, no. 1, pp. 39-52, 2001.
A. Jacoff, E. Messina, B. A. Weiss, S. Tadokoro, and Y. Nakagawa, "Test arenas and performance metrics for urban search and rescue robots," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 2003.
H. L. Akin, N. Ito, A. Jacoff, A. Kleiner, J. Pellenz, and A. Visser, “Robocup rescue robot and simulation leagues”. AI magazine, vol. 34, no. 1, pp. 78-86, 2013.
S. Balakirsky, S. Carpin, A. Kleiner, M. Lewis, A. Visser, J. Wang, and V. A. Ziparo, “Towards heterogeneous robot teams for disaster mitigation: Results and performance metrics from robocup rescue”, Journal of Field Robotics, vol. 24, no. 11, pp. 943-967, 2007.
S. Balakirsky, S. Carpin, and A. Visser, “Evaluating the robocup 2009 virtual robot rescue competition.”, In Proceedings of the 9th Workshop on Performance Metrics for Intelligent Systems, pp. 109-114, 2009.
F. Amigoni, A. Visser, and M. Tsushima, “RoboCup 2012 Rescue Simulation League Winners.”, In Robocup 2012: Robot Soccer World Cup XVI,Lecture Notes in Artificial Intelligence, Berlin, Springer, 2013.
R. Sheh, S. Schwertfeger, A. Visser, “16 Years of RoboCup Rescue,” Künstliche Intelligenz, vol. 30, no.3, pp. 267-277, 2016.
ROS, Available: https://www.ros.org/, Accessed Time: 15.02.2021
GazeboSim, Available: http://gazebosim.org/, Accessed Time: 15.02.2021
K. Ito, K. Yokochi, and Y. Tanaka, “Chukyo Rescue A Team Description Paper for Robocup 2017 Virtual Robot League.”, 2017.

G. Grisetti, C. Stachniss, and W. Burgard, “Improved techniques for grid mapping with rao-blackwellized particle filters.”, IEEE transactions on Robotics, vol. 23 no. 1, pp. 34-46, 2007.
S. Kohlbrecher, O. Von Stryk, J. Meyer, and U. Klingauf, "A flexible and scalable SLAM system with full 3D motion estimation.", IEEE international symposium on safety, security, and rescue robotics, pp.155-160, 2011.
S. Yavuz, M. F. Amasyalı, E. Uslu, F. Cakmak, N. Altuntas, S. Marangoz, M. B. Dilaver, A. E. Kırlı, “YILDIZ Team Description Paper for Rescue Simulation League Virtual Robots Competition 2017”, 2017.
A. Hornung, K. M. Wurm, M. Bennewitz, C. Stachniss, and W. Burgard, "OctoMap: An efficient probabilistic 3D mapping framework based on octrees.", Autonomous robots, vol. 34 no. 3, pp. 189-206, 2013.
S. M. Ahmadi, M. H. G. Nejad, and A. Torabian, “SOS RS Team Description Paper RoboCup 2018 Rescue Virtual Robot League”, 2018.
M. Montemerlo, S. Thrun, D. Koller and B.Wegbreit, “FastSLAM: A Factored Solution to the Simultaneous Localization and Mapping Problem”, Association for the Advancement of Artificial Intelligence (AAAI) Conference, pp. 593-598, 2002.
J. Ziegler, J. Gleichhauf and C. Pfitzner, “RoboCup Rescue 2018 Team Description Paper AutonOHM”, 2018.
S. May, P. Koch, R. Koch, C. Merkl, C. Pfitzner, and A. Nuechter, “A Generalized 2D and 3D Multi-Sensor Data Integration Approach based on Signed Distance Functions for Multi-Modal Robotic Mapping.”, In Vision, Modeling, and Visualization (VMV), pp. 95-102, 2014.
J. H. Kim, X. Lin, N. Kanyok, A. Shaker, P. Poudel, I. Cardenas, N. Karina, C. Cabrera, H. Jeong, and G. P. Sharma, "RoboCup Rescue 2019 TDP Virtual Robot Simulation." 2019.
M. Labbé and F. Michaud, “Memory management for real-time appearance-based loop closure detection.”, IEEE/RSJ international conference on intelligent robots and systems, pp. 1271-1276, 2011.
rtabmap_ros, Available: http://wiki.ros.org/rtabmap_ros. Accessed Time: 15.02.2021.
T. Rabbani, F. A. Van Den Heuvel, G. Vosselman, "Segmentation of point clouds using smoothness constraints," ISPRS commission V symposium: image engineering and vision metrology. International Society for Photogrammetry and Remote Sensing (ISPRS), Dresden, Germany, 2006.
M. A. Fischler, R. C. Bolles, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography," Communications of the ACM, vol. 24, no. 6, pp. 381-395, 1981.
W. Deng, K. Huang, X. Chen, Z. Zhou, C. Shi, R. Guo, and H. Zhang,)., “Semantic RGB-D SLAM for Rescue Robot Navigation.”, IEEE Access, vol. 8, pp.221320-221329, 2020.
K. Turgut and B. Kaleci, "Comparison of Deep Learning Techniques for Semantic Classification of Ramps in Search and Rescue Arenas," 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), Istanbul, Turkey, pp. 1-6, 2020.
C. R. Qi, H. Su, M. Kaichun, L. J. Guibas, ‘‘PointNet: Deep learning on point sets for 3D classification and segmentation,’’ in Proc. IEEE Conf. Comput. Vision. Pattern Recognit. (CVPR), Honolulu, HI, USA, 2017.
C. R. Qi, L. Yi, H. Su, L. J. Guibas, ‘‘PointNet++: Deep hierarchical feature learning on point sets in a metric space,’’ Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA, 2017.
Y. Li, R. Bu, M. Sun, W. Wu, X. Di, B. Chen, “PointCNN: Convolution on x-transformed points,” Conference on Neural Information Processing Systems (NIPS), Montreal, Canada, 2018.
Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, J. M. Solomon, “Dynamic graph CNN for learning on point clouds,” ACM Transactions on Graphics, vol. 38, no. 5, Article 146, 12 pages, 2019.
Point Cloud Library Plane Segmentation Code. Available: https://pointclouds.org/documentation/tutorials/random_sample_consensus.html, Access date: 15.02.2021.
S. Thrun, “Learning Occupancy Grid Maps with Forward Sensor Models”. Autonomous Robots, vol. 15, pp. 111–127, 2003.
A. Vo, L. Truong-Hong, D. F. Laefer, and M. Bertolotto, “Octree-based region growing for point cloud segmentation”, ISPRS Journal of Photogrammetry and Remote Sensing, vol. 104, pp. 88-100, 2015.
Hector_nist_arenas_gazebo, Available: http://wiki.ros.org/hector_nist_arenas_gazebo, Accessed Time: 15.02.2021
teleop_twist_keyboard, Available: http://wiki.ros.org/teleop_twist_keyboard, Accessed Time: 15.02.2021
Tensorflow, Available: https://www.tensorflow.org/, Accessed Time: 15.02.2021
pybind11, Available: https://github.com/pybind/pybind11, Accessed Time: 15.02.2021
octree_viewer, Available: https://github.com/PointCloudLibrary/pcl/blob/master/tools/octree_viewer.cpp, Accessed Time: 15.02.2021
R. B. Rusu, S. Cousins, “3D is here: Point Cloud Library (PCL),” IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, 2011.

Ayrıntılar

Birincil Dil

İngilizce

Konular

Yapay Zeka

Bölüm

Araştırma Makalesi

Yazarlar

Muhammed Kocaoğlu
0000-0003-0956-1210
Türkiye

Yunus Emre Işıkdemir
0000-0001-7022-2854
Türkiye

Muhammed Ali Uzun
0000-0002-2679-3980
Türkiye

Kaya Turgut
0000-0003-3345-9339
Türkiye

Muhammed Oguz Tas
0000-0001-5689-8786
Türkiye

Burak Kaleci ^*
Türkiye

Yayımlanma Tarihi

21 Haziran 2021

Gönderilme Tarihi

16 Şubat 2021

Kabul Tarihi

8 Mart 2021

Yayımlandığı Sayı

Yıl 2021 Cilt: 2 Sayı: 1

DOI

https://doi.org/10.5281/zenodo.4589489

IZ

https://izlik.org/JA44NN23LR

APA

Kocaoğlu, M., Işıkdemir, Y. E., Uzun, M. A., Turgut, K., Tas, M. O., & Kaleci, B. (2021). A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning. Journal of Science, Technology and Engineering Research, 2(1), 11-22. https://doi.org/10.5281/zenodo.4589489

AMA

1.Kocaoğlu M, Işıkdemir YE, Uzun MA, Turgut K, Tas MO, Kaleci B. A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning. Journal of Science, Technology and Engineering Research. 2021;2(1):11-22. doi:10.5281/zenodo.4589489

Chicago

Kocaoğlu, Muhammed, Yunus Emre Işıkdemir, Muhammed Ali Uzun, Kaya Turgut, Muhammed Oguz Tas, ve Burak Kaleci. 2021. “A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning”. Journal of Science, Technology and Engineering Research 2 (1): 11-22. https://doi.org/10.5281/zenodo.4589489.

EndNote

Kocaoğlu M, Işıkdemir YE, Uzun MA, Turgut K, Tas MO, Kaleci B (01 Haziran 2021) A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning. Journal of Science, Technology and Engineering Research 2 1 11–22.

IEEE

[1]M. Kocaoğlu, Y. E. Işıkdemir, M. A. Uzun, K. Turgut, M. O. Tas, ve B. Kaleci, “A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning”, Journal of Science, Technology and Engineering Research, c. 2, sy 1, ss. 11–22, Haz. 2021, doi: 10.5281/zenodo.4589489.

ISNAD

Kocaoğlu, Muhammed - Işıkdemir, Yunus Emre - Uzun, Muhammed Ali - Turgut, Kaya - Tas, Muhammed Oguz - Kaleci, Burak. “A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning”. Journal of Science, Technology and Engineering Research 2/1 (01 Haziran 2021): 11-22. https://doi.org/10.5281/zenodo.4589489.

JAMA

1.Kocaoğlu M, Işıkdemir YE, Uzun MA, Turgut K, Tas MO, Kaleci B. A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning. Journal of Science, Technology and Engineering Research. 2021;2:11–22.

MLA

Kocaoğlu, Muhammed, vd. “A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning”. Journal of Science, Technology and Engineering Research, c. 2, sy 1, Haziran 2021, ss. 11-22, doi:10.5281/zenodo.4589489.

Vancouver

1.Muhammed Kocaoğlu, Yunus Emre Işıkdemir, Muhammed Ali Uzun, Kaya Turgut, Muhammed Oguz Tas, Burak Kaleci. A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning. Journal of Science, Technology and Engineering Research. 01 Haziran 2021;2(1):11-22. doi:10.5281/zenodo.4589489

Dergide yayınlanan çalışmalar

Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0) Uluslararası Lisansı ile lisanslanmıştır.

A Mobile Robot Application for Constructing Semantic and Metric Maps of Search and Rescue Arenas with Point-Based Deep Learning

Abstract

Keywords

Arama Kurtarma Alanlarında Metrik ve Anlamsal Harita Üretmek için Nokta Tabanlı Derin Öğrenme ile Bir Gezgin Robot Uygulaması

Öz

Anahtar Kelimeler

Kaynakça

Ayrıntılar

Birincil Dil

Konular

Bölüm

Yazarlar

Yayımlanma Tarihi

Gönderilme Tarihi

Kabul Tarihi

Yayımlandığı Sayı

DOI

IZ

Kaynak Göster