Deprem sonrasında nokta bulutlarından hasarlı binaların belirlenmesine yönelik yapay öğrenme modeli tasarımı: ‘EQDamage4B_Pnt’

Öz

Deprem sonrasında afet bölgesine ilişkin havadan elde edilmiş üç boyutlu (3B) nokta bulutlarından mevcut durumun 3B kent modelinin yani 3B sahnenin üretilmesi ve bunun üzerinde 3B bölütleme gibi bilgisayarlı görü görevlerinin gerçekleştirilmesi önemli bir konudur. Buradan hareketle deprem sonrasında elde edilen büyük ve karmaşık nokta bulutlarının 3B bölütlenmesi görevinde hem yapay öğrenme modelinin temsil gücünü hem de hesaplama verimliliğini arttıracak bir mimarinin tasarlanması ve bu mimariye dayalı bir modelin geliştirilmesi kritik öneme sahiptir. Ayrıca yapay öğrenme alanında da kullanılabilecek en iyi mimariden ve modelden ziyade, projenin amacına ve gereksinimlerine en uygun mimariyi seçebilmek ve buna ilişkin modeli geliştirebilmek çok daha önemli görülmektedir. Yapay öğrenme yaklaşımında; araştırma ve uygulama gereksinimlerine göre modelin ölçeklenebilirliği, ön eğitim yaklaşımları ve bütünleştirme kolaylığı gibi ölçütler mimari ve model seçiminde belirleyici olmaktadır. Çalışma kapsamında deprem sonrasında düzensiz ve sırasız yapıda bulunan nokta bulutlarından bina hasar tespiti için yüksek detay ve çok ölçekli bilgi gereksinimleri göz önüne alındığında SONATA öz denetim ön eğiticisinin, PointTransformer v3 özellik çıkarıcısının, Mask3D bölütleme başlığının, PointLLM etkileşiminin, Point2Surf grafik modelleyicisinin bir araya getirildiği bir süreç tasarımı geliştirilmiş ve EQDamage4B_Pnt olarak adlandırılmıştır. Süreç tasarımı sonucunda elde edilen mimari ile bina hasar tespiti, deprem sonrası hızlı hasar değerlendirmesi ve müdahale planlaması gibi görevlere yönelik çözümler uçtan-uca yapay öğrenme tabanlı olarak üretilebilir yapıya getirilmiş olacaktır.

Anahtar Kelimeler

• Deprem
• Hasar belirleme
• Nokta bulutu
• LiDAR
• Segmentasyon
• Derin öğrenme

Design of a machine learning model for identifying damaged buildings from post-earthquake point clouds: ‘EQDamage4B_Pnt’

Öz

After an earthquake, it is important to produce a 3D city model, i.e., a 3D scene, from 3D point clouds obtained from the sky regarding the disaster area and to perform computer vision tasks such as 3D segmentation on it. Based on this, designing an architecture that enhances both the representational power of the artificial intelligence model and computational efficiency, and developing a model based on this architecture, is of critical importance in the task of 3D segmentation of the large and complex point clouds obtained after the earthquake. Furthermore, rather than selecting the best architecture and model available in the field of artificial intelligence, it is considered much more important to choose the architecture that best suits the project's objectives and requirements and to develop the corresponding model. In the machine learning approach, criteria such as model scalability, pre-training approaches, and ease of integration are decisive in architectural and model selection, depending on research and application requirements. Considering the high-detail and multi-scale information requirements for building damage detection from point clouds with irregular and disordered structures after an earthquake, a process design combining the SONATA self-supervised pre-trainer, PointTransformer v3 feature extractor, Mask3D segmentation head, PointLLM interaction, and Point2Surf graphical modeler have been combined to develop a process design named EQDamage4B_Pnt. The resulting architecture enables end-to-end machine learning-based solutions for tasks such as building damage detection, post-earthquake rapid damage assessment, and intervention planning.

Anahtar Kelimeler

• Earthquake
• Building damage assessment
• Point cloud
• LiDAR
• Segmentation
• Deep learning

Kaynakça

1. Achlioptas, P., Diamanti, O., Mitliagkas, I., & Guibas, L. (2017). Learning Representations and Generative Models for 3D Point Clouds. 35th International Conference on Machine Learning, ICML 2018, 1, 67-85. https://arxiv.org/pdf/1707.02392.
2. Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2023). Deep learning for visual localization and mapping: A survey. IEEE Transactions on Neural Networks and Learning Systems, 35(12), 17000-17020.
3. Chen, C., Wu, Y., Dai, Q., Zhou, H.-Y., Xu, M., Yang, S., Han, X., & Yu, Y. (2024). A survey on graph neural networks and graph transformers in computer vision: A task-oriented perspective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 10297-10318.
4. Dai, A. and C., Angel X. and Savva, Manolis and Halber, Maciej and Funkhouser, & Thomas and Niessner, M. (2017). ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. http://www.scan-net.org/.
5. Duan, Y., Zheng, Y., Lu, J., Zhou, J., & Tian, Q. (2019, Haziran). Structural relational reasoning of point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition içinde (s. 949-958). Long Beach, California, ABD.
6. Frome, A., Huber, D., Kolluri, R., Bülow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 3023, 224-237.
7. Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023, Haziran). ImageBind: One Embedding Space To Bind Them All. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition içinde (s. 15180-15190). Vancouver, Kanada.
8. Graham, B., Engelcke, M., & van der Maaten, L. (2018, Haziran). 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks. CVPR 2018 içinde (s. 9224-9232). Salt Lake City, Utah, ABD.

9. Goodfellow, I., Bengio, Y., Courville, A., & Bengio, Y. (2016). Deep learning, 1(2). Cambridge: MIT press. Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R. R., & Hu, S.-M. (2020). PCT: Point cloud transformer. Computational Visual Media, 7(2), 187-199.
10. Guo, Z., Zhang, R., Zhu, X., Tang, Y., Ma, X., Han, J., Chen, K., Gao, P., Li, X., Li, H., & Heng, P. A. (2023). Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understanding, generation, and instruction following. arXiv preprint arXiv:2309.00615.
11. Gupta, R., Hosfelt, R., Sajeev, S., Patel, N., Goodman, B., Doshi, J., Heim, E., Choset, H., & Gaston, M. (2019). xbd: A dataset for assessing building damage from satellite imagery. arXiv preprint arXiv:1911.09296.
12. Hackel, T., Savinov, N., Ladicky, L., Wegner, J. D., Schindler, K., & Pollefeys, M. (2017). Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 4(1W1), 91-98.
13. He, Y., Yu, H., Liu, X., Yang, Z., Sun, W., Anwar, S., & Mian, A. (2021). Deep learning based 3D segmentation: A survey. arXiv preprint arXiv:2103.05423.
14. Hoang, L., Lee, S. H., Kwon, O. H., & Kwon, K. R. (2019). A deep learning method for 3D object classification using the wave kernel signature and a center point of the 3D-triangle mesh. Electronics, 8(10), 1196.
15. Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., & Stuetzle, W. (1992, Temmuz). Surface reconstruction from unorganized points. Proceedings of the 19th annual conference on computer graphics and interactive techniques içinde (s. 71-78). Los Angeles, California, ABD.
16. Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., & Markham, A. (2020a, Haziran). RandLA-Net: Efficient semantic segmentation of large-scale point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition içinde (s. 11108–11117).
17. Hu, Q., Yang, B., Khalid, S., Xiao, W., Trigoni, N., & Markham, A. (2020b, Haziran). Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition içinde (s. 4975-4985).
18. Hu, Q., Yang, B., Khalid, S., Xiao, W., Trigoni, N., & Markham, A. (2022). SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds. International Journal of Computer Vision, 130(2), 316-343.
19. Johnson, A. E., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433-449.
20. Khoshelham, K., & Elberink, S. O. (2012). Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors, 12(2), 1437-1454.
21. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90.
22. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2323.
23. Li, J., Chen, B. M., & Lee, G. H. (2018, Haziran). SO-Net: Self-Organizing Network for Point Cloud Analysis. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition içinde (s. 9397-9406). Salt Lake City, Utah, ABD.
24. Li, Y., Bu, R., Sun, M., Wu, W., Di, X., & Chen, B. (2018). PointCNN: Convolution on X-transformed points. Advances in Neural Information Processing Systems (NeurIPS), 31, 820–830.
25. Lin, L., Liu, Y., Hu, Y., Yan, X., Xie, K., & Huang, H. (2021). Urbanscene3d: A large scale urban scene dataset and simulator. arXiv preprint arXiv:2107.04286, 2(3).
26. Lin, Z., He, L., Yang, H., Sun, X., Zhang, G., Chen, W., Guan, Y., & Zhang, H. (2024, Ekim). SWCF-Net: similarity-weighted convolution and local-global fusion for efficient large-scale point cloud semantic segmentation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) içinde (s. 2602-2609). Abu Dhabi, BAE.
27. Liu, W., Sun, J., Li, W., Hu, T., & Wang, P. (2019). Deep learning on point clouds and its application: A survey. Sensors, 19(19), 4188.
28. Lu, D., Xie, Q., Wei, M., Gao, K., Xu, L., & Li, J. (2022, Haziran). Transformers in 3D Point Clouds: A Survey. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition içinde (s. 5560-5568). New Orleans, Louisiana, ABD.
29. ManyCore Research Team. (2025). SpatialLM: Large Language Model for Spatial Understanding. https://manycore-research.github.io/SpatialLM/.
30. Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017a, Temmuz). PointNet: Deep learning on point sets for 3D classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) içinde (s. 652–660). Honolulu, Hawaii, ABD.
31. Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017b). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30.
32. Quan, S., Yu, J., Nie, Z., Wang, M., Feng, S., An, P., & Yang, J. (2024). Deep learning for 3d point cloud enhancement: A survey. arXiv preprint arXiv:2411.00857.
33. Rusu, R. B., Blodow, N., & Beetz, M. (2009, Mayıs). Fast point feature histograms (FPFH) for 3D registration. 2009 IEEE international conference on robotics and automation içinde (s. 3212-3217). Kobe, Japonya.
34. Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic routing between capsules. Advances in neural information processing systems, 30.
35. Schult, J., Engelmann, F., Hermans, A., Litany, O., Tang, S., & Leibe, B. (2023, Mayıs). Mask3D: Mask Transformer for 3D Semantic Instance Segmentation. Proceedings - IEEE International Conference on Robotics and Automation içinde (s. 8216-8223). Londra, Birleşik Krallık.
36. Tang, H., Liu, Z., Zhao, S., Lin, Y., Lin, J., Wang, H., & Han, S. (2020, Ağustos). Searching efficient 3d architectures with sparse point-voxel convolution. European conference on computer vision içinde (s. 685-702).
37. Thomas, H., Qi, C. R., Deschaud, J. E., Marcotegui, B., Goulette, F., & Guibas, L. (2019, Ekim). KPConv: Flexible and Deformable Convolution for Point Clouds. Proceedings of the IEEE International Conference on Computer Vision içinde (s. 6410-6419). Seul, Kore.
38. Tombari, F., Salti, S., & Di Stefano, L. (2010). Unique signatures of histograms for local surface description. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6313 LNCS(PART 3), 356-369.
39. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is All You Need. Advances in Neural Information Processing Systems (NeurIPS 2017), 30.
40. Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2018). Dynamic Graph CNN for Learning on Point Clouds. ACM Transactions on Graphics, 38(5), 13.
41. Wehr, A., & Lohr, U. (1999). Airborne laser scanning—an introduction and overview. ISPRS Journal of Photogrammetry and Remote Sensing, 54(2-3), 68-82.
42. Wijaya, K. T., Paek, D. H., & Kong, S. H. (2024). Advanced feature learning on point clouds using multi-resolution features and learnable pooling. Remote Sensing, 16(11), 1835.
43. Wu, X., Lao, Y., Jiang, L., Liu, X., & Zhao, H. (2022). Point Transformer V2: Grouped Vector Attention and Partition-based Pooling. Advances in Neural Information Processing Systems, 35. https://arxiv.org/pdf/2210.05666.
44. Wu, X., DeTone, D., Frost, D., Shen, T., Xie, C., Yang, N., Engel, J., Newcombe, R., Zhao, H., & Straub, J. (2025). Sonata: Self-Supervised Learning of Reliable Point Representations. https://arxiv.org/pdf/2503.16429.
45. Wu, Y., Gu, Y., Chen, X., Liu, H., Zhuang, Y., & Ma, Y. (2023, Haziran). Change3D: A dataset and benchmark for 3D change detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) içinde (s. 15254–15264). Vancouver, Kanada.
46. Wu, Y. Q., Chen, H. X., & Zhang, Y. (2024). Review of 3D point cloud processing methods based on deep learning. Chinese Journal of Lasers, 51(5), 0509001.
47. Xu, R., Wang, X., Wang, T., Chen, Y., Pang, J., & Lin, D. (2023). PointLLM: Empowering Large Language Models to Understand Point Clouds. https://arxiv.org/pdf/2308.16911.
48. Yu, X., Tang, L., Rao, Y., Huang, T., Zhou, J., & Lu, J. (2022, Haziran). Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition içinde (s. 19291-19300). New Orleans, Louisiana, ABD.
49. Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., & Smola, A. J. (2017). Deep sets. Advances in neural information processing systems, 30.
50. Zhao, H., Jiang, L., Fu, C. W., & Jia, J. (2019, Haziran). Pointweb: Enhancing local neighborhood features for point cloud processing. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition içinde (s. 5560-5568). Long Beach, California, ABD.
51. Zhao, H., Jiang, L., Jia, J., Torr, P. H. S., & Koltun, V. (2021, Haziran). Point Transformer. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition içinde (s. 16259-16268). Montreal, Québec, Kanada.
52. Zhao, Y., Birdal, T., Deng, H., & Tombari, F. (2019, Haziran). 3D Point-Capsule Networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June, içinde (s. 1009-1018). Long Beach, California, ABD.
53. Zhou, X., Qi, X., & Xu, Y. (2022). 3DSiamese: A Siamese network for 3D change detection in point clouds. arXiv. https://arxiv.org/abs/2203.15227.
54. URL-1: LASer (LAS) File Format Exchange Activities – ASPRS. https://www.asprs.org/divisions-committees/lidar-division/laser-las-file-format-exchange-activities (Erişim Tarihi: 07 Mayıs 2025).
55. URL-2: Point Cloud Library 0.0 documentation. https://pcl.readthedocs.io/projects/tutorials/en/latest/normal_estimation.html (Erişim Tarihi: 08 Mayıs 2025).
56. URL-3: Yenilikçi Mekansal Veri Üretim Yöntemleri — 2 | by Caner Guney | Medium. https://canerguney.medium.com/yenilik%C3%A7i-mekansal-veri-%C3%BCretim-y%C3%B6ntemleri-2-b35885215fe (Erişim Tarihi: 07 Mayıs 2025).
57. URL-4: Nielsen, M. Neural Networks and Deep Learning. http://neuralnetworksanddeeplearning.com (Erişim Tarihi: 07 Mayıs 2025).
58. URL-5: xView – GitHub. https://github.com/DIUx-xView (Erişim Tarihi: 10 Nisan 2025).
59. URL-6: Princeton ModelNet. https://modelnet.cs.princeton.edu/ (Erişim Tarihi: 07 Mayıs 2025).
60. URL-7: ShapeNet. https://shapenet.org/ (Erişim Tarihi: 08 Mayıs 2025).
61. URL-8: Semantic3D. https://www.semantic3d.net/ (Erişim Tarihi: 07 Mayıs 2025).
62. URL-9: VCC. https://vcc.tech/UrbanScene3D (Erişim Tarihi: 07 Mayıs 2025).