Evaluation of Deterioration in Brick Materials in Monumental Buildings Using CNN Models

Abstract

The materials of historical structures undergo deterioration and deformation over time due to physical, chemical, and biological factors. Regular inspections by experts are essential for the preservation of these structures. However, the detection of such deteriorations involves significant labor, time, and cost, and incorrect diagnoses may lead to irreversible damage and structural issues. This study aims to minimize human-induced errors in identifying types of deterioration in structures by utilizing deep learning-based convolutional neural network (CNN) models, a subfield of artificial intelligence. With this study, deteriorations in historical brick buildings, which have an important place among historical buildings, will be detected with non-destructive methods, thus contributing to the preservation of historical buildings and the literature. Within the scope of the study, a total of 1709 data consisting of physical deterioration (cracks and fractures, joint discharge, abrasion and piece loss), chemical deterioration and biological deterioration types that are frequently encountered in brick materials in historical buildings were discussed. The classification process was carried out with the inputs given to the ResNet-18, ResNet-50, ResNet-101, VGG16 and VGG19 networks. Model performances were evaluated with precision, recall and F1 score metrics. The best performance values were obtained with ResNet101 (88% Precision, 88% Recall, 87% F1 Score, 88% Accuracy). Then, using Grad-CAM, the points on which the model focused while making predictions were determined. This study, which will include planning the basic principles of interventions to be applied to cultural property, will prevent the problems encountered in deterioration and deformation, and objective solutions will be produced.

Keywords

• Deep Learning
• Explainable Artificial Intelligence
• Brick Deterioration
• Cultural Heritage
• Conservation in Historical Buildings

References

1. [1] Török, A., Prikryl, R, “Current methods and future trends in test-ing, durability analyses and provenance studies of natural stonesused in historical monuments”, Eng. Geol., 115: 139–142, (2010). DOI: https://doi.org/10.1016/j.enggeo.2010.07.003
2. [2] Ince, I., Bozdag, A., Tosunlar, M.B., Hatır, M.E. and Korkanc¸ M, “Determination of deterioration of the main facade of the Ferit Paşa Cistern by non-destructive techniques (Konya, Turkey)”, Environmental Earth Sciences, 77: 420, (2018). DOI: https://doi.org/10.1007/s12665-018-7595-z
3. [3] Hatır, M.E., Korkanc¸ M. and Basar, M.E, “Evaluating the deterioration effects of building stones using NDT: the Kücükköy Church, Cappadocia Region, central Turkey”, Bulletin of Engineering Geology and the Environment, 78(5): 3465-3478, (2019). DOI: https://doi.org/10.1007/s10064-018-1339-x
4. [4] Mansuri, L.E., Patel, D.A, “Artificial intelligence-based automatic visual inspection system for built heritage”, Smart Sustainable Built Environ, 11(3): 622-646, (2021). DOI: https://www.emerald.com/insight/2046-6099.htm
5. [5] McCabe, S., Smith, B., Adamson, C., Mullan, D. and McAllister, D, “The ‘Greening’ of Natural Stone Buildings: Quartz Sandstone Performance as a Secondary Indicator of Climate Change in the British Isles?”, Atmospheric and Climate Sciences, 1(4): 165-171, (2011). DOI: http://www.SciRP.org/journal/acs
6. [6] Fitzner, B., Heinrichs, K. and Bouchardiere, D.L, “Weathering damage on Pharaonicsandstone monuments in Luxor-Egypt”, Building and Environment, 38: 1089–1103, (2003). DOI: https://doi.org/10.1016/S0360-1323(03)00086-6
7. [7] Thornbush, M.J, “A site-specific index based on weathering formsvisible in central Oxford”, UK, Geoscience, 2: 277–297, (2012). DOI: https://doi.org/10.3390/geosciences2040277
8. [8] Gizzi, F.T., Sileo, M., Biscione, M., Danese, M. and Buergo, M, “The conservation state of the Sassi of Matera site (Southern Italy) and its correlation withthe environmental conditions analysed through spatial analysis techniques”, Journal of Cultural Heritage, 1(17): 61–74, (2016). DOI: https://doi.org/10.1016/j.culher.2015.05.002

9. [9] Valero, E., Forster, A., Bosché, F., Hyslop, E., Wilson, L. and Turmel, A, “Automated defect detection and classification in ashlar masonry walls using machine learning”, Automation in Construction, 106: 102846, (2019). DOI: https://doi.org/10.1016/j.autcon.2019.102846
10. [10] Crespo, C., Armesto, J., González-Aguilera, D. and Arias, P, “Damage detection on his-torical buildings using unsupervised classification techniques, InternationalArchives of Photogrammetry”, Int. Arch. Photogramm. Remote Sens. Spat. Inf.Sci. 38:184–188, (2010). DOI: http://www.riegl.com/uploads/tx_pxpriegldownload
11. [11] Laefer, D.F., Gannon, J. and Deely, E, “Reliability of crack detection methods for base-line condition assessments”, Journal of Infrastructure Systems, 16(2): 129–137, (2010). DOI: https://doi.org/10.1061/(ASCE)1076-0342(2010)16:2(129)
12. [12] Fais, S., Cuccuru, F., Ligas, P., Casula, G. and Bianchi, M.G, “Integrated ultrasonic, laser scanning and petrographical characterisation of carbonate building materialson an architectural structure of a historic building”, Bulletin of Engineering Geology and the Environment, 76: 71–84, (2017). DOI: https://doi.org/10.1007/s10064-015-0815-9
13. [13] Russo, M., Carnevali, L., Russo, V., Savastano, D. and Taddia, Y, “Modeling anddeterioration mapping of façades in historical urban context by close-rangeultra-lightweight UAVs photogrammetry”, Int. J. Archit. Herit., 13: 549–568, (2019). DOI: https://doi.org/10.1080/15583058.2018.1440030
14. [14] Korkanc, M., Hüseyinca, M.Y., Hatır, M.E., Tosunlar, M.B., Bozdağ, A., Özen, L. and ˙Ince, ˙I, “Interpreting sulfated crusts on natural building stones using sulfurcontour maps and infrared thermography”, Environ. Earth Sci., 78: 378, (2019). DOI: https://doi.org/10.1007/s12665-019-8377-y
15. [15] Quagliarini, E., Clini, P. and Ripanti, M., “Low cost and safe methodology for the assessment of the state of conservation of historical buildings from 3D laser scanning: the case study of Santa Maria in Portonovo (Italy)”, J. Cult. Herit., 24: 175–183, (2017). DOI: https://doi.org/10.1016/j.culher.2016.10.006
16. [16] Martínez, J., Corbí, H., Martin-Rojas, I., Baeza-Carratala, J.F. and Giannetti, A, “Stratigraphy, petrophysical characterization and 3D geological modelling of the historical quarry of Nueva Tabarca island (western Mediterranean): implications on heritage conservation”, Eng. Geo., 231: 88–99, (2017). DOI: https://doi.org/10.1016/j.enggeo.2017.10.014
17. [17] Ferria, de L., Santagati, C., Catinoto, M., Tesser, E., di San Lio, E.M. and Pojana, G, “A multi-technique characterization study of building materials from the Exedra of S. Nicol`o l’Arena in Catania (Italy)”, J. Build. Eng., 23: 377–387, (2019). DOI: https://doi.org/10.1016/j.jobe.2019.01.028
18. [18] Rodriguez-Gonzalvez, P., Gonzalez-Aguilera, D., Lopez-Jimenez, G. and Picon-Cabrera, I, “Image-based modeling of built environment froman unmanned aerial system”, Automat. Constr. 48: 44–52, (2014). DOI: https://doi.org/10.1016/j.autcon.2014.08.010
19. [19] Cheng, L., Zhang, X. and Shen, J, “Road surface condition classification using deep learning, J. Vis. Commun. Image Represent”, 64: 102638, (2019). DOI: https://doi.org/10.1016/j.jvcir.2019.102638
20. [20] Long, J., Shelhamer, E. and Darrell, T, “Fully convolutional networks for semantic segmentation”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, 3431–3440, (2015). DOI: https://arxiv.org/abs/1411.4038
21. [21] Sun, Y., Slun, Z. and Chen, W, “The evolution of object detection methods”, Engineering Applications of Artificial Intelligence, 133(E): 1084582024, (2024). DOI: https://doi.org/10.1016/j.engappai.2024.108458
22. [22] Bai, M., Urtasun, R, “Deep watershed transform for instance segmentation”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5221–5229, (2017). DOI: https://doi.org/10.48550/arXiv.1611.08303
23. [23] Bassier, M., Vergauwen, M. and Van Genechten, B, “Automated classification of heritage buildings for as-built BIM using machine learning techniques”, Proceedings of ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 25–30, (2017). DOI: https://isprs-annals.copernicus.org/articles/IV-2-W2/25/2017/
24. [24] Valero, E., Forster, A., Bosch´e, F., Renier, C., Hyslop, E. and Wilson, L, “High level-of-detail BIM and machine learning for automated masonry wall defect surveying”, Proceedings of the International Symposium on Automation and Robotics in Construction, Berlin, 20–25, (2018). DOI: https://isarc2018.blogs.ruhr-uni-bochum.de/
25. [25] Nazarian, E., Taylor, T., Weifeng, T. and Ansari, F, “Machine-learning-based approach for post event assessment of damage in a turn-of-the-century building structure”, J. Civ. Struct. Health Monit., 8(2): 237–251, (2018). DOI: https://doi.org/10.1007/s13349-018-0275-6
26. [26] Hatır, M.E., Barstuğan, M. and Ince, I, “Deep learning-based weathering type recognition in historical stone monuments”, J. Cult. Herit., 45: 193-203, (2020). DOI: https://doi.org/10.1016/j.culher.2020.04.008
27. [27] Hatır, M.E., Ince, I., “Lithology mapping of stone heritage via state-of-the-art computer vision”, J. Build. Eng., 34: 101921, (2020). DOI: https://doi.org/10.1016/j.jobe.2020.101921
28. [28] • Zou, Z., Zhao, X., Zhao, P., Qi, F. and Wang, N, “CNN-based statistics and location estimation of missing components in routine inspection of historic buildings”, J. Cult. Herit., 38: 221–230, (2019). DOI: 10.1016/j.culher.2019.02.002
29. [29] Hatır, M.E., İnce, İ. and Korkanç, M, “Intelligent detection of deterioration in cultural stone heritage”, Journal of Building Engineering, 44:102690, (2021). DOI: https://doi.org/10.1016/j.jobe.2021.102690
30. [30] Hatır, E., Korkanç, M., Schachner, A. and İnce, İ, “The deep learning method applied to the detection and mapping of stone deterioration in open-air sanctuaries of the Hittite period in Anatolia”, Journal of Cultural Heritage, 51: 37-49, (2021). DOI: https://doi.org/10.1016/j.culher.2021.07.004
31. [31] Wang, N., Zhao, X., Zou, Z., Zhao, P. and Qi, F, “Autonomous damage segmentation and measurement of glazed tiles in historic buildings via deep learning”, Comput-Aided Civ. Inf., 35(3): 277–291, (2020). DOI: https://doi.org/10.1016/j.culher.2024.05.009
32. [32] https://brickarchitecture.com/about-brick/why-brick/the-history-of-bricks-brickmaking#:~:text=Bricks%20date%20back%20to%207000,in%20the%20sun%20for%20hardening. Access date: 10.11.2024
33. [33] Güler, S, “Anadolu Türk Mimarisinde Tuğla ve Koruma-Restorasyon Önerileri”, Yüksek Lisans Tezi, Dokuz Eylül Üniversitesi Fen Bilimleri Enstitüsü, İzmir, (1998).
34. [34] https://szerelmey.com/. Access date: 5.10.2024
35. [35] Moriconi, G., Castellano, M. G. and Collepardi, M, “Mortar deterioration of the masonry walls in historic buildings. A case history: Vanvitelli's Mole in Ancona”, Materials and Structures, 27: 408-414, (1994). DOI: https://doi.org/10.1007/BF02473445
36. [36] Ludwig, U., Mehr, S, “Destruction of historical buildings by the formation of ettringite or thaumasite”, in Proceedings of 8th International Congress on the Chemistry of Cement, Rio de Janeiro, V: 181-188, (1986). DOI: https://scholar.google.com/scholar_lookup?title=Destruction%20of%20historical%20buildings%20by%20formation%20of%20ettringite%20and%20thaumasite&publication_year=1986&author=U%20Ludwig&author=S%20Mehr
37. [37] Icomos Sıngapore Urban Redevelopment Authorıty, Conservatıon Technıcal Handbook A Guıde For Best Practıces. https://www.icomos-sg.org/conservation-technical-handbook-a-guide-for-best-practices.html, (2017). DOI: https://www.icomos-sg.org/conservation-technical-handbook-a-guide-for-best-practices.html
38. [38] Lynch, G., Roundtree, S, Department of the Environment and Local Government, Bricks: A Guide to the Repair of Historic Brickwork. https://www.scribd.com/document/326989851/A-Guide-to-the-Repair-of-Historic-Brickwork, (2009). DOI: https://www.scribd.com/document/326989851/A-Guide-to-the-Repair-of-Historic-Brickwork
39. [39] Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1), 18, (2020). DOI: https://doi.org/10.3390/e23010018
40. [40] Simonyan, K., Zisserman, A, “Very Deep Convolutional Networks for Large-Scale Image Recognition”, The 3rd International Conference on Learning Representations, 1409-1556, (2015). DOI: https://doi.org/10.48550/arXiv.1409.1556
41. [41] Hasanah, S. A., Pravitasari, A. A., Abdullah, A. S., Yulita, I. N. and Asnawi, M. H, ‘A deep learning review of resnet architecture for lung disease Identification in CXR Image’, Applied Sciences, 13(24): 13111, (2023). DOI: https://doi.org/10.3390/app132413111
42. [42] Surinta, O., Khamket, T, “Recognizing pornographic images using deep convolutional neural networks. In 2019 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical”, Electronics, Computer and Telecommunications Engineering, (ECTI DAMT-NCON):150-154, (2019). DOI: 10.1109/ECTI-NCON.2019.8692296
43. [43] Kim, J. K., Jung, S., Park, J. and Han, S. W, “Arrhythmia detection model using modified DenseNet for comprehensible Grad-CAM visualization”. Biomedical Signal Processing and Control, 73, 103408, (2022). DOI: https://doi.org/10.1016/j.bspc.2021.103408