Evaluation of Deterioration in Brick Materials in Monumental Buildings Using CNN Models

Year 2025, Volume: 38 Issue: 3, 1080 - 1092

Murat Şahin , Seda Arslan Tuncer , Çağla Danacı , Gökhan Genç

https://doi.org/10.35378/gujs.1596450

Abstract

The materials of historical structures undergo deterioration and deformation over time due to physical, chemical, and biological factors. Regular inspections by experts are essential for the preservation of these structures. However, the detection of such deteriorations involves significant labor, time, and cost, and incorrect diagnoses may lead to irreversible damage and structural issues. This study aims to minimize human-induced errors in identifying types of deterioration in structures by utilizing deep learning-based convolutional neural network (CNN) models, a subfield of artificial intelligence. With this study, deteriorations in historical brick buildings, which have an important place among historical buildings, will be detected with non-destructive methods, thus contributing to the preservation of historical buildings and the literature. Within the scope of the study, a total of 1709 data consisting of physical deterioration (cracks and fractures, joint discharge, abrasion and piece loss), chemical deterioration and biological deterioration types that are frequently encountered in brick materials in historical buildings were discussed. The classification process was carried out with the inputs given to the ResNet-18, ResNet-50, ResNet-101, VGG16 and VGG19 networks. Model performances were evaluated with precision, recall and F1 score metrics. The best performance values were obtained with ResNet101 (88% Precision, 88% Recall, 87% F1 Score, 88% Accuracy). Then, using Grad-CAM, the points on which the model focused while making predictions were determined. This study, which will include planning the basic principles of interventions to be applied to cultural property, will prevent the problems encountered in deterioration and deformation, and objective solutions will be produced.

Keywords

Deep Learning, Explainable Artificial Intelligence, Brick Deterioration, Cultural Heritage, Conservation in Historical Buildings

References

• [1] Török, A., Prikryl, R, “Current methods and future trends in test-ing, durability analyses and provenance studies of natural stonesused in historical monuments”, Eng. Geol., 115: 139–142, (2010). DOI: https://doi.org/10.1016/j.enggeo.2010.07.003
• [2] Ince, I., Bozdag, A., Tosunlar, M.B., Hatır, M.E. and Korkanc¸ M, “Determination of deterioration of the main facade of the Ferit Paşa Cistern by non-destructive techniques (Konya, Turkey)”, Environmental Earth Sciences, 77: 420, (2018). DOI: https://doi.org/10.1007/s12665-018-7595-z
• [3] Hatır, M.E., Korkanc¸ M. and Basar, M.E, “Evaluating the deterioration effects of building stones using NDT: the Kücükköy Church, Cappadocia Region, central Turkey”, Bulletin of Engineering Geology and the Environment, 78(5): 3465-3478, (2019). DOI: https://doi.org/10.1007/s10064-018-1339-x
• [4] Mansuri, L.E., Patel, D.A, “Artificial intelligence-based automatic visual inspection system for built heritage”, Smart Sustainable Built Environ, 11(3): 622-646, (2021). DOI: https://www.emerald.com/insight/2046-6099.htm
• [5] McCabe, S., Smith, B., Adamson, C., Mullan, D. and McAllister, D, “The ‘Greening’ of Natural Stone Buildings: Quartz Sandstone Performance as a Secondary Indicator of Climate Change in the British Isles?”, Atmospheric and Climate Sciences, 1(4): 165-171, (2011). DOI: http://www.SciRP.org/journal/acs
• [6] Fitzner, B., Heinrichs, K. and Bouchardiere, D.L, “Weathering damage on Pharaonicsandstone monuments in Luxor-Egypt”, Building and Environment, 38: 1089–1103, (2003). DOI: https://doi.org/10.1016/S0360-1323(03)00086-6
• [7] Thornbush, M.J, “A site-specific index based on weathering formsvisible in central Oxford”, UK, Geoscience, 2: 277–297, (2012). DOI: https://doi.org/10.3390/geosciences2040277
• [8] Gizzi, F.T., Sileo, M., Biscione, M., Danese, M. and Buergo, M, “The conservation state of the Sassi of Matera site (Southern Italy) and its correlation withthe environmental conditions analysed through spatial analysis techniques”, Journal of Cultural Heritage, 1(17): 61–74, (2016). DOI: https://doi.org/10.1016/j.culher.2015.05.002
• [9] Valero, E., Forster, A., Bosché, F., Hyslop, E., Wilson, L. and Turmel, A, “Automated defect detection and classification in ashlar masonry walls using machine learning”, Automation in Construction, 106: 102846, (2019). DOI: https://doi.org/10.1016/j.autcon.2019.102846
• [10] Crespo, C., Armesto, J., González-Aguilera, D. and Arias, P, “Damage detection on his-torical buildings using unsupervised classification techniques, InternationalArchives of Photogrammetry”, Int. Arch. Photogramm. Remote Sens. Spat. Inf.Sci. 38:184–188, (2010). DOI: http://www.riegl.com/uploads/tx_pxpriegldownload
• [11] Laefer, D.F., Gannon, J. and Deely, E, “Reliability of crack detection methods for base-line condition assessments”, Journal of Infrastructure Systems, 16(2): 129–137, (2010). DOI: https://doi.org/10.1061/(ASCE)1076-0342(2010)16:2(129)
• [12] Fais, S., Cuccuru, F., Ligas, P., Casula, G. and Bianchi, M.G, “Integrated ultrasonic, laser scanning and petrographical characterisation of carbonate building materialson an architectural structure of a historic building”, Bulletin of Engineering Geology and the Environment, 76: 71–84, (2017). DOI: https://doi.org/10.1007/s10064-015-0815-9
• [13] Russo, M., Carnevali, L., Russo, V., Savastano, D. and Taddia, Y, “Modeling anddeterioration mapping of façades in historical urban context by close-rangeultra-lightweight UAVs photogrammetry”, Int. J. Archit. Herit., 13: 549–568, (2019). DOI: https://doi.org/10.1080/15583058.2018.1440030
• [14] Korkanc, M., Hüseyinca, M.Y., Hatır, M.E., Tosunlar, M.B., Bozdağ, A., Özen, L. and ˙Ince, ˙I, “Interpreting sulfated crusts on natural building stones using sulfurcontour maps and infrared thermography”, Environ. Earth Sci., 78: 378, (2019). DOI: https://doi.org/10.1007/s12665-019-8377-y
• [15] Quagliarini, E., Clini, P. and Ripanti, M., “Low cost and safe methodology for the assessment of the state of conservation of historical buildings from 3D laser scanning: the case study of Santa Maria in Portonovo (Italy)”, J. Cult. Herit., 24: 175–183, (2017). DOI: https://doi.org/10.1016/j.culher.2016.10.006
• [16] Martínez, J., Corbí, H., Martin-Rojas, I., Baeza-Carratala, J.F. and Giannetti, A, “Stratigraphy, petrophysical characterization and 3D geological modelling of the historical quarry of Nueva Tabarca island (western Mediterranean): implications on heritage conservation”, Eng. Geo., 231: 88–99, (2017). DOI: https://doi.org/10.1016/j.enggeo.2017.10.014
• [17] Ferria, de L., Santagati, C., Catinoto, M., Tesser, E., di San Lio, E.M. and Pojana, G, “A multi-technique characterization study of building materials from the Exedra of S. Nicol`o l’Arena in Catania (Italy)”, J. Build. Eng., 23: 377–387, (2019). DOI: https://doi.org/10.1016/j.jobe.2019.01.028
• [18] Rodriguez-Gonzalvez, P., Gonzalez-Aguilera, D., Lopez-Jimenez, G. and Picon-Cabrera, I, “Image-based modeling of built environment froman unmanned aerial system”, Automat. Constr. 48: 44–52, (2014). DOI: https://doi.org/10.1016/j.autcon.2014.08.010
• [19] Cheng, L., Zhang, X. and Shen, J, “Road surface condition classification using deep learning, J. Vis. Commun. Image Represent”, 64: 102638, (2019). DOI: https://doi.org/10.1016/j.jvcir.2019.102638
• [20] Long, J., Shelhamer, E. and Darrell, T, “Fully convolutional networks for semantic segmentation”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, 3431–3440, (2015). DOI: https://arxiv.org/abs/1411.4038
• [21] Sun, Y., Slun, Z. and Chen, W, “The evolution of object detection methods”, Engineering Applications of Artificial Intelligence, 133(E): 1084582024, (2024). DOI: https://doi.org/10.1016/j.engappai.2024.108458
• [22] Bai, M., Urtasun, R, “Deep watershed transform for instance segmentation”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5221–5229, (2017). DOI: https://doi.org/10.48550/arXiv.1611.08303
• [23] Bassier, M., Vergauwen, M. and Van Genechten, B, “Automated classification of heritage buildings for as-built BIM using machine learning techniques”, Proceedings of ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 25–30, (2017). DOI: https://isprs-annals.copernicus.org/articles/IV-2-W2/25/2017/
• [24] Valero, E., Forster, A., Bosch´e, F., Renier, C., Hyslop, E. and Wilson, L, “High level-of-detail BIM and machine learning for automated masonry wall defect surveying”, Proceedings of the International Symposium on Automation and Robotics in Construction, Berlin, 20–25, (2018). DOI: https://isarc2018.blogs.ruhr-uni-bochum.de/
• [25] Nazarian, E., Taylor, T., Weifeng, T. and Ansari, F, “Machine-learning-based approach for post event assessment of damage in a turn-of-the-century building structure”, J. Civ. Struct. Health Monit., 8(2): 237–251, (2018). DOI: https://doi.org/10.1007/s13349-018-0275-6
• [26] Hatır, M.E., Barstuğan, M. and Ince, I, “Deep learning-based weathering type recognition in historical stone monuments”, J. Cult. Herit., 45: 193-203, (2020). DOI: https://doi.org/10.1016/j.culher.2020.04.008
• [27] Hatır, M.E., Ince, I., “Lithology mapping of stone heritage via state-of-the-art computer vision”, J. Build. Eng., 34: 101921, (2020). DOI: https://doi.org/10.1016/j.jobe.2020.101921
• [28] • Zou, Z., Zhao, X., Zhao, P., Qi, F. and Wang, N, “CNN-based statistics and location estimation of missing components in routine inspection of historic buildings”, J. Cult. Herit., 38: 221–230, (2019). DOI: 10.1016/j.culher.2019.02.002
• [29] Hatır, M.E., İnce, İ. and Korkanç, M, “Intelligent detection of deterioration in cultural stone heritage”, Journal of Building Engineering, 44:102690, (2021). DOI: https://doi.org/10.1016/j.jobe.2021.102690
• [30] Hatır, E., Korkanç, M., Schachner, A. and İnce, İ, “The deep learning method applied to the detection and mapping of stone deterioration in open-air sanctuaries of the Hittite period in Anatolia”, Journal of Cultural Heritage, 51: 37-49, (2021). DOI: https://doi.org/10.1016/j.culher.2021.07.004
• [31] Wang, N., Zhao, X., Zou, Z., Zhao, P. and Qi, F, “Autonomous damage segmentation and measurement of glazed tiles in historic buildings via deep learning”, Comput-Aided Civ. Inf., 35(3): 277–291, (2020). DOI: https://doi.org/10.1016/j.culher.2024.05.009
• [32] https://brickarchitecture.com/about-brick/why-brick/the-history-of-bricks-brickmaking#:~:text=Bricks%20date%20back%20to%207000,in%20the%20sun%20for%20hardening. Access date: 10.11.2024
• [33] Güler, S, “Anadolu Türk Mimarisinde Tuğla ve Koruma-Restorasyon Önerileri”, Yüksek Lisans Tezi, Dokuz Eylül Üniversitesi Fen Bilimleri Enstitüsü, İzmir, (1998).
• [34] https://szerelmey.com/. Access date: 5.10.2024
• [35] Moriconi, G., Castellano, M. G. and Collepardi, M, “Mortar deterioration of the masonry walls in historic buildings. A case history: Vanvitelli's Mole in Ancona”, Materials and Structures, 27: 408-414, (1994). DOI: https://doi.org/10.1007/BF02473445
• [36] Ludwig, U., Mehr, S, “Destruction of historical buildings by the formation of ettringite or thaumasite”, in Proceedings of 8th International Congress on the Chemistry of Cement, Rio de Janeiro, V: 181-188, (1986). DOI: https://scholar.google.com/scholar_lookup?title=Destruction%20of%20historical%20buildings%20by%20formation%20of%20ettringite%20and%20thaumasite&publication_year=1986&author=U%20Ludwig&author=S%20Mehr
• [37] Icomos Sıngapore Urban Redevelopment Authorıty, Conservatıon Technıcal Handbook A Guıde For Best Practıces. https://www.icomos-sg.org/conservation-technical-handbook-a-guide-for-best-practices.html, (2017). DOI: https://www.icomos-sg.org/conservation-technical-handbook-a-guide-for-best-practices.html
• [38] Lynch, G., Roundtree, S, Department of the Environment and Local Government, Bricks: A Guide to the Repair of Historic Brickwork. https://www.scribd.com/document/326989851/A-Guide-to-the-Repair-of-Historic-Brickwork, (2009). DOI: https://www.scribd.com/document/326989851/A-Guide-to-the-Repair-of-Historic-Brickwork
• [39] Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1), 18, (2020). DOI: https://doi.org/10.3390/e23010018
• [40] Simonyan, K., Zisserman, A, “Very Deep Convolutional Networks for Large-Scale Image Recognition”, The 3rd International Conference on Learning Representations, 1409-1556, (2015). DOI: https://doi.org/10.48550/arXiv.1409.1556
• [41] Hasanah, S. A., Pravitasari, A. A., Abdullah, A. S., Yulita, I. N. and Asnawi, M. H, ‘A deep learning review of resnet architecture for lung disease Identification in CXR Image’, Applied Sciences, 13(24): 13111, (2023). DOI: https://doi.org/10.3390/app132413111
• [42] Surinta, O., Khamket, T, “Recognizing pornographic images using deep convolutional neural networks. In 2019 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical”, Electronics, Computer and Telecommunications Engineering, (ECTI DAMT-NCON):150-154, (2019). DOI: 10.1109/ECTI-NCON.2019.8692296
• [43] Kim, J. K., Jung, S., Park, J. and Han, S. W, “Arrhythmia detection model using modified DenseNet for comprehensible Grad-CAM visualization”. Biomedical Signal Processing and Control, 73, 103408, (2022). DOI: https://doi.org/10.1016/j.bspc.2021.103408